Child development with the D-score: turning milestones into measurement

1.1 First 1000 days

The first 1000 days refers to the time needed for a child to grow from conception to the second birthday. It is a time of rapid change. During this period the architecture of the developing brain is very open to the influence of relationships and experiences (Shonkhoff et al., 2016). Early experiences affect the nature and quality of the brain’s developing architecture by reinforcing some synapses and pruning others through lack of use. The first 1000 days shape the brain’s architecture, but higher-order brain functions continue to develop into adolescence and early adulthood (Kolb et al., 2017).

The classic nature versus nurture debate contrasts the viewpoints that variation in development is primarily due to either genetic or environmental differences. The current scientific consensus is that both genetic predisposition and ecological differences influence all traits (Rutter, 2007). The environment in which a child develops (before and soon after birth) provides experiences that can modify gene activity (Caspi et al., 2010). Negative influences, such as exposure to stressful life circumstances or environmental toxins may leave a chemical signature on the genes, thereby influencing how genes work in that individual.

During the first 1000 days, infants are highly dependent on their caregivers to protect them from adversities and to help them regulate their physiology and behavior. As Figure 1.1 illustrates, caregivers can do this through responsive care, including routines for sleeping and feeding. To reach their developmental potential, children require nutrition, responsive caregiving, opportunities to explore and learn, and protection from environmental threats (Black et al., 2017). Gradually, children build self-regulatory skills that enable them to manage stress as they interact with the world around them (Johnson et al., 2013).

Figure 1.1. Serve and return interactions shape brain architecture.

Source: Shutterstock, under license.

1.2 Relevance of child development

The first 1000 days is a time of rapid change. Early experiences affect brain development and influence lifelong learning and health (Shonkhoff et al., 2016). Healthy development is associated with future school achievement, well-being, and success in life (Bellman et al., 2013).

Professionals and parents consider it important to monitor children’s development. Tracking child development enables professionals to identify children with signs of potential delay. Timely identification can help children and their parents to benefit from early intervention. In a normal population, developmental delay affects about 1–3% of children. A delay in development may indicate underlying disorders. About 1% of children have an autism spectrum disorder (Baird et al., 2006), 1–2% a mild learning disability, and 5–10% have a specific learning disability in a single domain (Horridge, 2011).

Children develop at different rates, and it is vital to distinguish those who are within the “normal” range from those who are following a more pathological course (Bellman et al., 2013). There is good evidence that early identification and early intervention improve the outcomes of children (Britto et al., 2017). Early intervention is crucial for children with developmental disabilities because barriers to healthy development early in life impede progress at each subsequent stage.

Monitoring child development provides caregivers and parents with reliable information about the child and an opportunity to intervene at an early age. Understanding the developmental health of populations of children allows organisations and policymakers to make informed decisions about programmes that support children’s greatest needs (Bellman et al., 2013).

1.3 Stunting as proxy for child development

Stunting is the impaired physical growth and development that children experience from poor nutrition, repeated infection, and inadequate psychosocial stimulation. Linear growth in children is commonly expressed as length-for-age or height-for-age in comparison to normative growth standards (Wit et al., 2017). According to the World Health Organization (WHO), children are stunted if their height-for-age is more than two standard deviations below the Child Growth Standards median. Stunting caused by chronic nutritional deprivation in early childhood is as an indicator of child development (Perkins et al., 2017).

There is consistent evidence for an association between stunting and poor child development, despite heterogeneity in the estimation of its magnitude (Miller et al., 2016; Sudfeld et al., 2015). Considering impaired linear growth as a proxy measure for child development is easy to do, and quite common. Yet, using impaired height growth as a measure for child development is not without limitations:

•    The relation between height and child development is weak after adjustment for age;

•    Height is a physical indicator that does not take into account a direct evaluation of a child’s cognitive or mental performance;

•    There is considerable heterogeneity in heights of children all over the world;

•    Height is not sensitive to rapid changes in child development.

1.4 Measuring neurocognitive development

Assessment of early neurocognitive development in children is challenging for many reasons (Ellingsen, 2016). During the first years of life, developmental change occurs rapidly, and the manifestation of different skills and abilities varies considerably across children. Moreover, a child’s performance on a cognitive task is very susceptible to measurement setting, timing and the health of the child that day.

Recently, a toolkit was published that reviews 147 assessment tools developed for children ages 0–8 years in low- and middle-income countries (Fernald et al., 2017). Some of the most widely used tools include the Ages & Stages Questionnaires (ASQ), Achenbach Child Behavior Checklist (CBCL), Bayley Scales of Infant Development (BSID), Denver Developmental Screening Test (DEN), Griffiths Scales of Child Development (GRF), Mullen Scale of Early Learning (MSEL), Strengths and Difficulties Questionnaire (SDQ), Wechsler Intelligence Scale for Children (WISC), and its younger age counterpart Wechsler Preschool and Primary Scale of Intelligence (WPPSI).

Each of these tools has its strengths and limitations. For example, the ASQ and DEN are screeners for general child development. The CBCL and SDQ are screeners for behavioral and mental health, not cognition or general development. DEN is relatively easy and quick to administer, but not very precise. It is out of production, not being sold or re-normed. The BSID, MSEL, and GRF provide a clinical assessment at the individual level and requires a skilled professional to administer. Some instruments collect observations through the caregiver (ASQ), whereas others emphasize traits and behavior over performance (SDQ, CBCL). Also, the age ranges to which the instruments are sensitive vary. Furthermore, they may cover different domains of development.

The ideal child development assessment would be easy to administer and has high reliability, validity, and cross-cultural appropriateness. It should also show appropriate sensitivity in scores at different ages and ability levels. It is no surprise that no test can meet all of these criteria. Many tests are too long, difficult to administer, lack cross-cultural validity, or have low reliability. Also, many instruments are proprietary and costly to use.

1.5 Why this chapter?

We believe that there cannot be one instrument for measuring child development that is suitable for all situations. In general, the tool needs tailoring to the setting. For example, to find a delayed child, we need an instrument that is precise for that individual child, and that is sensitive to different domains of delay. In contrast, if we want to estimate the proportion of children that is developmentally on track in a region, we need one culturally unbiased, relatively imprecise low-cost measurement made on many children across many ages. The optimal instrument will look quite different in both cases.

We also believe that there can be one scale for measuring child development and that this scale is useful for many applications. Such a scale is similar to well-known measures for body height, body weight or body temperature. These measurements have a clearly defined unit (i.e., centimetre, kilogram, degree Celsius), which moreover is assumed to be constant across all scale locations. We express measurements as the number of scale units (e.g. 92 cm). Note that there may be multiple instruments for measuring a child height (e.g. ruler, laser distance meter, echolocation, ability to reach the door handle, and so on). Still, their result translates into scale units (cm here). The opposite is also true, and perhaps more familiar. We may have one instrument and express the result in multiple units (e.g. cm, inches, light-years).

Instruments and scales are different things. Currently, instruments for measuring child development define their own scales, which renders the measurements made by distinct tools incomparable. No measurement unit for child development yet exists. It would undoubtedly be an advance if we could tailor the measurement instrument to the setting while retaining the advantage of a scale with a clearly defined unit across different tools. We can then compare the data collected by distinct devices. This chapter explores the theory and practice for making that happen.

1.6 Intended audience

We aim for three broad audiences:

•    Professionals in the field of child growth and development;

•    Policymakers in international settings;

•    Statisticians, methodologists, and data scientists.

Professionals in child development will become familiar with a new approach to measuring child development in early childhood. We plan to separate the measurement instrument from the scale used to express the result. This formulation allows the user to select the instrument most suited for a particular setting. Since instruments differ widely in age coverage, length, administration mode, and domain coverage (Boggs et al., 2019), the ability to choose the instrument, while not giving up comparability, represents a significant advance over routines that marry the scale to the instrument.

Policymakers in international settings wish to know the effect of different interventions on child development. Gaining insight into such effects is not so easy since different studies use different instruments. The ability to place measurements made by different instruments onto the same scale will allow for a more accurate understanding of policy effects. It also enables the setting of priorities and actions that are less dependent on the way the data were collected.

Statisticians and data scientists generally prefer numeric values with an unambiguous unit (e.g., centimeters, kilograms) over a vector of dichotomous data points. This chapter shows how to convert a series of PASS/FAIL scores to a numeric value with interval scale properties. The existence of such a scale opens the way for the application of precise analytic techniques, similar to those applied to child height and body weight. The techniques have a solid psychometric backing, and also apply to other types of problems.

2.1 What is child development?

In contrast to concepts like height or temperature, it is unclear what exactly constitutes child development. Shirley (1931) executed one of the first rigorous studies in the field with the explicit aim

        that the many aspects of development, anatomical, physical, motor, intellectual, and emotional, be studied simultaneously.

Shirley gave empirical definitions of each of these domains of development.

Certain domains advance through a fixed sequence. Figure 2.1 illustrates the various stages needed for going from a fetal posture to walking alone. The ages are indicative of when these events happen, but there is a considerable variation in timing between infants.

Figure 2.1. Gross motor development as a sequence of milestones.

Source: Shirley (1933), with permission.

Gesell (1943) (p. 88) formulated the following definition of development:

        Development is a continuous process that proceeds stage by stage in an orderly sequence.

Gesell’s definition emphasizes that development is a continuous process. The stages are useful as indicators to infer the level of maturity but are of limited interest by themselves.

Liebert et al. (1974) (p. 5) emphasized that development is not a phenomenon that unfolds in isolation.

        Development refers to a process in growth and capability over time, as a function of both maturation and interaction with the environment.

Cameron & Bogin (2012) (p. 11) defined an endpoint of development, as follows:

        “Growth” is defined as an increase in size, while “maturity” or “development” is an increase in functional ability…The endpoint of maturity is when a human is functionally able to procreate successfully … not just biological maturity but also behavioural and perhaps social maturity.

Berk (2011) (p. 30) presented a dynamic systems perspective on child development as follows:

        Development cannot be characterized as a single line of change, and is more like a web of fibres branching out in many directions, each representing a different skill area that may undergo both continuous and stagewise transformation.

There are many more definitions of child development. The ones described here illustrate the main points of view in the field.

2.2 Theories of child development

The field of child development is vast and spans multiple academic disciplines. This short overview, therefore, cannot do justice to the enormous richness. Readers new to the field might orient themselves by browsing through an introductory academic titles (Berk, 2011; Santrock, 2011), or by searching for the topic of interest in an encyclopedia, e.g., Salkind (2002).

The introductions by Santrock (2011) and Berk (2011) both distinguish major theories in child development according to how each answer to following three questions:

2.2.1 Continuous or discontinuous? Does development evolve gradually as a continuous process or are there qualitatively distinct stages, with jumps occurring from one step to another?

Many stage-based theories of human development have been proposed over the years: social and emotional development by psycho-sexual stages introduced by Freud and furthered by Erikson (Erikson, 1963), Kohlberg’s six stages of moral development (Kohlberg, 1984) and Piaget’s cognitive development theory (Piaget & Inhelder, 1969). Piaget distinguishes four main periods throughout childhood. The first period, the sensorimotor period (approximately 0–2 years), is subdivided into six stages. When taken together, these six stages describe “the road to conceptual thought.” Piaget’s stages are qualitatively different and aim to unravel the mechanism involved in intellectual development.

On the other hand, Gesell and others emphasize development as a continuous process. Gesell (1943) (p. 88) says:

        A stage represents a degree or level of maturity in the cycle of development. A stage is simply a passing moment, while development, like time, keeps marching on.

2.2.2 One course or multiple parallel tracks? Stage theorists assume that children progress sequentially through the same set of stages. This assumption is also explicit in the work of Gesell.

The ecological and dynamic systems theories view development as continuous, though not necessarily progressing in an orderly fashion, so there may be multiple, parallel ways to reach the same point. The developmental path taken by a given child will depend on the child’s unique combination of personal and environmental circumstances, including cultural diversity in development.

2.2.3 Nature or nurture? Figure 2.2 illustrates that children vary in appearance. Are genetic or environmental factors more important for influencing development? Most theories generally acknowledge the role of both but differ in emphasis. In practice, the debate centres on the question of how to explain individual differences.

Figure 2.2. A group of culturally diverse children.

Source: Shutterstock, under license.

Maturation is the process of becoming fully developed, much like the natural unfolding of a flower. The process depends on both genetic factors (species, breed) as well as environmental influences (sunlight, water, nutrition). Some theorists emphasize that differences in child development are innate and stable over time, although there may be differences in unfolding speed due to different environments. Others argue that environmental factors drive differences in development between children, and changing these factors could very well impact child development.

Our position in this debate has practical implications. If we believe that differences are natural and stable, then it may not make much sense trying to change the environment, as the impact on development is likely to be small. On the other hand, we may consider developmental potential as evenly distributed, with its expression governed by the environment. In the latter case, improving life circumstances may have substantial pay-offs in terms of better development.

2.3 Example of motor development

2.3.1 Shirley’s motor data. For illustration, we use data on loco-motor development from a classic study on child development among 25 babies. Shirley (1931) collected measurements of the baby’s walking ability, starting at ages around 13 weeks, in an ingenious way. The investigator lays out a white paper of twelve inches wide on the floor of the living room, and lightly greases the soles of the baby’s feet with olive oil. The baby was invited to “walk” on the sheet. Of course, very young infants need substantial assistance. Footprints left were later coloured by graphite and measured. Measurements during the first year were repeated every week or bi-weekly.

Table 2.1 (Shirley, 1931, Appendix 8) lists the age (in weeks) of the 21 babies when they started, respectively, stepping, standing, walking with help, and walking alone. Blanks indicate missing data. A blank in the first column means that the baby was already stepping when the observation started (Virginia Ruth, Sibyl, Donovan, Torey and Doris). Max and Martin, who have blanks in the second column, skipped standing and went directly from stepping to walking with help. Doris has a blank in the last column because she passed away before she could walk alone.

2.3.2 Individual trajectories of motor development. Figure 2.3 is a visual representation of the information in Table 2.1. Each data point is the age of the first occurrence of the next stage. Before that age, we assume the baby is in the previous stage.

Figure 2.3. Staircase plot indicating the age at which each baby achieves a new milestone of gross-motor functioning.

Figure 2.3 makes it easy to spot the quick walkers (Martin, Carol) and slow walkers (Patricia, Torey, Larry). Furthermore, we may also locate children who remain a long time in a particular stage (Torey, Larry) or who jump over stages (Martin, Max).

For ease of plotting, the categories on the vertical axis are equally spaced. The height of the jump from one stage to the next has no sensible interpretation. We might be inclined to think that the vertical distance portrays to how difficult it is to achieve the next stage, but this is inaccurate. Instead, the ability needed to set the next step corresponds to the horizontal line length between stages. For example, on average, the line for stepping is rather short in all plots, so going from stepping to standing is relatively easy.

Figure 2.3 presents data from only those visits where a jump occurred. The number of house visits made during the ages of 0–2 years was far higher. Shirley (1931) collected data from 1370 visits, whereas Figure 2.3 plot only the 76 occasions that showed a jump. Thus the data collection needs to be intense and costly to obtain individual curves. Fortunately, there are alternatives that are much more efficient.

2.4 Typical questions asked in child development

The emotional, social and physical development of the young child has a direct effect on the adult he or she will become. We may be interested in measuring child development for answering clinical, policy or public health questions.

Table 2.2 lists typical questions whose answers require measuring child development. Note that all questions compare the amount of child development between groups or time points. A few questions compare development for the same child, group or population at different ages. Others compare development at the same age across different children, groups or populations.

3.1 Age-based measurement of development

3.1.1 Motivation for age-based measurement. Milestones form the based building blocks for instruments to measure child development. Methods to quantify growth using separate milestones relate the milestone behaviour to the child’s age. Gesell (1943) (p. 89) formulated this goal as follows:

        We think of behaviour in terms of age, and we think of age in terms of behaviour. For any selected age it is possible to sketch a portrait which delineates the behaviour characteristics typical of the age.

There is an extensive literature that quantifies development in terms of the ages at which the child is expected to show a specific behaviour. The oldest methods for quantifying child development calculate an age equivalent for achieving a milestone, and compare the child’s age to this age equivalent.

3.1.2 Age equivalent and developmental age. Figure 3.1 graphs the ages at which each of the 21 children enter a given stage in Shirley’s motor data of Table 2.1. Since standing follows stepping, children who can stand are older than the children who are stepping. Hence the ages for standing are located more to the right.

Figure 3.1. Ages at which 21 children achieve four motor development milestones.

Since age and development are so intimately related, we can express the difficulty of a milestone as the mean age at which children achieve it. For example, Stott (1967) (p. 25) defines the age equivalent and its use for measurement, as follows:

        The age equivalent of a particular stage is simply the average age at which children reach that particular stage.

Figure 3.2 adds the mean age and the boxplot at which the children enter the four stages. The difficulty of these milestones can thus be expressed as age equivalents: 16.1 weeks for stepping, 27.2 weeks for standing, 43.3 weeks for walking with help and 63.3 weeks for walking alone.

Figure 3.2. Mean (symbol x) and spread of the ages at which 21 children achieve four motor development milestones.

Thus, a child that is stepping beyond the age of 16.1 weeks is considered later than average, whereas a child already stepping before 27.2 weeks earlier than average. We may also calculate age delta as the difference between the child’s age and the norm age, and express it as “two weeks late” or “three weeks ahead.” Summarizing age delta’s over different milestones has led to concepts like developmental age as a measure of a child’s development.

3.1.3 Limitations of age-based measurement. Age-based measurement is easy to understand, and widely used in the popular press, but not without pitfalls:

1.   Age-based measurement requires us to know the ages at which the child entered a new stage. The mean age can be a biased estimate of item difficulty if visits are widely apart, irregular or missing.

2.   Age-based measurement can inform us whether a child is achieving a given milestone early of late. However, it does not tell us what behaviours are characteristic for children of a given age.

3.   Age-based measurement cannot exist without an age norm. When there are no norms, we cannot quantify development.

4.   Age-based measurement works only at the item level. Although we may average age delta’s over milestones, the choice of milestones is arbitrary.

3.2 Probability-based measurement

An alternative is to calculate the probability of achieving a milestone at a given age and compare the child’s response to that probability.

The passing probability is an interpretable and relevant measure. An operational advantage of the approach is that the necessary calculations place fewer demands on the available data and can be done even for cross-sectional studies.

3.2.1 Example of probability-based measurement. Figure 3.3 plots the percentage of children achieving each of Shirley’s motor stages against age. There are four cumulative curves, one for each milestone, that indicate the percentage of children that pass.

Figure 3.3. Probability of achieving four motor milestones against age.

In analogy to the age equivalent introduced in Section 3.1.2 we can define the difficulty of the milestone as the age at which 50 per cent of the children pass. In the Figure we see that the levels of difficulty are approximately 14.2 weeks (stepping), 27.0 weeks (standing), 43.8 weeks (walking with help) and 64.0 weeks (walking alone). Also, we may easily find the ages at which 10 per cent or 90 per cent of the children pass each milestone.

Observe there is a gradual decline in the steepness as we move from stepping to walk_alone. For example, we need an age interval of 13 weeks (33 - 20) to go from 10 to 90 per cent in standing, but need 19 weeks (71 - 52) to go from 10 to 90 per cent in walking alone. Thus, one step on the age axis corresponds to different increments in probability. The flattening pattern is typical for child development and represents evidence that evolution is faster at earlier ages.

3.2.2 Limitations of probability-based measurement. Probability-based measurement is a popular way to create instruments for screening on developmental delay. For example, each milestone in the Denver II (Frankenburg et al., 1992) has markers for the 25th, 50th, 75th and 90th age percentile.

1.   The same age step corresponds to different probabilities.

2.   The measurement cannot exist without some norm population. When norms differ, we cannot compare the measurements.

3.   Interpretation is at the milestone level, sometimes supplemented by procedures for counting the number of delays. No aggregate takes all responses into account.

3.3 Score-based measurement of development

3.3.1 Motivation for score-based measurement. Score-based measurement takes the responses on multiple milestones and counts the total number of items passed as a measure of development. This approach takes all answers into account, hence leading to a more stable result.

One may order milestones in difficulty, and skip those that are too easy, and stop administration for those that are too difficult. In such cases, we cannot merely interpret the sum score of a measure of development. Instead, we need to correct for the subset of administered milestones. The usual working assumption is that the child would have passed all easier milestones and failed on all more difficult ones. We may repeat this procedure for different domains, e.g. motor, cognitive, and so on.

3.3.2 Example of score-based measurement. Figure 3.4 is a gross-motor score calculated as the number of milestones passed. It varies from 0 to 3.

Figure 3.4. Same data as in Figure 2.3, but now with the vertical axis representing gross-motor score.

The plot suggests that the difference in development between scores 0 and 1 is the same as the difference between, say, scores 2 and 3. This is not correct. For example, suppose that we express the difficulty of the milestone as an age-equivalent. From Section 3.1.2 we see that the difference between stepping and standing is 27.2 - 16.1 = 11.1 weeks, whereas the difference between walking alone and walking with help is 63.3 - 43.3 = 20 weeks. Thus, according to age equivalents scores 0 and 1 should be closer to each other, and ratings 2 and 3 should be drawn more apart.

3.3.3 Limitations of score-based measurement. Score-based measurement is today’s dominant approach, but is not without conceptual and logistical issues.

1.   The total score depends not only on the actual developmental status of the child, but also on the set of milestones administered. If a milestone is skipped or added, the sum score cannot be interpreted anymore as a measure of developmental status. It might be possible to correct for starting and stopping rules under the assumptions described in Section 3.3.1, but such will be involved if intermediate milestones are missing.

2.  It is not possible to compare the scores made by different instruments. Some instruments allow conversion to age-conditional scores. However, the sample used to derive such transformations pertain to that tool and does not generalise to others.

3.   Domains are hard to separate. For example, some cognitive milestones tap into fine motor capabilities, and vice versa. There are different ways to define domains, so domain interpretation varies by instrument.

4.   Administration of a full test may take substantial time. The materials are often proprietary and costly.

3.4 Unit-based measurement of development

3.4.1 Motivation for unit-based measurement. Unit-based measurement starts by defining ideal properties and derives a procedure to aggregate the responses on milestones into an overall score that will meet this ideal.

Section 2.4 highlighted questions for individuals, groups and populations. There are three questions:

•    What is the difference in development over time for the same child, group or community?

•    What is the difference in development between different children, groups or populations of the same age?

•    How does child development compare to a norm?

In the ideal situation, we would like to have a continuous (latent) variable D (for development) that measures child development. The scale should allow us to quantify ability of persons, groups or populations from low to high. It should have a constant unit so that a given difference in ability refers to the same quantity across the entire scale. We find the same property in height, where a distance of 10 cm represents the same amount for molecules, people or galaxies. When are these conditions are met, we say that we measure on an interval scale.

If we succeed in creating an interval scale for child development, an enormous arsenal of techniques developed for quantitative variables opens up to measure, track and analyze child development. We may then evaluate the status of a child in terms of D points gained, create age-dependent diagrams (just like growth charts for height and weight), devise age-conditional measures for child development, and intelligent adaptive testing schemes. Promising studies on Dutch data (Jacobusse et al., 2006; van Buuren, 2014) suggest that such benefits are well within reach.

3.4.2 Example of unit-based measurement. Figure 3.5 is similar to Figure 3.3, but with Age replaced by Ability. Also, modelled curves have replaced empirical ones, but this is not essential.

Figure 3.5. Modeled probability of achieving four motor milestones against the D-score.

We estimated the ability values on the horizontal axis from the data. The values correspond to the amount of development of each visit. Likewise, we calculated the logistic curves from the data. These reflect the probability of passing each milestone at a given level of ability.

Figure 3.5 shows that the probability of passing a milestone increases with ability. Items are sorted according to difficulty from left to right. Milestone stepping is the easiest and walk_alone is the most difficult. The point at which a logistic curve crosses the 50 per cent line (marked by a cross) is the difficulty of the milestone.

The increase in ability that is needed to go from 10 to 90 per cent is about five units here. Since all curves are parallel, the interval is constant for all scale locations. Thus, the scale is an interval scale with a constant unit of measurement, the type of measurement needed for answering the basic questions identified in Section 3.4.1.

3.4.3 Limitations of unit-based measurement. While unit-based measurement has many advantages, it cannot perform miracles.

1.   An important assumption is that the milestones “measure the same thing,” or put differently, are manifestations of a continuous latent variable that can be measured by empirical observations. Unit-based measurement won’t work if there is no sensible latent scale.

2.   The portrayed advantages hold only if the discrepancies between the data and the model are relatively small. Since the simplest and most powerful measurement models are strict, it is essential to obtain a good fit between the data and the model.

3.   The construction of unit-based measurement requires psychometric expertise, specialized computer software and considerable sample sizes.

3.5 A unified framework

This section brings together the four approaches outlined in this section into a unified framework.

Figure 3.6 shows the imaginary positions on a gross-motor continuum of three babies from Figure 2.1 at the age of 30 weeks. Both milestones and children are ordered along the same continuum. Thus, standing is more difficult than stepping, and at week 30, Doris is ahead of Walley in terms of motor development.

Figure 3.6. Placing milestones and children onto the same line reveals their positions.

More generally, measurement is the process of locating milestones and children on a line. This line represents a latent variable, a continuous construct that defines the different poles of the concept that we want to measure. A latent variable ranges from low to high.

The first part of measurement is to determine the location of the milestones on the latent variable. In many cases, the instrument maker has already done that. For example, each length marker on a ruler corresponds to a milestone for measuring length. The manufacturer of the ruler has already placed the marks at the appropriate places on the tool, and we take for granted that each marker has been calibrated correctly.

A milestone for child development is similar to a length marker, but

•    we may not know how much development the milestone measures, so its location on the line is unknown, or uncertain;

•    we may not know whether the milestone measures child development at all so that it may have no location on the line.

The second part of measurement is to find the location of each child on the line. For child height, this is easy: We place the horizontal headpiece on top of the child’s head and read off the closest height marker. Since we lack a physical ruler for development, we must deduce the child’s location on the line from the responses on a series of well-chosen milestones.

By definition, we cannot observe the values of a latent variable directly. However, we may be able to measure variables (milestones) that are related to the latent variable. For example, we may have scores on tasks like standing or walking with help.

The measurement model specifies the relations between the actual measurements and the latent variable. Under a given measurement model, we may estimate the locations of milestones and children on the line. Section 4.5 discusses measurement models in more detail.

3.6 Why unit-based measurement

This section distinguished four approaches to measure child development: age-based, probability-based, score-based and unit-based measurement. Table 3.1 summarizes how the approaches evaluate on nine criteria.

Age-based measurement expresses development in age equivalents, whose precise definition depends on the reference population. Age-based measurement does not support multiple milestones and does not use the concept of a latent variable.

Probability-based measurement expresses development as age percentiles for a reference population. It is useful for individual milestones but does not support multiple items or a latent variable interpretation.

Score-based measurement quantifies development by summing the number of passes. Different instruments make different selections of milestones, so the scores taken are unique to the tool. Thus comparing the measurement obtained by different devices is difficult. Skipping or adding items require corrections.

Unit-based measurement defines a unit by a theoretical model. When the data fit the model, we are able to construct instruments that produce values in a standard metric.

4.1 The Dutch Development Instrument (DDI)

4.1.1 Setting. The Dutch Youth Health Care (YHC) routinely monitors the development of almost all children living in The Netherlands. During the first four years, there are 13 scheduled visits. During these visits, the YHC professionals evaluate the growth and development of the child.

The Dutch Development Instrument (DDI; in Dutch: Van Wiechenschema) is the standard instrument used to measure development during the ages 0–4 years. The DDI consists of 75 milestones. The instrument assesses three developmental domains:

1.   Fine motor, adaptation, personality and social behaviour;

2.   Communication;

3.   Gross motor.

The milestones form two sets, one for children aged 0–15 months, and another for children aged 15–54 months. The YHC professionals administer an age-appropriate subset of milestones at each of the scheduled visits, thus building a longitudinal developmental profile for each child.

4.1.2 Description of SMOCC study. The Social Medical Survey of Children Attending Child Health Clinics (SMOCC) study is a nationally representative cohort of 2,151 children born in The Netherlands during the years 1988–1989 (Herngreen et al., 1994). The study monitored child development using observations made on the DDI during nine visits covering the first 24 months of life. The SMOCC study collected information during the first two years on 57 (out of 75) milestones.

The standard set in the DDI consists of relatively easy milestones that 90 per cent of the children can pass at the scheduled age. This set is designed to have maximal sensitivity for picking up delays in development. A distinctive feature of the SMOCC study was the inclusion of more difficult milestones beyond the standard set. The additional set originates from the next time point. The success rate on these milestones is about 50 per cent.

4.1.3 Codebook of DDI 0–30 months. Table 4.1 shows the 57 milestones from the DDI for ages 0 – 30 months as administered in the SMOCC study. Items are sorted according to debut, the age at which the item appears in the DDI. The response to each milestone is either a PASS (1) or a FAIL (0). Children who did not pass a milestone at the debut age were re-measured on that milestone during the next visit. The process continued until the child passed the milestone.

4.2 Probability of passing a milestone given age

Figure 4.1 summarizes the response obtained on each milestone as a curve against age. The percentage of pass scores increases with age for all milestones. Note that curves on the left have steeper slopes than those on the right, thus indicating that development is faster for younger children.

Figure 4.1. Empirical percentage of passing each milestone in the DDI against age (Source: SMOCC data, n = 2151, 9 occasions).

The domain determines the coloured (blue: gross motor, green: fine motor, red: communication). In general, domains are well mixed across age, though around some ages, e.g., at four months, multiple milestones from the same domain appear.

4.3 Probability of passing a milestone given D-score

Figure 4.2 is similar to Figure 4.1, but with the horizontal axis replaced by the D-score. The D-score summarizes development into one number. See 5.3 for a detailed explanation on how to calculate the D-score. The vertical axis with per cent pass is unchanged.

Figure 4.2. Empirical percentage of passing each milestone in the DDI against the D-score (Source: SMOCC data, 2151 children, 9 occasions).

The percentage of successes increases with D-score for all milestones. In contrast to Figure 4.1 all curves have a similar slope, a desirable property needed for an interval scale with a constant unit of measurement (c.f. Section 3.4).

How can the relation between per cent pass and age be so different from the relation between per cent pass and the D-score? The next section explains the reason.

4.4 Relation between age and the D-score

Figure 4.3 shows that the relation between D-score and age is nonlinear. Development in the first year is more rapid than in the second year. During the first year, infants gain about 40 D, whereas in the second year they gain about 20 D. A similar change in growth rate occurs in length (first year: 23 cm, second year: 12 cm, for Dutch children).

Figure 4.3. Relation between child D-score and child age in a cohort of Dutch children (Source: SMOCC data, n = 2151, 9 occasions).

Figure 4.4 shows the mutual relations between age, percentage of milestone passing and the D-score. There are three main orientations.

Figure 4.4. 3D-line graph illustrating how the patterns in Figure 4.1 and Figure 4.2 induce the curvature in the relation between D-score and age.

The printed version shows three orientations of the relation between age, percent pass and D-score. The online version holds an interactive 3D graph that the reader can actively manipulate the orientation of the graph by click-hold-drag mouse operations.

•    In the default orientation (age on the horizontal axis, D-score on the vertical axis), we see a curvilinear relation between the age and item difficulty.

•    Rotate the graph (age on the horizontal axis, passing percentage on the vertical axis). Observe that this is the same pattern as in Figure 4.1 (with unequal slopes). Curves are coloured by domain.

•    Rotate the graph (D-score on the horizontal axis, passing percentage on the vertical axis). Observe that this pattern is the same as in Figure 4.2 (with equal slopes).

All patterns can co-exist because of the curvature in the relation between D-score and age. The curvature is never explicitly modelled or defined, but a consequence of the equal-slopes assumption in the relation between the D-score and the passing percentage of a milestone.

4.5 Measurement model for the D-score

4.5.1 What are measurement models?

From section 3.5 we quote:

        The measurement model specifies the relations between the data and the latent variable.

The term Item Response Theory (IRT) refers to the scientific theory of measurement models. Good introductory works include Embretsen & Reise (2000); Wright & Masters (1982) and Engelhard Jr. (2013).

IRT models enable quantification of the locations of both items (milestones) and persons* on the latent variable. We reserve the term item for generic properties, and milestone for child development. In general, items are part of the measurement instrument, persons are the objects to be measured.

An IRT model has three major structural components:

•    Specification of the underlying latent variable(s). In this work, we restrict ourselves to models with just one latent variable. Multi-dimensional IRT models do have their uses, but they are complicated to fit and not widely used;

•    For a given item, a specification of the probability of success given a value on the latent variables. This specification can take many forms. Section 4.6 focuses on this in more detail;

•    Specification how probability models for the different items should be combined. In this work, we will restrict to models that assume local independence of the probabilities. In that case, the probability of passing two items is equal to the product of success probabilities.

4.5.2 Adapt the model? Or adapt the data? The measurement model induces a predictable pattern in the observed items. We can test this pattern against the observed data. When there is misfit between the expected and observed data, we can follow two strategies:

•    Make the measurement model more general;

•    Discard items (and sometimes persons) to make the model fit.

These are very different strategies that have led to heated debates among psychometricians. See Engelhard Jr. (2013) for an overview.

In this work, we opt for the - rigorous - Rasch model (Rasch (1960)) and will adapt the data to reduce discrepancies between model and data. Arguments for this choice are given later, in Section 4.8.

4.6 Item response functions

Most measurement models describe the probability of passing an item as a function of the difference between the person’s ability and the item’s difficulty. A person with low ability will almost inevitably fail a heavy item, whereas a highly able person will almost surely pass an easy item.

Let us now introduce a few symbols. We adopt the notation used in Wright & Masters (1982). We use β_n (ability) to refer to the true (but unknown) developmental score of child n. Symbol δ_i (difficulty) is the true (but unknown) difficulty of an item i, and π_ni is the probability that child n passes item i. See Appendix A for a complete list.

The difference between the ability of child n and difficulty of item i is

In the special case that β_n = δ_i, the person will have a probability of 0.5 of passing the item.

4.6.1 Logistic model. A widely used method is to express differences on the latent scale in terms of logistic units (or logits) (Berkson, 1944). The reason preferring the logistic over the linear unit is that its output returns a probability value that maps to discrete events. In our case, we can describe the probability of passing an item (milestone) as a function of the difference between β_n and δ_i expressed in logits.

Figure 4.5 shows how the percentage of children that pass the item varies in terms of the ability-difficulty gap β_n – δ_i. The gap can vary either by β_n or δ_i so that we may use the graph in two ways:

Figure 4.5. Standard logistic curve.

Percentage of children passing an item for a given ability-difficulty gap β_n – δ_i.

•    To find the probability of passing items with various difficulties for a child with ability β_n. If δ_i = β_n then π_ni = 0.5. If δ_i < β_n then π_ni > 0.5, and if δ_i > β_n then π_ni < 0.5. In words: If the difficulty of the item is equal to the child’s ability, then the child has a 50/50 chance to pass. The child will have a higher than 50/50 chance of passing for items with lower difficulty and have a lower than 50/50 chance of passing for items with difficulties that exceed the child’s ability.

•    To find the probability of passing a given item δ_i for children that vary in ability. If β_n < δ_i then π_ni < 0.5, and if β_n > δ_i then π_ni > 0.5. In words: Children with abilities lower than the item’s difficulty will have lower than 50/50 chance of passing, whereas children with abilities that exceed the item’s difficulty will have a higher than 50/50 chance of passing.

Formula (4.1) defines the standard logistic curve:

One way to interpret the formula is as follows. The logarithm of the odds that a person with ability β_n passes an item of difficulty δ_i is equal to the difference β_n – δ_i (Wright & Masters, 1982). For example, suppose that the probability that person n passes milestone i is π_ni = 0.5. In that case, the odds of passing is equal to 0.5/(1 – 0.5) = 1, so log(1) = 0 and thus β_n = δ_i. If β_n – δ_i = log(2) = 0.693 person n is two times more likely to pass than to fail. Likewise, if the difference is β_n – δ_i = log(3) = 1.1, then person n is three more likely to pass. And so on.

4.6.2 Types of item response functions. The standard logistic function is by no means the only option to map the relationship between the latent variable and the probability of passing an item. The logistic function is the dominant choice in IRT, but it is instructive to study some other mappings. The item response function maps success probability against ability.

Figure 4.6 illustrates several other possibilities. Let us consider five hypothetical items, A–E. Note that the horizontal axis now refers to the ability, instead of the ability-item gap in 4.5.

Figure 4.6. Item response functions for five hypothetical items, each demonstrating a positive relation between ability and probability to pass.

•    A: Item A is the logistic function discussed in Section 4.6.

•    B: For item B, the probability of passing is constant at 30 per cent. This 30 per cent is not related to ability. Item B does not measure ability, only adds to the noise, and is of low quality.

•    C: Item C is a step function centred at an ability level of 1, so all children with an ability below 1 logit fail and all children with ability above 1 logit pass. Item C is the ideal item for discriminating children with abilities above and below 1. The item is not sensitive to differences at other ability levels, and often not so realistic in practice.

•    D: Like A, item D is a smoothly increasing logistic function, but it has an extra parameter that allows it to vary its slope (or discrimination). The extra parameter can make the curve steeper (more discriminatory) than the red curve, in the limit approaching a step curve. It can also become shallower (less discriminatory) than the red curve (as plotted here), in the limit approaching a constant curve (item B). Thus, item D generalizes items A, B or C.

•    E: Item E is even more general in the sense that it need not be logistic, but a general monotonically increasing function. As plotted, the item is insensitive to abilities between -1 and 0 logits, and more sensitive to abilities between 0 to 2 logits.

These are just some examples of how the relationship between the child’s ability and passing probability could look. In practice, the curves need not start at 0 per cent or end at 100 per cent. They could also be U-shaped, or have other non-monotonic forms. See Coombs (1964) for a thorough overview of such models. In practice, most models are restricted to shapes A-D.

4.6.3 Person response functions. We can reverse the roles of persons and items. The person response function tells us how likely it is that a single person can pass an item, or more commonly, a set of items.

Let us continue with items A, C and D from Figure 4.6, and calculate the response function for three children, respectively with abilities β₁ = –2, β₂ = 0 and β₃ = 2.

Figure 4.7 presents the person response functions from three persons with abilities of -2, 0 and +2 logits. We calculate the functions as the average of response probabilities on items A, C and D. Thus, on average, we expect that child 1 logit will pass an easy item of difficulty -3 in about 60 per cent of the time, whereas for an intermediate item of difficulty of -1 the passing probability would be 10 per cent. For child 3, with higher ability, these probabilities are quite different: 97% and 90%. The substantial drop in the middle of the curve is due to the step function of item A.

Figure 4.7. Person response functions for three children with abilities -2, 0 and +2, using a small test of items A, C and D.

4.7 Engelhard criteria for invariant measurement

In this work, we strive to achieve invariant measurement, a strict form of measurements that is subject to the following requirements (Engelhard Jr., 2013, 14):

1.   Item-invariant measurement of persons: The measurement of persons must be independent of the particular items used for the measuring.

2.   Non-crossing person response functions: A more able person must always have a better chance of success on an item that a less able person.

3.   Person-invariant calibration of test items: The calibration of the items must be independent of the particular persons used for calibration.

4.   Non-crossing item response functions: Any person must have a better chance of success on an easy item than on a more difficult item.

5.   Unidimensionality: Items and persons take on values on a single latent variable. Under this assumption, the relations between the items are fully explainable by the scores on the latent scale. In practice, the requirement implies that items should measure the same construct. (Hattie, 1985)

Three families of IRT models support invariant measurement:

1.   Scalogram model (Guttman, 1950)

2.   Rasch model (Andrich, 1978; Rasch, 1960; Wright & Masters, 1982)

3.   Mokken scaling model (Mokken, 1971; Molenaar, 1997)

The Guttman and Mokken models yield an ordinal latent scale, while the Rasch model yields an interval scale (with a constant unit).

4.8 Why take the Rasch model?

•    Invariant measurement: The Rasch model meets the five Engelhard criteria (c.f. Section 4.7).

•    Interval scale: When it fits, the Rasch model provides an interval scale, the de-facto requirement for any numerical comparisons (c.f. Section 3.4.1).

•    Parsimonious: The Rasch model has one parameter for each item and one parameter for each person. The Rash model one of the most parsimonious IRT models, and can easily be applied to thousands of items and millions of persons.

•    Specific objectivity: Person and item parameters are mathematically separate entities in the Rasch model. In practice, this means that the estimated difference in ability between two persons does not depend on the difficulty of the test. Also, the estimated differences in difficulties between two items do not depend on the abilities in the calibration sample. The property is especially important in the analysis of combined data, where abilities can vary widely between sources. See Rasch (1977) for derivations and examples.

•    Unified model: The Rasch model unifies distinct traditions in measurement theory. One may derive the Rasch model from

•    Fits child development data: Last but not least, as we will see in Section 6, the Rasch model provides an excellent fit to child development milestones.

Note that the Rasch model is not unique in all aspects. A reviewer indicated that specific objectivity and invariant measurement might also be achieved in certain 2PL models. For us, the combination of simplicity, interpretability, and convenient properties makes the Rasch model stand out.

5.1 Identify nature of the problem

The SMOCC dataset, introduced in Section 4.1.2, contains scores on the DDI of Dutch children aged 0–2 years made during nine visits.

Table 5.1 contains data of three children, measured on nine visits between ages 0 – 2 years. The DDI scores take values 0 (FAIL) and 1 (PASS). In order to save horizontal space, we truncated the column headers to the last two digits of the item names.

Since the selection of milestones depends on age, the dataset contains a large number of empty cells. Naive use of sum scores as a proxy to ability is therefore problematic. An empty cell is not a FAIL, so it is incorrect to impute those cells by zeroes.

Note that some rows contain only 1’s, e.g., in row 2. Many computer programs for Rasch analysis routinely remove such perfect scores before fitting. However, unless the number of perfect scores is very small, this is not recommended because doing so can severely affect the ability distribution.

In order to effectively handle the missing data and to preserve all persons in the analysis we separate estimation of item difficulties (c.f. Section 5.2) and person abilities (c.f. Section 5.3).

5.2 Item parameter estimation

5.2.1 Pairwise estimation of item difficulties. There are many methods for estimating the difficulty parameters of the Rasch estimation. See Linacre (2004) for an overview.

We will use the pairwise estimation method. This method writes the probability that child n passes item i but not item j given that the child passed one of them as exp(δ_i)/(exp(δ_i) + exp(δ_j)). The method optimizes the pseudo-likelihood of all item pairs over the difficulty estimates by a simple iterative procedure.

Zwinderman (1995) has shown that this procedure provides consistent estimates with similar efficiency computationally more-intensive conditional and marginal maximum likelihood methods.

The beauty of the method is that it is independent of the ability distribution, so there is no need to remove perfect scores. We use the function rasch.pairwise.itemcluster() as implemented in the sirt package (Robitzsch, 2016).

Figure 5.1 summarizes the estimated item difficulty parameters. Although the model makes no distinction between domains, the results have been ordered to ease spotting of the natural progression of the milestones per domain. The figure also suggests that not all domain have equal representation across the scale. For example, there are no communication milestones around the logit of –10.

Figure 5.1. Estimated item difficulty parameters (d_i) for 57 milestones of the DDI (0 – 2 years).

5.2.2 Anchoring. The Rasch model identifies the item difficulties up to a linear transformation. By default, the software produces estimates in the logit scale (c.f. Figure 5.1). The logit scale is inconvenient for two reasons:

•    The logit scale has negative values. Negative values do not have a sensible interpretation in child development, and are likely to introduce errors in practice;

•    Both the zero in the logit scale, as well as its variance, depend on the sample used to calibrate the item difficulties.

Rescaling preserves the properties of the Rasch model. To make the scale independent of the specified sample, we transform the scale so that two items will always have the same value on the transformed scale. The choice of the two anchor items is essentially arbitrary, but they should correspond to milestones that are easy to measure with small error. In the sequel, we use the two milestones to anchor the D-score scale:

With the choice of Table 5.2, D-score values are approximately 0 D around birth. At the age of 1 year, the score will around 50 D, so during the first year of life, one D unit corresponds to approximately a one-week interval. Figure 5.2 shows the difficulty estimates in the D-score scale.

Figure 5.2. Estimated item difficulty parameters (d_i) for 57 milestones of the DDI (0 – 2 years).

Milestones ddigmd057 and ddigmd063 are anchored at values of 20D and 40D, respectively.

5.3 Estimation of the D-score

The second part of the estimation process is to estimate a D-score. The D-score quantifies the development of a child at a given age. Whereas the instrument developer is responsible for the estimation of item parameters, D-score estimation is more of a task for the user. To calculate the D-score, we need the following ingredients:

•    Child’s PASS/FAIL scores on the milestones administered;

•    The difficulty estimates of each milestone administered;

•    A prior distribution, an estimate of the D-score distribution before seeing any PASS/FAIL score.

Using these inputs, we may use Bayes theorem to calculate the position of the person on the latent variable.

5.3.1 Role of the starting prior. The first two inputs to the D-score will be self-evident. The third component, the prior distribution, is needed to be able to deal with perfect responses. The prior distribution summarizes our knowledge about the D-score before we see any of the child’s PASS/FAIL scores. In general, we like the prior to be non-informative, so that the observed responses and item difficulties entirely determine the value of the D-score. In practice, we cannot use truly non-informative prior because that would leave the D-score for perfect responses (i.e., all PASS or all FAIL) undefined. The choice of the prior is essentially arbitrary, but we can make it in such a way that its impact on the value D-score is negligible, especially for tests where we have more than, say, four items.

Since we know that the D-score depends on age, a logical choice for the prior is to make it dependent on age. In particular, we will define the prior as a normal distribution equal to the expected mean in Figure 4.3 at the child’s age, and with a standard deviation that considerably higher than in Figure 4.3. Numerical example: the mean D-score at the age of 15 months is equal to 53.6 D. The standard deviation in Figure 4.3 varies between 2.6 D and 3.0 D, with an average of 2.9 D. After some experimentation, we found that using a value of 5.0 D for the prior yields a good compromise between non-informativeness and robustness of D-score estimates for perfect patterns. The resulting starting prior for a child aged 15 months is thus N(53.6,5).

The reader now probably wonders about a chicken-and-egg problem: To calculate the D-score, we need a prior, and to determine the prior we need the D-score. So how did we calculate the D-scores in Figure 4.3? The answer is that we first took at rougher prior, and calculated two temporary models in succession using the D-scores obtained after solution 1 to inform the prior before solution 2, and so on. It turned out that D-scores in Figure 4.3 hardly changed after two steps, and so there we stopped.

5.3.2 Starting prior: Numerical example. Figure 5.3 illustrates starting distributions (priors) chosen according to the principles set above for the ages of 1, 15 and 24 months. As expected, the assumed ability of an infant aged one month is much lower than that of a child aged 15 months, which in turn is lower than the ability of a toddler aged 24 months. The green distribution for 15 months corresponds to the normal distribution N (53.6,5).

Figure 5.3. Age-dependent starting priors for the D-score at the ages of 1, 15 and 24 months.

Another choice that we need to make is the grid of points on which we calculate the prior and posterior distributions. Figure 5.3 uses a grid from -10 D to +80 D, with a step size of 1 D. These are fixed quadrature points, and there are 91 of them. While these quadrature points are sufficient to estimate D-score for ages up to 2.5 years, it is wise to extend the range for older children with higher D-scores.

5.3.3 EAP algorithm. The algorithm for estimating the D-score is known as the Expected a posteriori (EAP) method, first described by Bock & Mislevy (1982). Calculation of the D-score proceeds item by item. Suppose we have some vague and preliminary idea about the distribution of D, the starting prior (c.f. section 5.3.1), based on age. The procedure uses Bayes rule to update this prior knowledge with data from the first item (using the child’s FAIL/PASS score and the estimated item difficulty) to calculate the posterior. The next step uses this posterior as prior before processing the next item, and so on. The procedure stops when the item pool is exhausted. The order in which items enter does not matter for the result. The D-score is equal to the mean of the posterior calculated after the last question.

5.3.4 EAP algorithm: Numerical example. Suppose we measure two boys aged 15 months, David and Rob, by the DDI. David passes the first four milestones but does not complete the test. Rob completes the test but fails on two out of five items.

Table 5.3 shows the difficulty of each milestone (in the column labelled “Delta”), and the responses of David and Rob for the standard five DDI milestones for the age of 15 months.

The mean D-score for Dutch children aged 15 months is 53.6 D, so the milestones are easy to pass at this age, with the most difficult is ddicmm037. David passed all milestones but has no score on the last. Rob fails on ddifmm012 and ddigmm067. How do we calculate the D-score for David and Rob?

Figure 5.4 shows how the prior transforms into the posterior after we successively feed the measurements into the calculation. There are five milestones, so the calculation comprises five steps:

Figure 5.4. D-score distribution for David and Rob before (prior) and after (posterior) a milestone is taken into account.

1.    Both David and Rob pass ddifmd011. The prior (light green) is the same as in Figure 5.3. After a PASS, the posterior will be located more to the right, and will often be more peaked. Both happen here, but the change is small. The reason is that a PASS on this milestone is not very informative. For a child with a true D-score of 53 D, the probability of passing ddifmd011 is equal to 0.966. If passing is so common, there is not much information in the measurement.

2.    David passes ddifmm012, but Rob does not. Observe that the prior is identical to the posterior of ddifmd011. For David, the posterior is only slightly different from the prior, for the same reason as above. For Rob, we find a considerable change to the left, both for location (from 54.3 D to 47.1 D) and peakedness. This one FAIL lowers Rob’s score by 7.2 D.

3.    Milestone ddicmm037 is more difficult than the previous two milestones, so a pass on ddicmm037 does have a definite effect on the posterior for both David and Rob.

4.    David’s PASS on ddigmm066 does not bring any additional information, so his prior and posterior are virtually indistinguishable. For Rob, we find a slight shift to the right.

5.    There is no measurement for David on ddigmm067, so the prior and posterior are equivalent. For Rob, we observe a FAIL, which shifts his posterior to the left.

We calculate the D-score as the mean of the posterior. David’s D-score is equal to 55.7 D. Note that the measurement error, as estimated from the variance of the posterior, is relatively large. Rob’s D-score is equal to 47.7 D, with a much smaller measurement error. This result is consistent with the design principles of the DDI, which is meant to detect children with developmental delay.

The example illustrates that the quality of the D-score depends on two factors, the match between the true (but unknown) D-score of the child and the difficulty of the milestone.

5.3.5 Technical observations on D-score estimation

•    Administration of a too easy set of milestones introduces a ceiling with children that pass all milestones, but whose true D-score could extend well beyond the maximum. Depending on the goal of the measurement, this may or may not be a problem.

•    The specification of the prior and posterior distributions requires a set of quadrature points. The quadrature points are taken here as the static and evenly-spaced set of integers between -10 and +80. Using other quadrature points may affect the estimate, especially if the range of the quadrature points does not cover the entire D-score range.

•    The actual calculations are here done item by item. A more efficient method is to handle all responses at once. The result will be the same.

5.4 Age-conditional references

5.4.1 Motivation. The last step involves estimation an age-conditional reference distribution for the D-score. This distribution can be used to construct growth charts that portray the normal variation in development. Also, the references can be used to calculate age-standardized D-scores, called DAZ, that emphasize the location of the measurement in comparison to age peers.

Estimation of reference centiles is reasonably standard. Here we follow van Buuren (2014) to fit age-conditional references of the D-score for boys and girls combined by the LMS method. The LMS method by Cole & Green (1992) assumes that the outcome has a normal distribution after a Box-Cox transformation. The reference distribution has three parameters, which model respectively the location (M), the spread (S), and the skewness (L) of the distribution. Each of the three parameters can vary smoothly with age.

5.4.2 Estimation of the reference distribution. The parameters are estimated using the BCCG distribution of gamlss 5.1-3 (Stasinopoulos & Rigby, 2008) using cubic splines smoothers. The final solution used a log-transformed age scale and fitted the model with smoothing parameters df(M) = 2, df(S) = 2 and df(L) = 1.

Figure 4.3 plots the D-scores together with five grey lines, corresponding to the centiles -2SD (P2), -1SD (P16), 0SD (P50), +1SD (P84) and +2SD (P98). The area between the -2SD and +2SD lines delineates the D-score expected if development is healthy. Note that the shape of the reference is quite similar to that of weight and height, with rapid growth occurring in the first few months.

Table 5.4 defines age-conditional references for Dutch children as the M-curve (median), S-curve (spread) and L-curve (skewness) by age. This table can be used to calculate centile lines and Z-scores.

The references are purely cross-sectional and do not account for the correlation structure between ages. For prediction purposes, it is useful to extend the modelling to include velocities and change scores.

5.4.3 Conversion of D to DAZ, and vice versa. Suppose that M_t, S_t and L_t are the parameter values at age t. Cole (1988) shows that the transformation

converts measurement D_t into its normal equivalent deviate Z. If L_t is close to zero, we use

We may derive any required centile curve from Table 5.4. First, choose Z_α as the Z-score that delineates 100α per cent of the distribution, for example, Z_0.05 = –1.64. The D-score that defines the 100α centile is equal to

If L_t is close to zero, we use

6.1 Item fit

The philosophy of the Rasch model is different from conventional statistical modelling. It is not the task of the Rasch model to account for the data. Rather it is the task of the data to fit the Rasch model. We saw this distinction before in Section 4.5.2.

The goal of model-fit assessment is to explore and quantify how well empirical data meet the requirements of the Rasch model. One way to gauge model-fit is to compare the observed probability of passing an item to the fitted item response curve for endorsing the item.

The fitted item response curve for each item i is modeled as:

where ${\widehat{\beta }}_n$ is the estimated ability of child n (the child’s D-score), and where ${\widehat{\delta }}_i$ is the estimated difficulty of item i. This is equivalent to formula (4.1) with the parameters replaced by estimates. Section 5 described process of parameter estimation in some detail.

6.1.1 Well-fitting item response curves. The study of item fit involves comparing the empirical and fitted probabilities at various levels of ability. Figure 6.1 shows the item characteristics curves of two DDI milestones. The orange line represents the empirical probability at different ability levels. The dashed line represents the estimated item response curve according to the Rasch model. The observed and estimated curves are close together, so both items fit the model very well.

Figure 6.1. Empirical and fitted item response curves for two milestones from the DDI (SMOCC data).

6.1.2 Item response curves showing severe underfit. There are many cases where things are less bright.

Figure 6.2 shows three forms of severe underfit from three artificial items. These items were simulated to have a low fit, added to the DDI, and we estimated their parameters by the methods of Section 5. For the first item, hypgmd001, the probability of passing is almost constant across ability, so retaining this item essentially only adds to the noise. Item hypgmd002 converges to an asymptote around 80 per cent and has a severe dip in the middle. The strong relation to age causes the drop. Item hypgmd003 appears to have the wrong coding. Also, we often see the spike-like behaviour in the middle when two or more different items erroneously share identical names.

Figure 6.2. Three simulated items that illustrate various forms of item misfit.

Removal of items with a low fit can substantially improve overall model fit.

6.1.3 Item response curves showing overfit. Figure 6.3 shows two artificial items with two forms of overfitting. The curve of item hypgmd004 is much steeper than the modelled curve. Thus, just this one item is exceptionally well-suited to distinguish children with a D-score below 50 D from those with a score above 50 D. Note that the item isn’t sensitive anywhere else on the scale. In general, having items like these is good news, because they allow us to increase the reliability of the instrument. One should make sure, though, that FAIL and PASS scores are all measured (not imputed) values.

Figure 6.3. Two simulated items that illustrate item overfit.

Multiple perfect items could hint to a violation of the local independence assumption (c.f. Section 4.5). Developmental milestones sometimes have combinations of responses that are impossible. For example, one cannot walk without being able to stand, so we will not observe the inconsistent combination (stand: FAIL, walk: PASS). This impossibility leads to more consistent responses that would be expected by chance alone. In principle, one could combine the two such items into one three-category item, which effectively set the probability of inconsistent combinations to zero.

Item hypgmd005 is also steep, but has an asymptote around 80 per cent. This tail behaviour causes discrepancies between the empirical and modeled curves around the middle of the probability scale. In general, we may remove such items if a sufficient number of alternatives is available.

6.1.4 Item infit and outfit. We quantify item fit by item infit and outfit. Both are aggregates of the model residuals. The observed response x_ni of person n on item i can be 0 or 1.

The standardized residual z_ni is the difference between the observed response x_ni and the expected response p_ni, divided by the expected binomial standard deviation,

where the expected response variance W_ni is calculated as

Item infit is the total of the squared residuals divided by the sum of the expected response variances W_ni

Item outfit is calculated as the average (over N measurements) of the squared standardized residual

The expected value of both infit and outfit is equal to 1.0. The interpretation is as follows:

•    If infit and outfit are 1.0, then the item perfectly fits the Rasch model, as in Figure 6.1;

•    If infit and outfit > 1.0, then the item is not fitting well. The amount of underfit is quantified by infit and outfit, as in 6.2;

•    If infit and outfit < 1.0, then the item fits the model better than expected (overfit). Overfitting is quantified by infit and outfit, as in 6.3.

Infit is more sensitive to disparities in the middle of the probability scale, whereas outfit is the better measure for discrepancies at probabilities close to 0 or 1. Lack of fit is generally easier to spot at the extremes. The two measures are highly correlated. Achieving good infit is more valuable than a high outfit.

Values near 1.0 are desirable. There is no cut and dried cut-off value for infit and outfit. In general, we want to remove underfitting items with infit or outfit values higher than, say, 1.3. Overfitting items (with values lower than 1.0) are not harmful. Preserving these items may help to increase the reliability of the scale. The cut-off chosen also depends on the number of available items. When there are many items to choose from, we could use a stricter criterion, say infit and outfit < 1.0 to select only the absolute best items.

6.1.5 Infit and outfit in the DDI. Figure 6.4 displays the histogram of the 57 milestones from the DDI (c.f. Section 4.1). Most infit values are within the range 0.6 - 1.1, thus indicating excellent fit. The two milestones with shallow infit values are ddigmd052 and ddigmd053. These two items screen for paralysis for newborns, so the data contain hardly any fails on these milestones. The outfit statistics also indicate a good fit.

Figure 6.4. Frequency distribution of infit (left) and outfit (right) of 57 milestones from the DDI (SMOCC data).

6.2 Person fit

Person fit quantifies the extent to which the responses of a given child conform to the Rasch model expectation. The Rasch model expects that a more able child has a higher probability of passing an item than a less developed child. Person fit analysis evaluates the extent to which this is true.

6.2.1 Person infit and outfit. In parallel to item fit, we can calculate person infit and person outfit. Both statistics evaluate how well the responses of the persons are consistent with the model. Outlying answers that do not fit the expected pattern increase the outfit statistic. The outfit is high, for example, when the child fails easy items but passes difficult ones. The infit is the information weighted fit and is more sensitive to inlaying, on-target, unexpected responses.

Similar to item fit, person fit is also calculated from the residuals, but aggregated differently. We calculate person infit as

and person outfit as

A threshold for person fit > 3.0 is customary to clean out children with implausible response patterns.

6.2.2 Person infit and outfit in the DDI. Figure 6.5 displays the frequency distribution of person infit and person outfit 16538 measurements of the DDI in the SMOCC data. The majority of the values falls below 3.0. For infit, only 43 out of 16538 fit values (0.3 per cent) is above 3.0. There are 446 out of 16538 outfit value (2.7 per cent) above 3.0. Expect the solution to improve after deleting these measurements.

Figure 6.5. Frequency distribution of person infit (left) and person outfit (right) for 16538 measurements of the DDI (SMOCC data).

6.3 Differential item functioning (DIF)

6.3.1 Relevance of DIF for cross-cultural equivalence. An essential assumption in the Rasch model is that a given item has the same difficulty in different subgroups of respondents. Climbing stairs is an example where this assumption is suspect. The exposure to stairs, and hence the opportunity for a child to practice, varies across different cultures. It could thus be that two children with the same ability but from different cultures have different success probabilities for climbing stairs. When these probabilities systematically vary between subgroup, we say there is Differential Item Functioning, or DIF (Holland & Wainer, 1983). DIF is undesirable since it can make the instrument culturally biased.

6.3.2 How to detect DIF?

Zumbo (1999) provided a clear definition of DIF:

There are various ways to detect DIF. Here we will model the probability of endorsing an item by logistic regression using the observed item responses as the outcome. Predictors include the ability, the grouping variable, and the ability-grouping interaction. If the latter two terms explain the residual variance of the item scores after adjusting for ability, the item shows DIF for that group. DIF can be visually inspected by plotting the curves for the subgroups separately.

There are two forms of DIF:

•      Uniform DIF: The item response curves differ between groups in location, but are parallel;

•      Non-uniform DIF: The item response curve differ between groups in location, in slope and possibly in other characteristics.

These forms correspond to, respectively, the main effect of group and the ability-group interaction in the logistic regression model.

6.3.3 Examples of DIF. Figure 6.6 shows an example comparing boys and girls. For both milestones, the item response curves are similar for boys and girls, so we see no evidence of DIF here.

Figure 6.6. Two milestones from the DDI with similar item response curves for boys and girls.

There is no DIF for sex.

Figure 6.7 displays two milestones with DIF between boys and girls. Provided that the ability estimate (as estimated from all items in the model) is fair for both boys and girls, we see that milestone ddifmm019 (“Takes off shoes and socks”) is easier for girls by about 0.86 logits (= the difference in ability at the intersection of 50 per cent). Conversely, milestone ddigmm064 (“Crawls forward, abdomen on the floor”) is easier for boys by about 0.84 logits. These are the most substantial differences found for sex in the DDI. Both are uniform DIF.