Why measure child development?

Introduction: Why measure child development?>200 million disadvantaged children worldwidePercentage of disadvantaged children under 5 years old by country in 2004Grantham-McGregor et al., Lancet (2007)

Differential risk and vulnerability• Childrens development from0-5 is dependent on quality ofearly environments andrelationship with caregiver.• Young children growing upin poverty aredisproportionately exposedto a wide range of riskfactors:• Poor nutrition• Less stimulating learningenvironments• Poor sanitation• Stressful life events• Exposure to environmentalrisks

Poverty and cumulative risk• Number of risk factorsincreases over time.– Cumulative effect of risk factorsbecomes more evident aschildren get older• Higher cumulative levels ofrisk are associated with:– Poorer cognitive development– Psychological distress andbehavioral problems– Slower and lower qualitycommunicative development

Conceptual frameworkDirect&IndirectEffectsFrom Walker and al. Lancet, 2007• Environmental factors– Psychosocial risks: harsh disciplinary techniques, maternal depression– Biological risks: malnutrition and infectious diseases• Poverty and socio-cultural factors increase likelihood of bothtypes of risks

Timeline of developmentTiming of human brain development, from Grantham-McGregor, et al., 2007• Early childhood is characterized by developmental spurtsand plateaus• Skills emerge at different rates and ages

Ecological model of child developmentAdapted representation of Bronfenbrenners ecological model of child development (Wortham, 2007)

The Toolkit• Importance of measuringchild development• Domains of development tobe measured• Theoretical decisions inselecting instruments• Modification, adaptation andstandardization of existingtests• Creation of new tests• Training and quality control• Conclusions andrecommendations

Domains of development• Domains of development:– Cognitive– Language– Motor– Executive function/selfregulatory– Social/emotional• Domains are overlappingand mutually influencing• Every effort should be madeto include multiple domainswhen assessing childrensdevelopment

Step 1:Definepurpose ofassessmentSTEP 1: Define purpose of assessmentFor example:1 . To plan interventions or services;2 . To monitor programs;3 . To conduct impact evaluations;4 . To investigate the effect of interventions orprograms on specific outcomes of interest;5 . To design a curriculum for a particular child;or6 . To diagnose and assess child progressSTEP 2: Determine type of assessmentScreeningBrief assessment; identifies children likely tohave problems based on cutoffs derived in testpopulation. Does not yield continuous scores.Useful for examples 1-4 above.AbilitiesDetailed assessment of childs maximum skilllevel for age. Provides continuous scores thatallow comparisons within and across children/groups. Suitable for all examples above.STEP 3: Determine mode of assessmentDirectRatings/ReportsObservationDirectRatings/ReportsObservationSTEP 4: Determine which assessment to use (examples below)Denver(DDST II)Ages andStagesQuestionnairesNaturalisticsample orstructuredsamplingBayley Scales IIIWoodcock-JohnsonWPPSIStanford-BinetKaufman-ABCExecutivefunction tasksMacArthurCommunicative InventoriesNaturalisticsample orstructuredsampling (seeIEAs ChildCodingSystem)

Key Questions in Selecting Instruments• What are the goals of the assessment/evaluation?• What dimensions of childs development do you expect to beaffected by the intervention?– What developmental systems are most vulnerable at a given age range?– What are immediate outcomes and longer term outcomes?• What are the mechanisms at work?– What physiologic processes are influenced by iodine/iron/poverty?• What are key elements of context that must be considered inselecting the test?– Urban/rural, level of poverty, parent education.• At what level will effect be measured?– Individual? Household? Population (then consider test such as EDI)?• How will the sample be selected?– Population sample? Sub-sample?• What is the analytic plan?– Are norms relevant and/or available? Will a cut-off score be used?

Step 2:Determinetype ofassessmentSTEP 1: Define purpose of assessmentFor example:1 . To plan interventions or services;2 . To monitor programs;3 . To conduct impact evaluations;4 . To investigate the effect of interventions orprograms on specific outcomes of interest;5 . To design a curriculum for a particular child;or6 . To diagnose and assess child progressSTEP 2: Determine type of assessmentScreeningBrief assessment; identifies children likely tohave problems based on cutoffs derived in testpopulation. Does not yield continuous scores.Useful for examples 1-4 above.AbilitiesDetailed assessment of childs maximum skilllevel for age. Provides continuous scores thatallow comparisons within and across children/groups. Suitable for all examples above.STEP 3: Determine mode of assessmentDirectRatings/ReportsObservationDirectRatings/ReportsObservationSTEP 4: Determine which assessment to use (examples below)Denver(DDST II)Ages andStagesQuestionnairesNaturalisticsample orstructuredsamplingBayley Scales IIIWoodcock-JohnsonWPPSIStanford-BinetKaufman-ABCExecutivefunction tasksMacArthurCommunicative InventoriesNaturalisticsample orstructuredsampling (seeIEAs ChildCodingSystem)

Types of assessments: Direct tests• Pros:– Data are gathered first hand– Data can be less biased thanparental reports– Potentially wider range ofoutcomes can be assessed– Many of the cons can beovercome with careful planningand preparation• Cons:– Young children can be difficultto test (sleeping, hungry)– Testers need a lot of trainingand oversight– Accuracy depends on testingdemands and child must befamiliar with parameters (e.g.best v. worst)

Types of assessments: Parent report• Pros– Easy to administer and requireminimal training and instruction– Often are quick and easy tocomplete and to score– Parents can become involved andexpress concerns– Often correlate well with directassessments– Teachers can be an additionalsource of information as children getolder• Cons– Parents and teachers may artificiallyinflate scores– Parents may not accurately reportabilities– Parents and teachers may havedifferent interpretations of items indifferent cultures

Types of assessments: ObservationTypes of observation: Naturalistic observation, Sampled observation, Structured situation• Pros:– Highly valid– Measures behavior in anidentified context– Can provide additional orconfirmatory information for othertypes of assessments• Cons:– Requires a lot of time andtraining– Need to identify if culturallyappropriate– Difficult coding sinceobservational codes anddefinitions are not always clearlydefined

Step 4:Determinewhichassessmentto useSTEP 1: Define purpose of assessmentFor example:1 . To plan interventions or services;2 . To monitor programs;3 . To conduct impact evaluations;4 . To investigate the effect of interventions orprograms on specific outcomes of interest;5 . To design a curriculum for a particular child;or6 . To diagnose and assess child progressScreeningBrief assessment; identifies children likely tohave problems based on cutoffs derived in testpopulation. Does not yield continuous scores.Useful for examples 1-4 above.STEP 2: Determine type of assessmentAbilitiesDetailed assessment of childs maximum skilllevel for age. Provides continuous scores thatallow comparisons within and across children/groups. Suitable for all examples above.STEP 3: Determine mode of assessmentDirectRatings/ReportsObservationDirectRatings/ReportsObservationSTEP 4: Determine which assessment to use (examples below)Denver(DDST II)Ages andStagesQuestionnairesNaturalisticsample orstructuredsamplingBayley Scales IIIWoodcock-JohnsonWPPSIStanford-BinetKaufman-ABCExecutivefunction tasksMacArthurCommunicative InventoriesNaturalisticsample orstructuredsampling (seeIEAs ChildCodingSystem)

Other constraints to consider• Budget: Tests can be very expensive (e.g. $1000 forBayley); administration time is a budget issue, too.• Copyright issues: Must obtain permission for most tests.• Time allocated for testing: Direct assessment v. parent rep.• Training: Capacity for administration.• Test setting: Set-up, lighting, noise, observers• Capacity of respondent: Education/knowledge of parent• Language and cultural differences: Words used in testingmaterials, approach used for testing (e.g. speedyresponse)• Materials: Must be familiar and/or available (e.g. mirror,ball)

Ethical risks and responsibilities• All assessment protocols must bereviewed and approved by anethical review board• Accuracy and validity areextremely important especially iftest scores are being used toidentify children with delays• Follow-up (e.g. referrals for atriskchildren) should bemandatory even in the context ofa developing country.

Broad recommendations• Assess characteristics of the child that the intervention is intending toaffect.– Make sure to measure variables that could also be contributing to theoutcomes (e.g. maternal responsiveness, home environment)• Decide on the type of outcome measure that is appropriate for theevaluation.• Rely upon multiple measures of childrens development.– Include assessments of executive function and socio-emotional development• Consider the cultural context and how it may affect childrensdevelopment and school readiness– Always work with local collaborators!• Look for national level tests where possible and use parent/teacherreport when possible.• Begin following children early in life.

Criteria for being recommended• Psychometrically adequate, valid and reliable;• Balanced in terms of number of items at the lower endto avoid children with low scores;• Enjoyable for children to take (e.g. interactive, colorfulmaterials);• Relatively easy to adapt to various cultures;• Easy to use in low-resource settings, e.g. not requiringmuch material;• Not too difficult to obtain or too expensive;• Able to be used in a wide age range.

ELPI 2010Area Test Evaluation ProposalEEDP (Scale of Psychomotor Development Evaluation)Validity, Reliability andNormsValidity, Reliability andBDI (Battelle Developmental Inventory)OverallNormsDevelopmentValidity, Reliability andTEPSI (Psychomotor Development Test)NormsTVIP / PPVT (Peabody Picture Vocabulary Test)Validity, Reliability andNormsArea Test Evaluation ProposalCognitive AbilitiesScale WAIS DigitScale WAIS VocabularyValidity, Reliability andNormsValidity, Reliability andNorms

ELPI 2012Area Test Evaluation ProposalOverallDevelopmentTADI (Test of Learning and Child Development)BDI-ST2 (Battelle Developmental Inventory -Screening Test 2)TVIP / PPVT (Peabody Picture Vocabulary Test)Made it by CEDEPValidity, Reliability andNormsValidity, Reliability andNormsExecutive FunctionSDT (Snack Delay Task)PTT (Pencil Tapping Task)BDS (Backward Digit Span Task)HTKS (Head Toes Knees Shoulders Task)Validity, Reliability andNormsValidity, Reliability andNormsArea Test Evaluation ProposalCognitive AbilitiesScale WAIS DigitScale WAIS VocabularyValidity, Reliability andNormsValidity, Reliability andNorms

ELPI: Home stimulation (HOME score)

ELPI: Wealth gradients

ELPI (PPVT): gradients over time

Advantages of longitudinal cohort• Document specific trajectories of development• Examine shape and timing of emergence ofparticular skills• Steepening of gradient at different ages due toindividual or household shocks• Potential for interventions to affectdevelopmental trajectories

Acknowledgements• Funding and support from World Bank for the Toolkit– Barbara Bruns– Sophie Naudeau– Harold Alderman– Ariel Fitzbein• External reviewers for the Toolkit– Frances Aboud, McGill University– Santiago Cueto, Catholic University, Peru– Ed Frongillo, University of South Carolina– Jane Kvalsvig, University of Kwa-Zulu Natal, South Africa– Ann Weber, University of California, Berkeley– Paul Wassenich, University of California, Berkeley– Michelle Neuman, The World Bank– Mary Eming Young, The World Bank• Research assistants– Robin Dean (UC Berkeley), Kallista Bley (UC Berkeley), Melissa Hidrobo (UCBerkeley), Anna Moore (Cal Poly), Rose Calnin Kagawa• Photo credits for photographs included in presentation– Lia Fernald, Emanuela Galasso, Lisy Ratsifandrihamanana, Ann Weber, Tricia Kariger

Thank you!Contact information:Lia FernaldSchool of Public HealthUniversity of California, Berkeleyfernald@berkeley.edu(510) 643-9113