Family Assessment in Child Welfare - Center for Social Services ...
Family Assessment in Child Welfare - Center for Social Services ...
Family Assessment in Child Welfare - Center for Social Services ...
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
Appendix AMeasurement Criteria (Perlmutter & Czar, 2001; Pedhazur & Schmelk<strong>in</strong>, 1991)MeasurementCriteriaVALIDITYContent-RelatedValidityCriterion-RelatedValidityConstruct-RelatedValidityRELIABILITYTest-RetestReliabilityAlternate-FormInter-rater ReliabilityInternal ConsistencyDef<strong>in</strong>itionsA valid <strong>in</strong>strument measures what it claims to measure. Major types of validity <strong>in</strong>cludecontent-related validity, criterion-related validity, and construct-related validity.A well-constructed <strong>in</strong>strument can be considered to adequately represent a specified areaof knowledge as well as avoid the effects of unrelated variables. Judges that renderop<strong>in</strong>ions about the suitability of items <strong>in</strong> an <strong>in</strong>strument commonly assess content validity.There is no accepted standard of agreement established <strong>for</strong> reta<strong>in</strong><strong>in</strong>g an item, however,better scales tend to elicit greater agreement among judges.Relates to predict<strong>in</strong>g an <strong>in</strong>dividual’s per<strong>for</strong>mance aga<strong>in</strong>st a score on an exist<strong>in</strong>g <strong>in</strong>strumentor a future outcome. Two methods <strong>for</strong> establish<strong>in</strong>g criterion-related validity <strong>in</strong>cludeconcurrent and predictive strategies.Concurrent validity. Us<strong>in</strong>g this method, tests are adm<strong>in</strong>istered to <strong>in</strong>dividuals <strong>for</strong> whomcriterion data are already available <strong>for</strong> accurate <strong>in</strong>dicators of the construct under study(e.g., an exist<strong>in</strong>g measure of family function<strong>in</strong>g). Correlations between scores or rat<strong>in</strong>gs onthe new <strong>in</strong>strument and those obta<strong>in</strong>ed on the exist<strong>in</strong>g <strong>in</strong>strument are then established.Predictive validity. Denotes an <strong>in</strong>strument’s ability to predict future outcomes or statusfrom scores on an <strong>in</strong>strument.Differential predictive validity refers to an <strong>in</strong>strument’s ability to predict these outcomes<strong>for</strong> different groups (e.g., abusive vs. non-abusive families).Relates to the degree to which an <strong>in</strong>strument successfully measures a theoretical concept.Two methods, convergent and divergent validity, establish the construct-related validity ofa test.Convergent-Divergent Validity. When different measures of a concept yield similar resultsthey converge; demonstrat<strong>in</strong>g convergent validity typically <strong>in</strong>volves correlat<strong>in</strong>g twoexist<strong>in</strong>g measures or correlat<strong>in</strong>g a new measure with an exist<strong>in</strong>g measure. When conceptscan be empirically differentiated from other concepts they diverge; there<strong>for</strong>e, measures ofdifferent constructs will possess low l<strong>in</strong>ear correlations.Reliability is an <strong>in</strong>dex of the degree to which <strong>in</strong>dividual differences <strong>in</strong> scor<strong>in</strong>g reflectactual differences <strong>in</strong> the characteristic under consideration versus chance errors. There arefour types of reliability: test-retest, alternate <strong>for</strong>m, <strong>in</strong>ter-rater, and <strong>in</strong>ternal consistency.Refers to the degree to which generalizations can be made about test scores from oneadm<strong>in</strong>istration to the next. Established by correlat<strong>in</strong>g results of basel<strong>in</strong>e and subsequentadm<strong>in</strong>istrations. Higher test-retest coefficients generally mean that scores are lesssusceptible to random changes <strong>in</strong> the condition of test takers or the test<strong>in</strong>g environment.When equivalent <strong>for</strong>ms of a test are adm<strong>in</strong>istered to the same person on two occasions, thereliability coefficient is the correlation between scores obta<strong>in</strong>ed on the two tests. Thehigher the scores, the more likely it is that the different test <strong>for</strong>ms are measur<strong>in</strong>g the samecharacteristics.Relates to the extent to which two or more people arrive at the same result when observ<strong>in</strong>gand/or rat<strong>in</strong>g the same event. It is most frequently reported <strong>for</strong> observational techniqueswhen events are recorded or rat<strong>in</strong>gs of behaviors are made.Cronbach’s alpha is the measure of <strong>in</strong>ternal consistency by which we <strong>in</strong>fer that itemswith<strong>in</strong> a scale or subscale measure the same construct. Alpha rises when the average <strong>in</strong>teritemcorrelation between items <strong>in</strong>creases. It also <strong>in</strong>creases with <strong>in</strong>creased numbers ofitems, so long as the quality of those items rema<strong>in</strong>s high. There<strong>for</strong>e, when consider<strong>in</strong>g thevalue of alpha <strong>for</strong> a given <strong>in</strong>strument or subscale, both the reported alpha and the numberof items must be considered.27