RESEARCH METHOD COHEN ok

VALIDITY AND RELIABILITY IN TESTS 163
in question. Content validity is achieved by
making professional judgements about the relevance
and sampling of the contents of the
test to a particular domain. It is concerned
with coverage and representativeness rather
than with patterns of response or scores. It is
a matter of judgement rather than measurement
(Kerlinger 1986). Content validity will
need to ensure several features of a test (Wolf
1994): (a) test coverage (the extent to which
the test covers the relevant field); (b) test relevance
(the extent to which the test items
are taught through, or are relevant to, a particular
programme); (c) programme coverage
(the extent to which the programme covers
the overall field in question).
Criterion-related validity is where a high correlation
coefficient exists between the scores on the
test and the scores on other accepted tests of
the same performance: this is achieved by comparing
the scores on the test with one or more
variables (criteria) from other measures or tests
that are considered to measure the same factor.
Wolf (1994) argues that a major problem
facing test devisers addressing criterion-related
validity is the selection of the suitable criterion
measure. He cites the example of the difficulty
of selecting a suitable criterion of academic
achievement in a test of academic aptitude.
The criterion must be: relevant (and agreed to
be relevant); free from bias (i.e. where external
factors that might contaminate the criterion
are removed); reliable – precise and accurate;
capable of being measured or achieved.
Construct validity (e.g. the clear relatedness
of a test item to its proposed construct/unobservable
quality or trait, demonstrated
by both empirical data and logical
analysis and debate, i.e. the extent to which
particular constructs or concepts can give an
account for performance on the test): this is
achieved by ensuring that performance on the
test is fairly explained by particular appropriate
constructs or concepts. As with content validity,
it is not based on test scores, but is more a
matter of whether the test items are indicators
of the underlying, latent construct in question.
In this respect construct validity also subsumes
content and criterion-related validity. It is argued
(Loevinger 1957) that, in fact, construct
validity is the queen of the types of validity because
it is subsumptive and because it concerns
constructs or explanations rather than methodological
factors. Construct validity is threatened
by under-representation of the construct,
i.e. the test is too narrow and neglects significant
facets of a construct, and by the inclusion
of irrelevancies – excess reliable variance.
Concurrent validity is where the results of the
test concur with results on other tests or instruments
that are testing/assessing the same
construct/performance – similar to predictive
validity but without the time dimension. Concurrent
validity can occur simultaneously with
another instrument rather than after some time
has elapsed.
Face validity is where, superficially, the test appears
– at face value – to test what it is designed
to test.
Jury validity is an important element in construct
validity, where it is important to agree
on the conceptions and operationalization of
an unobservable construct.
Predictive validity is where results on a test accurately
predict subsequent performance – akin
to criterion-related validity.
Consequential validity is where the inferences
that can be made from a test are sound.
Systemic validity (Frederiksen and Collins
1989) is where programme activities both
enhance test performance and enhance performance
of the construct that is being addressed
in the objective. Cunningham (1998) gives an
example of systemic validity where, if the test
and the objective of vocabulary performance
leads to testees increasing their vocabulary,
then systemic validity has been addressed.
To ensure test validity, then, the test must
demonstrate fitness for purpose as well as
addressing the several types of validity outlined
above. The most difficult for researchers to
address, perhaps, is construct validity, for it
argues for agreement on the definition and
Chapter 6