RESEARCH METHOD COHEN ok

COMPUTERIZED ADAPTIVE TESTING 433
How ethical it would be to undertake the final
four of these is perhaps questionable, or indeed
any apart from the first on the list. Are they
cheating or legitimate test preparation Should
one teach to a test; is not to do so a dereliction
of duty (e.g. in criterion- and domain-referenced
tests) or giving students an unfair advantage and
thus reducing the reliability of the test as a true and
fair measure of ability or achievement In highstakes
assessment (e.g. for public accountability
and to compare schools and teachers) there
is even the issue of not entering for tests
students whose performance will be low (see, for
example, Haladyna et al.1991).Thereisariskofa
correlation between the ‘stakes’ and the degree
of unethical practice – the greater the stakes,
the greater the incidence of unethical practice.
Unethical practice, observes Gipps (1994), occurs
where scores are inflated but reliable inference
on performance or achievement is not, and
where different groups of students are prepared
differentially for tests, i.e. giving some students an
unfair advantage over others. To overcome such
problems, she suggests, it is ethical and legitimate
for teachers to teach to a broader domain than
the test, that teachers should not teach directly
to the test, and the situation should only be that
better instruction rather than test preparation is
acceptable (Cunningham 1998).
One can add to this list of considerations
(Cronbach 1970; Hanna 1993; Cunningham
1998) the following views:
Tests must be valid and reliable (see
Chapter 6).
The administration, marking and use of the
test should be undertaken only by suitably
competent/qualified people (i.e. people and
projects should be vetted).
Access to test materials should be controlled,
thus test items should not be reproduced apart
from selections in professional publication;
the tests should be released only to suitably
qualified professionals in connection with
specific professionally acceptable projects.
Tests should benefit the testee (beneficence).
Clear marking and grading protocols should
exist (the issue of transparency is discussed in
Chapter 6).
Test results are reported only in a way that
cannot be misinterpreted.
The privacy and dignity of individuals should
be respected (e.g. confidentiality, anonymity,
non-traceability).
Individuals should not be harmed by the test
or its results (non-maleficence).
Informed consent to participate in the test
should be sought.
Computerized adaptive testing
Computerized adaptive testing (Wainer 1990;
Aiken 2003: 50–2) is the decision on which
particular test items to administer, which is based
on the subjects’ responses to previous items. It is
particularly useful for large-scale testing, where a
wide range of ability can be expected. Here a test
must be devised that enables the tester to cover
this wide range of ability; hence it must include
some easy to some difficult items – too easy and it
does not enable a range of high ability to be charted
(testees simply getting all the answers right), too
difficult and it does not enable a range of low ability
to be charted (testees simply getting all the answers
wrong). We find out very little about a testee if we
ask a battery of questions which are too easy or too
difficult. Further, it is more efficient and reliable if
atestcanavoidtheproblemforhighabilitytestees
of having to work through a mass of easy items in
order to reach the more difficult items and for low
ability testees of having to try to guess the answers
to more difficult items. Hence it is useful to have
atestthatisflexibleandthatcanbeadaptedto
the testees. For example, if a testee found an item
too hard the next item could adapt to this and be
easier, and, conversely, if a testee was successful
on an item the next item could be harder.
Wainer (1990) indicates that in an adaptive
test the first item is pitched in the middle of
the assumed ability range; if the testee answers it
correctly then it is followed by a more difficult item,
and if the testee answers it incorrectly then it is
followed by an easier item. Computers here provide
an ideal opportunity to address the flexibility,
Chapter 19