RESEARCH METHOD COHEN ok

432 TESTS
precision of detail required. For a full analysis of
determining cut-off grades see Linn (1993).
The issue of scoring takes in a range of
factors, for example: grade norms, age norms,
percentile norms and standard score norms (e.g.
z-scores, T-scores, stanine scores, percentiles).
These are beyond the scope of this book to
discuss, but readers are referred to Cronbach
(1970), Gronlund and Linn (1990), Cohen and
Holliday (1996), Hopkins et al.(1996).
Devising a pretest and post-test
The construction and administration of tests is
an essential part of the experimental model of
research, where a pretest and a post-test have to be
devised for the control and experimental groups.
The pretest and post-test must adhere to several
guidelines:
The pretest may have questions which differ in
form or wording from the post-test, though the
two tests must test the same content, i.e. they
will be alternate forms of a test for the same
groups.
The pretest must be the same for the control
and experimental groups.
The post-test must be the same for both groups.
Care must be taken in the construction of a
post-test to avoid making the test easier to
complete by one group than another.
The level of difficulty must be the same in both
tests.
Test data feature centrally in the experimental
model of research; additionally they may feature as
part of a questionnaire, interview and documentary
material.
Reliability and validity of tests
Chapter 6 covers issues of reliability and validity.
Suffice it here to say that reliability concerns
the degree of confidence that can be placed
in the results and the data, which is often a
matter of statistical calculation and subsequent
test redesigning. Validity, on the other hand,
concerns the extent to which the test tests what
it is supposed to test. This devolves on content,
construct, face, criterion-related and concurrent
validity.
Ethical issues in preparing for tests
Amajorsourceofunreliabilityoftestdataderives
from the extent and ways in which students have
been prepared for the test. These can be located on
acontinuumfromdirectandspecificpreparation,
through indirect and general preparation, to no
preparation at all. With the growing demand
for test data (e.g. for selection, for certification,
for grading, for employment, for tracking, for
entry to higher education, for accountability, for
judging schools and teachers) there is a perhaps
understandable pressure to prepare students for
tests. This is the ‘high-stakes’ aspect of testing
(Harlen 1994), where much hinges on the test
results. At one level this can be seen in the
backwash effect of examinations on curricula and
syllabuses; at another level it can lead to the direct
preparation of students for specific examinations.
Preparation can take many forms (Mehrens and
Kaminski 1989; Gipps 1994):
ensuring coverage, among other programme
contents and objectives, of the objectives and
programme that will be tested
restricting the coverage of the programme
content and objectives to only those that will
be tested
preparing students with ‘exam technique’
practising with past or similar papers
directly matching the teaching to specific
test items, where each piece of teaching and
contents is the same as each test item
practising on an exactly parallel form of the
test
telling students in advance what will appear
on the test
practising on and preparing the identical test
itself (e.g. giving out test papers in advance)
without teacher input
practising on and preparing the identical
test itself (e.g. giving out the test papers in
advance), with the teacher working through
the items, maybe providing sample answers.