RESEARCH METHOD COHEN ok

CONSTRUCTING A TEST 419
achievement of sections of the programme,
and to diagnose strengths and weaknesses. It is
typically criterion-referenced.
Diagnostic testing is an in-depth test to discover
particular strengths, weaknesses and difficulties
that a student is experiencing, and is designed
to expose causes and specific areas of weakness
or strength. This often requires the test to
include several items about the same feature,
so that, for example, several types of difficulty
in a student’s understanding will be exposed;
the diagnostic test will need to construct test
items that will focus on each of a range of
very specific difficulties that students might be
experiencing, in order to identify the exact
problems that they are having from a range of
possible problems. Clearly this type of test is
criterion-referenced.
Summative testing is the test given at the
end of the programme, and is designed
to measure achievement, outcomes, or
‘mastery’. This might be criterion-referenced
or norm-referenced, depending to some extent
on the use to which the results will be put
(e.g. to award certificates or grades, to identify
achievement of specific objectives).
Identify the test specifications
The test specifications include:
which programme objectives and student
learning outcomes will be addressed
which content areas will be addressed
the relative weightings, balance and coverage
of items
the total number of items in the test
the number of questions required to address a
particular element of a programme or learning
outcomes
the exact items in the test.
To ensure validity in a test it is essential to ensure
that the objectives of the test are fairly addressed
in the test items. Objectives, it is argued (Mager
1962; Wiles and Bondi 1984), should
be specific and be expressed with an appropriate
degree of precision
represent intended learning outcomes
identify the actual and observable behaviour
that will demonstrate achievement
include an active verb
be unitary (focusing on one item per objective).
One way of ensuring that the objectives are fairly
addressed in test items can be done through a
matrix frame that indicates the coverage of content
areas, the coverage of objectives of the programme,
and the relative weighting of the items on the test.
Such a matrix is set out in Box 19.1 taking the
example from a secondary school history syllabus.
Box 19.1 indicates the main areas of the
programme to be covered in the test (content areas);
then it indicates which objectives or detailed
content areas will be covered (1a–3c) – these
numbers refer to the identified specifications in the
Chapter 19
Box 19.1
Amatrixoftestitems
Content areas Objective/area of Objective/area of Objective/area
programme content programme content programme content
Aspects of the Second World War 1a 1b 1c 2a 2b 2c 3a 3b 3c Total
The build-up to the Second World
War 1 2 2 1 1 1 1 1 10
The invasion of Poland 2 1 1 3 2 2 3 3 3 20
The invasion of France 3 4 5 4 4 3 4 4 4 35
The Allied invasion 3 2 3 3 4 3 3 2 2 25
The end of the conflict 2 1 1 1 1 2 2 10
Total 11 10 9 13 12 10 13 12 10 100