RESEARCH METHOD COHEN ok

CONSTRUCTING A TEST 427
short-answer items to overcome some of these
problems:
Make the blanks close to the end of the
sentence.
Keep the blanks the same length.
Ensure that there can be only a single correct
answer.
Avoid putting several blanks close to each
other (in a sentence or paragraph) such that
the overall meaning is obscured.
Only make blanks of key words or concepts,
rather than of trivial words.
Avoid addressing only trivial matters.
Ensure that students know exactly the kind
and specificity of the answer required.
Specify the units in which a numerical answer
is to be given.
Use short-answers for testing knowledge recall.
With regard to multiple choice items there are
several potential problems:
the number of choices in a single multiple
choice item and whether there is one or more
right answer(s)
the number and realism of the distractors in a
multiple choice item (e.g. there might be many
distractors but many of them are too obvious
to be chosen – there may be several redundant
items)
the sequence of items and their effects on each
other
the location of the correct response(s) in a
multiple choice item.
Gronlund and Linn (1990), Hanna (1993:
161–75), Cunningham (1998) and Aiken (2003)
set out several suggestions for constructing
effective multiple choice test items:
ensure that they catch significant knowledge
and learning rather than low-level recall of
facts
frame the nature of the issue in the stem of the
item, ensuring that the stem is meaningful
in itself (e.g. replace the general ‘sheep’:
(a) are graminivorous, (b) are cloven footed,
(c) usually give birth to one or two calves at a
time’ with ‘how many lambs are normally born
to a sheep at one time’)
ensure that the stem includes as much of the
item as possible, with no irrelevancies
avoid negative stems to the item
keep the readability levels low
ensure clarity and unambiguity
ensure that all the options are plausible so that
guessing of the only possible option is avoided
avoid the possibility of students making the
correct choice through incorrect reasoning
include some novelty to the item if it is being
used to measure understanding
ensure that there can only be a single correct
option (if a single answer is required) and that
it is unambiguously the right response
avoid syntactical and grammatical clues
by making all options syntactically and
grammatically parallel and by avoiding
matching the phrasing of a stem with similar
phrasing in the response
avoid including in the stem clues as to which
may be the correct response
ensure that the length of each response item is
the same (e.g. to avoid one long correct answer
from standing out)
keep each option separate, avoiding options
which are included in each other
ensure that the correct option is positioned
differently for each item (e.g. so that it is not
always option 2)
avoid using options like ‘all of the above’ or
‘none of the above’
avoid answers from one item being used to cue
answers to another item – keep items separate.
The response categories of tests need to be
considered, and we refer readers to our discussion
of this topic in Chapter 15 on questionnaires (e.g.
Likert scales, Guttman scales, semantic differential
scales, Thurstone scales).
Morris et al. (1987: 161), Gronlund and
Linn (1990), Hanna (1993: 147), Cunningham
(1998) and Aiken (2003) also indicate particular
problems in true–false questions:
ambiguity of meaning
some items might be partly true or partly false
Chapter 19