RESEARCH METHOD COHEN ok

426 TESTS
0.85), negative correlations between items 9 and
10 and all the other items (e.g. −0.79), and a very
low intercorrelation between items 6 and 8 and all
the others (e.g. 0.26). Item-to-total correlations
(one kind of item analysis in which the item in
question is correlated with the sum of the other
items) vary here. What do you do You can retain
items 1–5. For items 9 and 10 you can reverse the
scoring (as these items looked at positive rather
than negative aspects), and for items 6 and 8 you
can consider excluding them from the test, as they
appear to be measuring something else. Such item
analysis is designed to include items that measure
the same construct and to exclude items that do
not. We refer readers to Howitt and Cramer (2005:
Ch. 12) for further discussion of this.
An alternative approach to deciding which
items to retain or exclude from the list of ten items
above is to use factor analysis (see Chapter 25), a
method facilitated greatly by SPSS. Factor analysis
will group together a cluster of similar items
and keep that cluster separate from clusters of
other items. So, for our example above, the factor
analysis could have found, by way of illustration,
three factors:
positive feelings (items 9 and 10)
negative psychological states (items 2, 3, 4,
5, 7)
physical, behavioural changes (items 1, 6, 8).
By looking at the factor loadings (see Chapter 25)
the researcher would have to decide which
were the most appropriate factors to retain, and,
thereby, which items to include and exclude. As
a general rule, items with low factor loadings
(e.g. ≤ 0.3) should be considered for exclusion, as
they do not contribute sufficiently to the factor.
Factor analysis will indicate, also, whether the
construct is unidimensional or multidimensional
(if there is only one factor it is probably
unidimensional).
Consider the form of the test
Much of the discussion in this chapter
assumes that the test is of the pen-andpaper
variety. Clearly this need not be the
case; for example, tests can be written, oral,
practical, interactive, computer-based, dramatic,
diagrammatic, pictorial, photographic, involve the
use of audio and video material, presentational and
role-play, simulations. Oral tests, for example, can
be conducted if the researcher feels that reading
and writing will obstruct the true purpose of
the test (i.e. it becomes a reading and writing
test rather than, say, a test of mathematics).
This does not negate the issues discussed in this
chapter, for the form of the test will still need
to consider, for example, reliability and validity,
difficulty, discriminability, marking and grading,
item analysis, timing. Indeed several of these
factors take on an added significance in nonwritten
forms of testing; for example, reliability is a
major issue in judging live musical performance or
the performance of a gymnastics routine – where a
‘one-off’ event is likely. Furthermore, reliability
and validity are significant issues in group
performance or group exercises – where group
dynamics may prevent a testee’s true abilities
from being demonstrated. Clearly the researcher
will need to consider whether the test will be
undertaken individually, or in a group, and what
form it will take.
Write the test item
The test will need to address the intended and
unintended clues and cues that might be provided
in it, for example (Morris et al.1987):
The number of blanks might indicate the
number of words required.
The number of dots might indicate the number
of letters required.
The length of blanks might indicate the length
of response required.
The space left for completion will give cues
about how much to write.
Blanks in different parts of a sentence will
be assisted by the reader having read the
other parts of the sentence (anaphoric and
cataphoric reading cues).
Hanna (1993: 139–41) and Cunningham
(1998) provide several guidelines for constructing