RESEARCH METHOD COHEN ok

342 QUESTIONNAIRES
The above outline describes a particular kind
of pilot: one that does not focus on data, but on
matters of coverage and format, gaining feedback
from a limited number of respondents and experts
on the items set out above.
There is a second type of pilot. This is one
which starts with a long list of items and, through
statistical analysis and feedback, reduces those
items (Kgaile and Morrison 2006). For example,
a researcher may generate an initial list of,
for example, 120 items to be included in a
questionnaire, and wish to know which items
to excise. A pilot is conducted on a sizeable
and representative number of respondents (e.g.
50–100) and this generates real data – numerical
responses. These data can be analysed for the
following factors:
Reliability: those items with low reliability
(Cronbach’s alpha for internal consistency: see
Part Five) can be removed.
Collinearity: if items correlate very strongly
with others then a decision can be taken
to remove one or more of them, provided,
of course, that this does not result in the
loss of important areas of the research (i.e.
human judgement would have to prevail over
statistical analysis).
Multiple regression: those items with low betas
(see Part Five) can be removed, provided,
of course, that this does not result in the
loss of important areas of the research (i.e.
human judgement would have to prevail over
statistical analysis).
Factor analysis: to identify clusters of key
variables and to identify redundant items (see
Part Five).
As a result of such analysis, the items for
removal can be identified, and this can result
in a questionnaire of manageable proportions. It is
important to have a good-sized and representative
sample here in order to generate reliable data
for statistical analysis; too few respondents to this
type of pilot and this may result in important items
being excluded from the final questionnaire.
Practical considerations in questionnaire
design
Taking the issues discussed so far in questionnaire
design, a range of practical implications for
designing a questionnaire can be highlighted:
Operationalize the purposes of the questionnaire
carefully.
Be prepared to have a pre-pilot to generate
items for a pilot questionnaire, and then be
ready to modify the pilot questionnaire for the
final version.
If the pilot includes many items, and the
intention is to reduce the number of items
through statistical analysis or feedback, then
be prepared to have a second round of piloting,
after the first pilot has been modified.
Decide on the most appropriate type of
question – dichotomous, multiple choice, rank
orderings, rating scales, constant sum, ratio,
closed, open.
Ensure that every issue has been explored
exhaustively and comprehensively; decide on
the content and explore it in depth and
breadth.
Use several items to measure a specific
attribute, concept or issue.
Ensure that the data acquired will answer the
research questions.
Ask more closed than open questions for ease
of analysis (particularly in a large sample).
Balance comprehensiveness and exhaustive
coverage of issues with the demotivating factor
of having respondents complete several pages
of a questionnaire.
Ask only one thing at a time in a question. Use
single sentences per item wherever possible.
Keep response categories simple.
Avoid jargon.
Keep statements in the present tense wherever
possible.
Strive to be unambiguous and clear in the
wording.
Be simple, clear and brief wherever possible.
Clarify the kinds of responses required in open
questions.