Qualitative_data_analysis

PRODUCING AN ACCOUNT 259
the vibrations of a quartz crystal. Reliability is not primarily an empirical issue at all,
but a conceptual one. It has to be rooted in a conceptual framework which explains
why in principle we can expect a measuring instrument to produce reliable results.
The empirical aspect comes later, when we check repeatedly to see whether in
practice these results are achieved. Here, in the absence of repeated observations, all
we can do is check that our ‘apparatus’ is in good working order.
If we cannot expect others to replicate our account, the best we can do is explain
how we arrived at our results. This gives our audience the chance to scrutinise our
procedures and to decide whether, at least in principle, the results ought to be
reliable. The crucial procedures will be those we have followed in categorizing or
linking data, but we may also explain the procedures we have followed in
summarizing data, splitting and splicing categories, making connections and using
our maps and matrices. It may be useful to lay these out in algorithmic form
(Figure 15.6).
Figure 15.7 shows how this can be done for the overall decision-making process
we have followed throughout our analysis. Using a similar approach, we can also
detail the substantive decision-making which has governed our conceptualizations
of the data, and the connections we have made between concepts. We can improve
internal reliability by ensuring our conceptualizations relate closely to our data, by
testing them against a variety of data sources.
As well as outlining our procedures, we can try to identify possible sources of error.
For example, what about ‘mistakes’ in categorization? As the assignment of
categories involves judging in terms of a range of criteria, which may be vaguely
articulated and which may also change over time, the opportunity for error is
obvious. Some data which should be assigned to a particular category may have been
overlooked. Other data have been assigned inappropriately. We could call these the
sins of commission and ommission. Similar sources of error can be found in linking
data and looking at connections between categories. There are procedures for
reducing such errors, for example, through repeating the process of categorizing the
data. The computer through its search facilities can help to locate data which may
have been overlooked in our initial categorization. The converse error, of assigning a
category (or a link) mistakenly to the data, can be identified through critical
assessment of the results of category retrievals.
We may correct such errors, but if time does not permit a full-blooded
reassessment, we may settle for an estimate of the degree of error, by taking a sample
of categories or databits and checking the accuracy with which categories have been
assigned.
In assessing the potential for error, we can also call upon the procedures for
corroborating evidence which are relevant in identifying sources of error. In looking
at the quality, distribution and weight of evidence under pinning our