RESEARCH METHOD COHEN ok
RESEARCH METHOD COHEN ok RESEARCH METHOD COHEN ok
DOCUMENTARY RESEARCH 203 What are you, the reader/researcher bringing to the document in trying to make sense of it What alternative interpretations of the document are possible and tenable How is the chosen interpretation justified What are the problems of reliability and validity in your reading of the document What is the place of the document in the overall research project Questions are being raised here about the reliability and validity of the documents (see http:// www.routledge.com/textbooks/9780415368780 – Chapter 8, file 8.3. ppt). They are social products, located in specific contexts, and, as such, have to be interrogated and interpreted rather than simply accepted. They are often selective, deliberately excluding certain details or information and serving purposes and audiences other than the researcher. Documents lie on several continua, for example: Formal/official ↔ Informal/lay Published ↔ Unpublished Public domain ↔ Private papers Anonymous ↔ Authored Facts ↔ Beliefs Professional ↔ Lay For circulation ↔ Not for circulation Placing documents along these several continua can assist the researcher in answering the preceding long list of questions. Reliability and validity in documentary analysis Validity may be strong in first person documents or in documents that were written for a specific purpose (Bailey 1994: 317). However, that purpose may not coincide with that of research, thereby undermining its validity for research purposes. We mentioned earlier the problem of bias, selectivity, being written for an audience and purposes different from those of the researcher, attrition and selective survival; all these undermine validity. In historical research great care is paid to authenticity and provenance, and documents may be subject to chemical analysis here (e.g. of inks, paper, parchment and so on) in order to detect forgeries. Bailey (1994: 318) suggests that face validity and construct validity in documents may be stronger and more sufficient than other forms of validity, though corroboration with other documents should be undertaken wherever possible. With regard to reliability, while subjectivity may feature highly in certain documents, reliability by corroboration may also be pursued. The standards and criteria of reliability have to be declared by the researcher. Scott (1990) suggests four criteria for validity and reliability in using documents: authenticity; credibility (including accuracy, legitimacy and sincerity); representativeness (including availability and which documents have survived the passage of time); and meaning (actual and interpreted). It is often difficult to disentangle fact from interpretation in a document and the research that is conducted using it (see http://www. routledge.com/textbooks/9780415368780 – Chapter 8, file 8.4. ppt). Understanding documents is a hermeneutic exercise, at several stages. Giddens (1979) remarked that researchers have to live with a ‘double hermeneutic’, that is, they interpret a world that is already interpreted by the participants, a pre-interpreted world. Actors or participants interpret or ascribe meaning to the world and then the researcher interprets or ascribes meaning to these interpretations. However, for the user of documents, the matter extends further. Documents record live events, so written data on social events become second hand because they translate the researcher’s/writer’s interpretation/inference of the world into another medium – from action to writing: a triple hermeneutic. Documents are part of the world and the action on which they are commenting. Then the reader places his or her interpretation/inference on the document, a quadruple hermeneutic. At each of these four stages interpretation, inference and bias and, thereby, unreliability could enter the scene. As Connelly and Clandinin (1997: 84) remark, converting field text into a research text is a process of (increasing) interpretation. Field texts and documents, they Chapter 8
204 HISTORICAL AND DOCUMENTARY RESEARCH suggest, are close to the actual experience under study, whereas research texts are at a remove; that distance lets in unreliability and invalidity. While acknowledgement of this by the researcher, and the researcher’s identification of the criteria for judging the research, may go some way towards addressing this issue (i.e. reflexivity), nevertheless it may not solve the problem, only articulate it. The issue is that the researcher has to exercise extreme caution in using documents. As well as having a life of their own, documents are interpretations of events. As Prior (2003: 26) suggests, the content of documents may not be the most important feature of the document, and documents are ‘situated products’. They are the drivers, media (channels), mediators (filters) and outcomes of social interaction (a clear exemplification of Giddens’ theory of structuration). Understanding their context is crucial to understanding the document. Documents are multilevelled and have to be interpreted at their many levels; they need to be contextualized. For a detailed analysis of several aspects of documents in research we refer readers to Prior (2003). For examples of documents, we refer the reader to the accompanying web site pages (http:// www.routledge.com/textbooks/9780415368780 – Chapter 8, file 8.1.doc; http://www.routledge. com/textbooks/9780415368780 – Chapter 8, file 8.2.doc).
- Page 171 and 172: 152 VALIDITY AND RELIABILITY typica
- Page 173 and 174: 154 VALIDITY AND RELIABILITY people
- Page 175 and 176: 156 VALIDITY AND RELIABILITY
- Page 177 and 178: 158 VALIDITY AND RELIABILITY sensit
- Page 179 and 180: 160 VALIDITY AND RELIABILITY certif
- Page 181 and 182: 162 VALIDITY AND RELIABILITY how m
- Page 183 and 184: 164 VALIDITY AND RELIABILITY operat
- Page 186 and 187: 7 Naturalistic and ethnographic res
- Page 188 and 189: ELEMENTS OF NATURALISTIC INQUIRY 16
- Page 190 and 191: PLANNING NATURALISTIC RESEARCH 171
- Page 192 and 193: PLANNING NATURALISTIC RESEARCH 173
- Page 194 and 195: PLANNING NATURALISTIC RESEARCH 175
- Page 196 and 197: PLANNING NATURALISTIC RESEARCH 177
- Page 198 and 199: PLANNING NATURALISTIC RESEARCH 179
- Page 200 and 201: PLANNING NATURALISTIC RESEARCH 181
- Page 202 and 203: PLANNING NATURALISTIC RESEARCH 183
- Page 204 and 205: PLANNING NATURALISTIC RESEARCH 185
- Page 206 and 207: CRITICAL ETHNOGRAPHY 187 Relatio
- Page 208 and 209: SOME PROBLEMS WITH ETHNOGRAPHIC AND
- Page 210 and 211: 8 Historical and documentary resear
- Page 212 and 213: DATA COLLECTION 193 One can see fro
- Page 214 and 215: WRITING THE RESEARCH REPORT 195 Ext
- Page 216 and 217: THE USE OF QUANTITATIVE METHODS 197
- Page 218 and 219: LIFE HISTORIES 199 Box 8.2 Atypolog
- Page 220 and 221: DOCUMENTARY RESEARCH 201 Documentar
- Page 224 and 225: 9 Surveys, longitudinal, cross-sect
- Page 226 and 227: SOME PRELIMINARY CONSIDERATIONS 207
- Page 228 and 229: PLANNING A SURVEY 209 structured or
- Page 230 and 231: LONGITUDINAL, CROSS-SECTIONAL AND T
- Page 232 and 233: LONGITUDINAL, CROSS-SECTIONAL AND T
- Page 234 and 235: STRENGTHS AND WEAKNESSES OF LONGITU
- Page 236 and 237: STRENGTHS AND WEAKNESSES OF LONGITU
- Page 238 and 239: POSTAL, INTERVIEW AND TELEPHONE SUR
- Page 240 and 241: POSTAL, INTERVIEW AND TELEPHONE SUR
- Page 242 and 243: POSTAL, INTERVIEW AND TELEPHONE SUR
- Page 244 and 245: EVENT HISTORY ANALYSIS 225 may be p
- Page 246 and 247: INTERNET-BASED SURVEYS 227 packages
- Page 248 and 249: INTERNET-BASED SURVEYS 229 instruc
- Page 250 and 251: INTERNET-BASED SURVEYS 231 Box 10.1
- Page 252 and 253: INTERNET-BASED SURVEYS 233 Box 10.1
- Page 254 and 255: INTERNET-BASED SURVEYS 235 Box 10.1
- Page 256 and 257: INTERNET-BASED SURVEYS 237 Witte et
- Page 258 and 259: INTERNET-BASED EXPERIMENTS 239 requ
- Page 260 and 261: INTERNET-BASED INTERVIEWS 241 ‘ne
- Page 262 and 263: SEARCHING FOR RESEARCH MATERIALS ON
- Page 264 and 265: COMPUTER SIMULATIONS 245 autho
- Page 266 and 267: COMPUTER SIMULATIONS 247 computer s
- Page 268 and 269: COMPUTER SIMULATIONS 249 On the oth
- Page 270 and 271: GEOGRAPHICAL INFORMATION SYSTEMS 25
DOCUMENTARY <strong>RESEARCH</strong> 203<br />
What are you, the reader/researcher bringing<br />
to the document in trying to make sense of it<br />
What alternative interpretations of the<br />
document are possible and tenable How is<br />
the chosen interpretation justified<br />
What are the problems of reliability and<br />
validity in your reading of the document<br />
What is the place of the document in the<br />
overall research project<br />
Questions are being raised here about the reliability<br />
and validity of the documents (see http://<br />
www.routledge.com/textbo<strong>ok</strong>s/9780415368780 –<br />
Chapter 8, file 8.3. ppt). They are social products,<br />
located in specific contexts, and, as such, have<br />
to be interrogated and interpreted rather than<br />
simply accepted. They are often selective, deliberately<br />
excluding certain details or information<br />
and serving purposes and audiences other than the<br />
researcher. Documents lie on several continua, for<br />
example:<br />
Formal/official ↔ Informal/lay<br />
Published ↔ Unpublished<br />
Public domain ↔ Private papers<br />
Anonymous ↔ Authored<br />
Facts ↔ Beliefs<br />
Professional ↔ Lay<br />
For circulation ↔ Not for circulation<br />
Placing documents along these several continua<br />
can assist the researcher in answering the<br />
preceding long list of questions.<br />
Reliability and validity in documentary<br />
analysis<br />
Validity may be strong in first person documents<br />
or in documents that were written for a specific<br />
purpose (Bailey 1994: 317). However, that purpose<br />
may not coincide with that of research, thereby<br />
undermining its validity for research purposes. We<br />
mentioned earlier the problem of bias, selectivity,<br />
being written for an audience and purposes<br />
different from those of the researcher, attrition and<br />
selective survival; all these undermine validity.<br />
In historical research great care is paid to<br />
authenticity and provenance, and documents may<br />
be subject to chemical analysis here (e.g. of<br />
inks, paper, parchment and so on) in order to<br />
detect forgeries. Bailey (1994: 318) suggests that<br />
face validity and construct validity in documents<br />
may be stronger and more sufficient than other<br />
forms of validity, though corroboration with<br />
other documents should be undertaken wherever<br />
possible.<br />
With regard to reliability, while subjectivity may<br />
feature highly in certain documents, reliability<br />
by corroboration may also be pursued. The<br />
standards and criteria of reliability have to<br />
be declared by the researcher. Scott (1990)<br />
suggests four criteria for validity and reliability<br />
in using documents: authenticity; credibility<br />
(including accuracy, legitimacy and sincerity);<br />
representativeness (including availability and<br />
which documents have survived the passage of<br />
time); and meaning (actual and interpreted).<br />
It is often difficult to disentangle fact from<br />
interpretation in a document and the research<br />
that is conducted using it (see http://www.<br />
routledge.com/textbo<strong>ok</strong>s/9780415368780 –<br />
Chapter 8, file 8.4. ppt). Understanding documents<br />
is a hermeneutic exercise, at several<br />
stages. Giddens (1979) remarked that researchers<br />
have to live with a ‘double hermeneutic’, that is,<br />
they interpret a world that is already interpreted<br />
by the participants, a pre-interpreted world. Actors<br />
or participants interpret or ascribe meaning to<br />
the world and then the researcher interprets or ascribes<br />
meaning to these interpretations. However,<br />
for the user of documents, the matter extends<br />
further. Documents record live events, so written<br />
data on social events become second hand<br />
because they translate the researcher’s/writer’s<br />
interpretation/inference of the world into another<br />
medium – from action to writing: a triple<br />
hermeneutic. Documents are part of the world<br />
and the action on which they are commenting.<br />
Then the reader places his or her interpretation/inference<br />
on the document, a quadruple<br />
hermeneutic. At each of these four stages interpretation,<br />
inference and bias and, thereby,<br />
unreliability could enter the scene. As Connelly<br />
and Clandinin (1997: 84) remark, converting field<br />
text into a research text is a process of (increasing)<br />
interpretation. Field texts and documents, they<br />
Chapter 8