RESEARCH METHOD COHEN ok
RESEARCH METHOD COHEN ok RESEARCH METHOD COHEN ok
DEFINING VALIDITY 139 is to abide by central ethical tenets of much research – non-traceability, anonymity and nonidentifiability. Cultural validity A type of validity related to ecological validity is cultural validity (Morgan 1999). This is particularly an issue in cross-cultural, intercultural and comparative kinds of research, where the intention is to shape research so that it is appropriate to the culture of the researched, and where the researcher and the researched are members of different cultures. Cultural validity is defined as ‘the degree to which a study is appropriate to the cultural setting where research is to be carried out’ (Joy 2003: 1). Cultural validity, Morgan (1999) suggests, applies at all stages of the research, and affects its planning, implementation and dissemination. It involves a degree of sensitivity to the participants, cultures and circumstances being studied. Morgan (2005) writes that cultural validity entails an appreciation of the cultural values of those being researched. This could include: understanding possibly different target culture attitudes to research; identifying and understanding salient terms as used in the target culture; reviewing appropriate target language literature; choosing research instruments that are acceptable to the target participants; checking interpretations and translations of data with native speakers; and being aware of one’s own cultural filters as a researcher. (Morgan 2005: 1) Joy (2003: 1) presents twelve important questions that researchers in different cultural contexts may face, to ensure that research is culture-fair and culturally sensitive: Is the research question understandable and of importance to the target group Is the researcher the appropriate person to conduct the research Are the sources of the theories that the research is based on appropriate for the target culture How do researchers in the target culture deal with the issues related to the research question (including their method and findings) Are appropriate gatekeepers and informants chosen Are the research design and research instruments ethical and appropriate according to the standards of the target culture How do members of the target culture define the salient terms of the research Are documents and other information translated in a culturally appropriate way Are the possible results of the research of potential value and benefit to the target culture Does interpretation of the results include the opinions and views of members of the target culture Are the results made available to members of the target culture for review and comment Does the researcher accurately and fairly communicate the results in their cultural context to people who are not members of the target culture Catalytic validity Catalytic validity embraces the paradigm of critical theory discussed in Chapter 1. Put neutrally, catalytic validity simply strives to ensure that research leads to action. However, the story does not end there, for discussions of catalytic validity are substantive; like critical theory, catalytic validity suggests an agenda. Lather (1986, 1991) and Kincheloe and McLaren (1994) suggest that the agenda for catalytic validity is to help participants to understand their worlds in order to transform them. The agenda is explicitly political, for catalytic validity suggests the need to expose whose definitions of the situation are operating in the situation. Lincoln and Guba (1986) suggest that the criterion of ‘fairness’ should be applied to research, meaning that it should not only augment and improve the participants’ experience of the world, but also improve the empowerment of the participants. In this respect the research might focus on what might be (the Chapter 6
140 VALIDITY AND RELIABILITY leading edge of innovations and future trends) and what could be (the ideal, possible futures) (Schofield 1990: 209). Catalytic validity – a major feature in feminist research which, Usher (1996) suggests, needs to permeate all research – requires solidarity in the participants, an ability of the research to promote emancipation, autonomy and freedom within a just, egalitarian and democratic society (Masschelein 1991), to reveal the distortions, ideological deformations and limitations that reside in research, communication and social structures (see also LeCompte and Preissle 1993). Validity, it is argued (Mishler 1990; Scheurich 1996), is no longer an ahistorical given, but contestable, suggesting that the definitions of valid research reside in the academic communities of the powerful. Lather (1986) calls for research to be emancipatory and to empower those who are being researched, suggesting that catalytic validity, akin to Freire’s (1970) notion of ‘conscientization’, should empower participants to understand and transform their oppressed situation. Validity, it is proposed (Scheurich 1996), is but amaskthatinfactpolicesandsetsboundaries to what is considered to be acceptable research by powerful research communities; discourses of validity in reality are discourses of power to define worthwhile knowledge. How defensible it is to suggest that researchers should have such ideological intents is, perhaps, amootpoint,thoughnottoaddressthisareais to perpetuate inequality by omission and neglect. Catalytic validity reasserts the centrality of ethics in the research process, for it requires researchers to interrogate their allegiances, responsibilities and self-interestedness (Burgess 1989). Consequential validity Partially related to catalytic validity is consequential validity, which argues that the ways in which research data are used (the consequences of the research) are in keeping with the capability or intentions of the research, i.e. the consequences of the research do not exceed the capability of the research and the action-related consequences of the research are both legitimate and fulfilled. Clearly, once the research is in the public domain, the researcher has little or no control over the way in which it is used. However, and this is often a political matter, research should not be used in ways in which it was not intended to be used, for example by exceeding the capability of the research data to make claims, by acting on the research in ways that the research does not support (e.g. by using the research for illegitimate epistemic support), by making illegitimate claims by using the research in unacceptable ways (e.g. by selection, distortion) and by not acting on the research in ways that were agreed, i.e. errors of omission and commission. A clear example of consequential validity is formative assessment. This is concerned with the extent to which students improve as a result of feedback given, hence if there is insufficient feedback for students to improve, or if students are unable to improve as a result of – a consequence of – the feedback, then the formative assessment has little consequential validity. Criterion-related validity This form of validity endeavours to relate the results of one particular instrument to another external criterion. Within this type of validity there are two principal forms: predictive validity and concurrent validity. Predictive validity is achieved if the data acquired at the first round of research correlate highly with data acquired at a future date. For example, if the results of examinations taken by 16 year olds correlate highly with the examination results gained by the same students when aged 18, then we might wish to say that the first examination demonstrated strong predictive validity. Avariationonthisthemeisencounteredin the notion of concurrent validity. Todemonstrate this form of validity the data gathered from using one instrument must correlate highly with data gathered from using another instrument. For example, suppose it was decided to research a
- Page 107 and 108: 88 PLANNING EDUCATIONAL RESEARCH Bo
- Page 109 and 110: 90 PLANNING EDUCATIONAL RESEARCH Bo
- Page 111 and 112: 92 PLANNING EDUCATIONAL RESEARCH Bo
- Page 113 and 114: 94 PLANNING EDUCATIONAL RESEARCH Bo
- Page 115 and 116: 96 PLANNING EDUCATIONAL RESEARCH Or
- Page 117 and 118: 98 PLANNING EDUCATIONAL RESEARCH Pa
- Page 119 and 120: 4 Sampling Introduction The quality
- Page 121 and 122: 102 SAMPLING sample of 200 might be
- Page 123 and 124: 104 SAMPLING Box 4.1 Sample size, c
- Page 125 and 126: 106 SAMPLING would be insufficient
- Page 127 and 128: 108 SAMPLING The formula assumes th
- Page 129 and 130: 110 SAMPLING school governors, scho
- Page 131 and 132: 112 SAMPLING terms of sex, a random
- Page 133 and 134: 114 SAMPLING the required sample si
- Page 135 and 136: 116 SAMPLING Snowball sampling In s
- Page 137 and 138: 118 SAMPLING the kind of sample (d
- Page 139 and 140: 120 SENSITIVE EDUCATIONAL RESEARCH
- Page 141 and 142: 122 SENSITIVE EDUCATIONAL RESEARCH
- Page 143 and 144: 124 SENSITIVE EDUCATIONAL RESEARCH
- Page 145 and 146: 126 SENSITIVE EDUCATIONAL RESEARCH
- Page 147 and 148: 128 SENSITIVE EDUCATIONAL RESEARCH
- Page 149 and 150: 130 SENSITIVE EDUCATIONAL RESEARCH
- Page 151 and 152: 132 SENSITIVE EDUCATIONAL RESEARCH
- Page 153 and 154: 134 VALIDITY AND RELIABILITY It is
- Page 155 and 156: 136 VALIDITY AND RELIABILITY using
- Page 157: 138 VALIDITY AND RELIABILITY includ
- Page 161 and 162: 142 VALIDITY AND RELIABILITY social
- Page 163 and 164: 144 VALIDITY AND RELIABILITY this i
- Page 165 and 166: 146 VALIDITY AND RELIABILITY prese
- Page 167 and 168: 148 VALIDITY AND RELIABILITY by ass
- Page 169 and 170: 150 VALIDITY AND RELIABILITY Validi
- Page 171 and 172: 152 VALIDITY AND RELIABILITY typica
- Page 173 and 174: 154 VALIDITY AND RELIABILITY people
- Page 175 and 176: 156 VALIDITY AND RELIABILITY
- Page 177 and 178: 158 VALIDITY AND RELIABILITY sensit
- Page 179 and 180: 160 VALIDITY AND RELIABILITY certif
- Page 181 and 182: 162 VALIDITY AND RELIABILITY how m
- Page 183 and 184: 164 VALIDITY AND RELIABILITY operat
- Page 186 and 187: 7 Naturalistic and ethnographic res
- Page 188 and 189: ELEMENTS OF NATURALISTIC INQUIRY 16
- Page 190 and 191: PLANNING NATURALISTIC RESEARCH 171
- Page 192 and 193: PLANNING NATURALISTIC RESEARCH 173
- Page 194 and 195: PLANNING NATURALISTIC RESEARCH 175
- Page 196 and 197: PLANNING NATURALISTIC RESEARCH 177
- Page 198 and 199: PLANNING NATURALISTIC RESEARCH 179
- Page 200 and 201: PLANNING NATURALISTIC RESEARCH 181
- Page 202 and 203: PLANNING NATURALISTIC RESEARCH 183
- Page 204 and 205: PLANNING NATURALISTIC RESEARCH 185
- Page 206 and 207: CRITICAL ETHNOGRAPHY 187 Relatio
DEFINING VALIDITY 139<br />
is to abide by central ethical tenets of much<br />
research – non-traceability, anonymity and nonidentifiability.<br />
Cultural validity<br />
A type of validity related to ecological validity<br />
is cultural validity (Morgan 1999). This is<br />
particularly an issue in cross-cultural, intercultural<br />
and comparative kinds of research, where the<br />
intention is to shape research so that it is<br />
appropriate to the culture of the researched,<br />
and where the researcher and the researched are<br />
members of different cultures. Cultural validity<br />
is defined as ‘the degree to which a study is<br />
appropriate to the cultural setting where research<br />
is to be carried out’ (Joy 2003: 1). Cultural<br />
validity, Morgan (1999) suggests, applies at all<br />
stages of the research, and affects its planning,<br />
implementation and dissemination. It involves a<br />
degree of sensitivity to the participants, cultures<br />
and circumstances being studied. Morgan (2005)<br />
writes that<br />
cultural validity entails an appreciation of the<br />
cultural values of those being researched. This<br />
could include: understanding possibly different<br />
target culture attitudes to research; identifying<br />
and understanding salient terms as used in the<br />
target culture; reviewing appropriate target language<br />
literature; choosing research instruments that are<br />
acceptable to the target participants; checking<br />
interpretations and translations of data with native<br />
speakers; and being aware of one’s own cultural filters<br />
as a researcher.<br />
(Morgan 2005: 1)<br />
Joy (2003: 1) presents twelve important questions<br />
that researchers in different cultural contexts<br />
may face, to ensure that research is culture-fair and<br />
culturally sensitive:<br />
<br />
<br />
<br />
Is the research question understandable and of<br />
importance to the target group<br />
Is the researcher the appropriate person to<br />
conduct the research<br />
Are the sources of the theories that the research<br />
is based on appropriate for the target culture<br />
<br />
<br />
How do researchers in the target culture deal<br />
with the issues related to the research question<br />
(including their method and findings)<br />
Are appropriate gatekeepers and informants<br />
chosen<br />
Are the research design and research<br />
instruments ethical and appropriate according<br />
to the standards of the target culture<br />
<br />
<br />
<br />
<br />
<br />
<br />
How do members of the target culture define<br />
the salient terms of the research<br />
Are documents and other information translated<br />
in a culturally appropriate way<br />
Are the possible results of the research of<br />
potential value and benefit to the target<br />
culture<br />
Does interpretation of the results include the<br />
opinions and views of members of the target<br />
culture<br />
Are the results made available to members of<br />
the target culture for review and comment<br />
Does the researcher accurately and fairly<br />
communicate the results in their cultural<br />
context to people who are not members of<br />
the target culture<br />
Catalytic validity<br />
Catalytic validity embraces the paradigm of critical<br />
theory discussed in Chapter 1. Put neutrally,<br />
catalytic validity simply strives to ensure that<br />
research leads to action. However, the story does<br />
not end there, for discussions of catalytic validity<br />
are substantive; like critical theory, catalytic<br />
validity suggests an agenda. Lather (1986, 1991)<br />
and Kincheloe and McLaren (1994) suggest that<br />
the agenda for catalytic validity is to help<br />
participants to understand their worlds in order<br />
to transform them. The agenda is explicitly<br />
political, for catalytic validity suggests the need<br />
to expose whose definitions of the situation are<br />
operating in the situation. Lincoln and Guba<br />
(1986) suggest that the criterion of ‘fairness’ should<br />
be applied to research, meaning that it should<br />
not only augment and improve the participants’<br />
experience of the world, but also improve the<br />
empowerment of the participants. In this respect<br />
the research might focus on what might be (the<br />
Chapter 6