RESEARCH METHOD COHEN ok

RESEARCH METHOD COHEN ok RESEARCH METHOD COHEN ok

12.01.2015 Views

DEFINING VALIDITY 139 is to abide by central ethical tenets of much research – non-traceability, anonymity and nonidentifiability. Cultural validity A type of validity related to ecological validity is cultural validity (Morgan 1999). This is particularly an issue in cross-cultural, intercultural and comparative kinds of research, where the intention is to shape research so that it is appropriate to the culture of the researched, and where the researcher and the researched are members of different cultures. Cultural validity is defined as ‘the degree to which a study is appropriate to the cultural setting where research is to be carried out’ (Joy 2003: 1). Cultural validity, Morgan (1999) suggests, applies at all stages of the research, and affects its planning, implementation and dissemination. It involves a degree of sensitivity to the participants, cultures and circumstances being studied. Morgan (2005) writes that cultural validity entails an appreciation of the cultural values of those being researched. This could include: understanding possibly different target culture attitudes to research; identifying and understanding salient terms as used in the target culture; reviewing appropriate target language literature; choosing research instruments that are acceptable to the target participants; checking interpretations and translations of data with native speakers; and being aware of one’s own cultural filters as a researcher. (Morgan 2005: 1) Joy (2003: 1) presents twelve important questions that researchers in different cultural contexts may face, to ensure that research is culture-fair and culturally sensitive: Is the research question understandable and of importance to the target group Is the researcher the appropriate person to conduct the research Are the sources of the theories that the research is based on appropriate for the target culture How do researchers in the target culture deal with the issues related to the research question (including their method and findings) Are appropriate gatekeepers and informants chosen Are the research design and research instruments ethical and appropriate according to the standards of the target culture How do members of the target culture define the salient terms of the research Are documents and other information translated in a culturally appropriate way Are the possible results of the research of potential value and benefit to the target culture Does interpretation of the results include the opinions and views of members of the target culture Are the results made available to members of the target culture for review and comment Does the researcher accurately and fairly communicate the results in their cultural context to people who are not members of the target culture Catalytic validity Catalytic validity embraces the paradigm of critical theory discussed in Chapter 1. Put neutrally, catalytic validity simply strives to ensure that research leads to action. However, the story does not end there, for discussions of catalytic validity are substantive; like critical theory, catalytic validity suggests an agenda. Lather (1986, 1991) and Kincheloe and McLaren (1994) suggest that the agenda for catalytic validity is to help participants to understand their worlds in order to transform them. The agenda is explicitly political, for catalytic validity suggests the need to expose whose definitions of the situation are operating in the situation. Lincoln and Guba (1986) suggest that the criterion of ‘fairness’ should be applied to research, meaning that it should not only augment and improve the participants’ experience of the world, but also improve the empowerment of the participants. In this respect the research might focus on what might be (the Chapter 6

140 VALIDITY AND RELIABILITY leading edge of innovations and future trends) and what could be (the ideal, possible futures) (Schofield 1990: 209). Catalytic validity – a major feature in feminist research which, Usher (1996) suggests, needs to permeate all research – requires solidarity in the participants, an ability of the research to promote emancipation, autonomy and freedom within a just, egalitarian and democratic society (Masschelein 1991), to reveal the distortions, ideological deformations and limitations that reside in research, communication and social structures (see also LeCompte and Preissle 1993). Validity, it is argued (Mishler 1990; Scheurich 1996), is no longer an ahistorical given, but contestable, suggesting that the definitions of valid research reside in the academic communities of the powerful. Lather (1986) calls for research to be emancipatory and to empower those who are being researched, suggesting that catalytic validity, akin to Freire’s (1970) notion of ‘conscientization’, should empower participants to understand and transform their oppressed situation. Validity, it is proposed (Scheurich 1996), is but amaskthatinfactpolicesandsetsboundaries to what is considered to be acceptable research by powerful research communities; discourses of validity in reality are discourses of power to define worthwhile knowledge. How defensible it is to suggest that researchers should have such ideological intents is, perhaps, amootpoint,thoughnottoaddressthisareais to perpetuate inequality by omission and neglect. Catalytic validity reasserts the centrality of ethics in the research process, for it requires researchers to interrogate their allegiances, responsibilities and self-interestedness (Burgess 1989). Consequential validity Partially related to catalytic validity is consequential validity, which argues that the ways in which research data are used (the consequences of the research) are in keeping with the capability or intentions of the research, i.e. the consequences of the research do not exceed the capability of the research and the action-related consequences of the research are both legitimate and fulfilled. Clearly, once the research is in the public domain, the researcher has little or no control over the way in which it is used. However, and this is often a political matter, research should not be used in ways in which it was not intended to be used, for example by exceeding the capability of the research data to make claims, by acting on the research in ways that the research does not support (e.g. by using the research for illegitimate epistemic support), by making illegitimate claims by using the research in unacceptable ways (e.g. by selection, distortion) and by not acting on the research in ways that were agreed, i.e. errors of omission and commission. A clear example of consequential validity is formative assessment. This is concerned with the extent to which students improve as a result of feedback given, hence if there is insufficient feedback for students to improve, or if students are unable to improve as a result of – a consequence of – the feedback, then the formative assessment has little consequential validity. Criterion-related validity This form of validity endeavours to relate the results of one particular instrument to another external criterion. Within this type of validity there are two principal forms: predictive validity and concurrent validity. Predictive validity is achieved if the data acquired at the first round of research correlate highly with data acquired at a future date. For example, if the results of examinations taken by 16 year olds correlate highly with the examination results gained by the same students when aged 18, then we might wish to say that the first examination demonstrated strong predictive validity. Avariationonthisthemeisencounteredin the notion of concurrent validity. Todemonstrate this form of validity the data gathered from using one instrument must correlate highly with data gathered from using another instrument. For example, suppose it was decided to research a

DEFINING VALIDITY 139<br />

is to abide by central ethical tenets of much<br />

research – non-traceability, anonymity and nonidentifiability.<br />

Cultural validity<br />

A type of validity related to ecological validity<br />

is cultural validity (Morgan 1999). This is<br />

particularly an issue in cross-cultural, intercultural<br />

and comparative kinds of research, where the<br />

intention is to shape research so that it is<br />

appropriate to the culture of the researched,<br />

and where the researcher and the researched are<br />

members of different cultures. Cultural validity<br />

is defined as ‘the degree to which a study is<br />

appropriate to the cultural setting where research<br />

is to be carried out’ (Joy 2003: 1). Cultural<br />

validity, Morgan (1999) suggests, applies at all<br />

stages of the research, and affects its planning,<br />

implementation and dissemination. It involves a<br />

degree of sensitivity to the participants, cultures<br />

and circumstances being studied. Morgan (2005)<br />

writes that<br />

cultural validity entails an appreciation of the<br />

cultural values of those being researched. This<br />

could include: understanding possibly different<br />

target culture attitudes to research; identifying<br />

and understanding salient terms as used in the<br />

target culture; reviewing appropriate target language<br />

literature; choosing research instruments that are<br />

acceptable to the target participants; checking<br />

interpretations and translations of data with native<br />

speakers; and being aware of one’s own cultural filters<br />

as a researcher.<br />

(Morgan 2005: 1)<br />

Joy (2003: 1) presents twelve important questions<br />

that researchers in different cultural contexts<br />

may face, to ensure that research is culture-fair and<br />

culturally sensitive:<br />

<br />

<br />

<br />

Is the research question understandable and of<br />

importance to the target group<br />

Is the researcher the appropriate person to<br />

conduct the research<br />

Are the sources of the theories that the research<br />

is based on appropriate for the target culture<br />

<br />

<br />

How do researchers in the target culture deal<br />

with the issues related to the research question<br />

(including their method and findings)<br />

Are appropriate gatekeepers and informants<br />

chosen<br />

Are the research design and research<br />

instruments ethical and appropriate according<br />

to the standards of the target culture<br />

<br />

<br />

<br />

<br />

<br />

<br />

How do members of the target culture define<br />

the salient terms of the research<br />

Are documents and other information translated<br />

in a culturally appropriate way<br />

Are the possible results of the research of<br />

potential value and benefit to the target<br />

culture<br />

Does interpretation of the results include the<br />

opinions and views of members of the target<br />

culture<br />

Are the results made available to members of<br />

the target culture for review and comment<br />

Does the researcher accurately and fairly<br />

communicate the results in their cultural<br />

context to people who are not members of<br />

the target culture<br />

Catalytic validity<br />

Catalytic validity embraces the paradigm of critical<br />

theory discussed in Chapter 1. Put neutrally,<br />

catalytic validity simply strives to ensure that<br />

research leads to action. However, the story does<br />

not end there, for discussions of catalytic validity<br />

are substantive; like critical theory, catalytic<br />

validity suggests an agenda. Lather (1986, 1991)<br />

and Kincheloe and McLaren (1994) suggest that<br />

the agenda for catalytic validity is to help<br />

participants to understand their worlds in order<br />

to transform them. The agenda is explicitly<br />

political, for catalytic validity suggests the need<br />

to expose whose definitions of the situation are<br />

operating in the situation. Lincoln and Guba<br />

(1986) suggest that the criterion of ‘fairness’ should<br />

be applied to research, meaning that it should<br />

not only augment and improve the participants’<br />

experience of the world, but also improve the<br />

empowerment of the participants. In this respect<br />

the research might focus on what might be (the<br />

Chapter 6

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!