RESEARCH METHOD COHEN ok
RESEARCH METHOD COHEN ok RESEARCH METHOD COHEN ok
CHARACTERISTICS OF EX POST FACTO RESEARCH 267 the hypothesis. Due to lack of control of X and other possible Xs, the truth of the hypothesized relation between X and O cannot be asserted with the confidence of the experimental researcher. Basically, then, ex post facto investigations have, so to speak, a built-in weakness: lack of control of the independent variable or variables. As Spector (1993: 43) suggests, it is impossible to isolate and control every possible variable, or to know with absolute certainty which are the most crucial variables. This brief comparison highlights the most important difference between the two designs – control. In the experimental situation, investigators at least have manipulative control; they have as a minimum one active variable. If an experiment is a ‘true’ experiment, they can also exercise control by randomization. They can assign subjects to groups randomly; or, at the very least, they can assign treatments to groups at random. In the ex post facto research situation, this control of the independent variable is not possible, and, perhaps more important, neither is randomization. Investigators must take things as they are and try to disentangle them, though having said this, they can make use of selected procedures that will give them an element of control in this research. These we shall touch upon shortly. By their very nature, ex post facto experiments can provide support for any number of different, perhaps even contradictory, hypotheses; they are so completely flexible that it is largely a matter of postulating hypotheses according to one’s personal preference. The investigator begins with certain data and looks for an interpretation consistent with them; often, however, a number of interpretations may be at hand. Consider again the hypothetical increase in road accidents in a given town. A retrospective search for causes will disclose half a dozen plausible ones. Experimental studies, by contrast, begin with a specific interpretation and then determine whether it is congruent with externally derived data. Frequently, causal relationships seem to be established on nothing more substantial than the premise that any related event occurring prior to the phenomenon under study is assumed to be its cause – the classical post hoc, ergo propter hoc fallacy. 2 Overlooked is the fact that even when we do find a relationship between two variables, we must recognize the possibility that both are individual results of a common third factor rather than the first being necessarily the cause of the second. As we have seen earlier, there is also the real possibility of reverse causation, e.g. that a heart condition promotes obesity rather than the other way around, or that they encourage each other. The point is that the evidence simply illustrates the hypothesis; it does not test it, since hypotheses cannot be tested on the same data from which they were derived. The relationship noted may actually exist, but it is not necessarily the only relationship, or perhaps the crucial one. Before we can accept that smoking is the primary cause of lung cancer, we have to rule out alternative hypotheses. Further, a researcher may find that watching television correlates with poor school performance. Now, it may be there is a causal effect here: watching television causes poor school performance; or there may be reverse causality: poor school performance causes students to watch more television. However, there may be a third explanation: students who, for whatever reason (e.g. ability, motivation), do not do well at school also like watching television; it may be the third variable (the independent variable of ability or motivation) that is causing the other two outcomes (watching a lot of television or poor school performance). We must not conclude from what has just been said that ex post facto studies are of little value; many of our important investigations in education and psychology are ex post facto designs. There is often no choice in the matter: an investigator cannot cause one group to become failures, delinquent, suicidal, brain-damaged or dropouts. Research must of necessity rely on existing groups. On the other hand, the inability of ex post facto designs to incorporate the basic need for control (e.g. through manipulation or randomization) makes them vulnerable from a scientific point of view and the possibility of their being misleading should be clearly acknowledged. Ex post facto Chapter 12
268 EX POST FACTO RESEARCH designs are probably better conceived more circumspectly, not as experiments with the greater certainty that these denote, but more as surveys, useful as sources of hypotheses to be tested by more conventional experimental means at a later date. Occasions when appropriate Ex post facto designs are appropriate in circumstances where the more powerful experimental method is not possible. These arise when, for example, it is not possible to select, control and manipulate the factors necessary to study cause-and-effect relationships directly; or when the control of all variables except a single independent variable may be unrealistic and artificial, preventing the normal interaction with other influential variables; or when laboratory controls for many research purposes would be impractical, costly or ethically undesirable. Ex post facto research is particularly suitable in social, educational and – to a lesser extent – psychological contexts where the independent variable or variables lie outside the researcher’s control. Examples of the method abound in these areas: the research on cigarette-smoking and lung cancer, for instance; or studies of teacher characteristics; or studies examining the relationship between political and religious affiliation and attitudes; or investigations into the relationship between school achievement and independent variables such as social class, race, sex and intelligence. Many of these may be divided into large-scale or small-scale ex post facto studies, for example, Stables’ (1990) large-scale study of differences between students from mixed and single-sex schools and Arnold and Atkins’s (1991) smallscale study of the social and emotional adjustment of hearing-impaired students. Advantages and disadvantages of ex post facto research Among the advantages of the approach are the following: Ex post facto research meets an important need of the researcher where the more rigorous experimental approach is not possible. In the case of the alleged relationship between smoking and lung cancer, for instance, this cannot be tested experimentally (at least as far as human beings are concerned). The method yields useful information concerning the nature of phenomena – what goes with what and under what conditions. In this way, ex post facto research is a valuable exploratory tool. Improvements in statistical techniques and general methodology have made ex post facto designs more defensible. In some ways and in certain situations the method is more useful than the experimental method, especially where the setting up of the latter would introduce a note of artificiality into research proceedings. Ex post facto research is particularly appropriate when simple cause-and-effect relationships are being explored. The method can give a sense of direction and provide a fruitful source of hypotheses that can subsequently be tested by the more rigorous experimental method. Among the limitations and weaknesses of ex post facto designs the following may be mentioned: There is the problem of lack of control in that the researcher is unable to manipulate the independent variable or to randomize her subjects. One cannot know for certain whether the causative factor has been included or even identified. It may be that no single factor is the cause. Aparticularoutcomemayresultfromdifferent causes on different occasions. When a relationship has been discovered, there is the problem of deciding which is the cause and which the effect; the possibility of reverse causation must be considered. The relationship of two factors does not establish cause and effect. Classifying into dichotomous groups can be problematic.
- Page 236 and 237: STRENGTHS AND WEAKNESSES OF LONGITU
- Page 238 and 239: POSTAL, INTERVIEW AND TELEPHONE SUR
- Page 240 and 241: POSTAL, INTERVIEW AND TELEPHONE SUR
- Page 242 and 243: POSTAL, INTERVIEW AND TELEPHONE SUR
- Page 244 and 245: EVENT HISTORY ANALYSIS 225 may be p
- Page 246 and 247: INTERNET-BASED SURVEYS 227 packages
- Page 248 and 249: INTERNET-BASED SURVEYS 229 instruc
- Page 250 and 251: INTERNET-BASED SURVEYS 231 Box 10.1
- Page 252 and 253: INTERNET-BASED SURVEYS 233 Box 10.1
- Page 254 and 255: INTERNET-BASED SURVEYS 235 Box 10.1
- Page 256 and 257: INTERNET-BASED SURVEYS 237 Witte et
- Page 258 and 259: INTERNET-BASED EXPERIMENTS 239 requ
- Page 260 and 261: INTERNET-BASED INTERVIEWS 241 ‘ne
- Page 262 and 263: SEARCHING FOR RESEARCH MATERIALS ON
- Page 264 and 265: COMPUTER SIMULATIONS 245 autho
- Page 266 and 267: COMPUTER SIMULATIONS 247 computer s
- Page 268 and 269: COMPUTER SIMULATIONS 249 On the oth
- Page 270 and 271: GEOGRAPHICAL INFORMATION SYSTEMS 25
- Page 272 and 273: 11 Case studies What is a case stud
- Page 274 and 275: WHAT IS A CASE STUDY 255 (providing
- Page 276 and 277: WHAT IS A CASE STUDY 257 argue that
- Page 278 and 279: EXAMPLES OF KINDS OF CASE STUDY 259
- Page 280 and 281: PLANNING A CASE STUDY 261 accounts
- Page 282 and 283: CONCLUSION 263 In the narrativ
- Page 284 and 285: CO-RELATIONAL AND CRITERION GROUPS
- Page 288 and 289: DESIGNING AN EX POST FACTO INVESTIG
- Page 290 and 291: PROCEDURES IN EX POST FACTO RESEARC
- Page 292 and 293: INTRODUCTION 273 Box 13.1 Independe
- Page 294 and 295: TRUE EXPERIMENTAL DESIGNS 275 motor
- Page 296 and 297: TRUE EXPERIMENTAL DESIGNS 277 2 Sub
- Page 298 and 299: TRUE EXPERIMENTAL DESIGNS 279 textb
- Page 300 and 301: TRUE EXPERIMENTAL DESIGNS 281 Facto
- Page 302 and 303: A QUASI-EXPERIMENTAL DESIGN: THE NO
- Page 304 and 305: PROCEDURES IN CONDUCTING EXPERIMENT
- Page 306 and 307: EXAMPLES FROM EDUCATIONAL RESEARCH
- Page 308 and 309: EVIDENCE-BASED EDUCATIONAL RESEARCH
- Page 310 and 311: EVIDENCE-BASED EDUCATIONAL RESEARCH
- Page 312 and 313: EVIDENCE-BASED EDUCATIONAL RESEARCH
- Page 314 and 315: EVIDENCE-BASED EDUCATIONAL RESEARCH
- Page 316 and 317: 14 Action research Introduction Act
- Page 318 and 319: PRINCIPLES AND CHARACTERISTICS OF A
- Page 320 and 321: PRINCIPLES AND CHARACTERISTICS OF A
- Page 322 and 323: ACTION RESEARCH AS CRITICAL PRAXIS
- Page 324 and 325: PROCEDURES FOR ACTION RESEARCH 305
- Page 326 and 327: PROCEDURES FOR ACTION RESEARCH 307
- Page 328 and 329: PROCEDURES FOR ACTION RESEARCH 309
- Page 330 and 331: SOME PRACTICAL AND THEORETICAL MATT
- Page 332: CONCLUSION 313 3 Actionresearchreso
268 EX POST FACTO <strong>RESEARCH</strong><br />
designs are probably better conceived more<br />
circumspectly, not as experiments with the greater<br />
certainty that these denote, but more as surveys,<br />
useful as sources of hypotheses to be tested by more<br />
conventional experimental means at a later date.<br />
Occasions when appropriate<br />
Ex post facto designs are appropriate in circumstances<br />
where the more powerful experimental<br />
method is not possible. These arise when, for<br />
example, it is not possible to select, control<br />
and manipulate the factors necessary to study<br />
cause-and-effect relationships directly; or when<br />
the control of all variables except a single independent<br />
variable may be unrealistic and artificial,<br />
preventing the normal interaction with other influential<br />
variables; or when laboratory controls<br />
for many research purposes would be impractical,<br />
costly or ethically undesirable.<br />
Ex post facto research is particularly suitable<br />
in social, educational and – to a lesser<br />
extent – psychological contexts where the independent<br />
variable or variables lie outside the researcher’s<br />
control. Examples of the method abound<br />
in these areas: the research on cigarette-sm<strong>ok</strong>ing<br />
and lung cancer, for instance; or studies of teacher<br />
characteristics; or studies examining the relationship<br />
between political and religious affiliation and<br />
attitudes; or investigations into the relationship<br />
between school achievement and independent<br />
variables such as social class, race, sex and intelligence.<br />
Many of these may be divided into<br />
large-scale or small-scale ex post facto studies, for<br />
example, Stables’ (1990) large-scale study of differences<br />
between students from mixed and single-sex<br />
schools and Arnold and Atkins’s (1991) smallscale<br />
study of the social and emotional adjustment<br />
of hearing-impaired students.<br />
Advantages and disadvantages of ex post<br />
facto research<br />
Among the advantages of the approach are the<br />
following:<br />
<br />
Ex post facto research meets an important need<br />
of the researcher where the more rigorous<br />
<br />
<br />
<br />
<br />
<br />
experimental approach is not possible. In<br />
the case of the alleged relationship between<br />
sm<strong>ok</strong>ing and lung cancer, for instance, this<br />
cannot be tested experimentally (at least as far<br />
as human beings are concerned).<br />
The method yields useful information concerning<br />
the nature of phenomena – what goes with<br />
what and under what conditions. In this way,<br />
ex post facto research is a valuable exploratory<br />
tool.<br />
Improvements in statistical techniques and<br />
general methodology have made ex post facto<br />
designs more defensible.<br />
In some ways and in certain situations the<br />
method is more useful than the experimental<br />
method, especially where the setting up of the<br />
latter would introduce a note of artificiality<br />
into research proceedings.<br />
Ex post facto research is particularly appropriate<br />
when simple cause-and-effect relationships are<br />
being explored.<br />
The method can give a sense of direction and<br />
provide a fruitful source of hypotheses that can<br />
subsequently be tested by the more rigorous<br />
experimental method.<br />
Among the limitations and weaknesses of ex post<br />
facto designs the following may be mentioned:<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
There is the problem of lack of control in<br />
that the researcher is unable to manipulate<br />
the independent variable or to randomize her<br />
subjects.<br />
One cannot know for certain whether the<br />
causative factor has been included or even<br />
identified.<br />
It may be that no single factor is the cause.<br />
Aparticularoutcomemayresultfromdifferent<br />
causes on different occasions.<br />
When a relationship has been discovered, there<br />
is the problem of deciding which is the cause<br />
and which the effect; the possibility of reverse<br />
causation must be considered.<br />
The relationship of two factors does not<br />
establish cause and effect.<br />
Classifying into dichotomous groups can be<br />
problematic.