RESEARCH METHOD COHEN ok

RESEARCH METHOD COHEN ok RESEARCH METHOD COHEN ok

12.01.2015 Views

CHARACTERISTICS OF EX POST FACTO RESEARCH 267 the hypothesis. Due to lack of control of X and other possible Xs, the truth of the hypothesized relation between X and O cannot be asserted with the confidence of the experimental researcher. Basically, then, ex post facto investigations have, so to speak, a built-in weakness: lack of control of the independent variable or variables. As Spector (1993: 43) suggests, it is impossible to isolate and control every possible variable, or to know with absolute certainty which are the most crucial variables. This brief comparison highlights the most important difference between the two designs – control. In the experimental situation, investigators at least have manipulative control; they have as a minimum one active variable. If an experiment is a ‘true’ experiment, they can also exercise control by randomization. They can assign subjects to groups randomly; or, at the very least, they can assign treatments to groups at random. In the ex post facto research situation, this control of the independent variable is not possible, and, perhaps more important, neither is randomization. Investigators must take things as they are and try to disentangle them, though having said this, they can make use of selected procedures that will give them an element of control in this research. These we shall touch upon shortly. By their very nature, ex post facto experiments can provide support for any number of different, perhaps even contradictory, hypotheses; they are so completely flexible that it is largely a matter of postulating hypotheses according to one’s personal preference. The investigator begins with certain data and looks for an interpretation consistent with them; often, however, a number of interpretations may be at hand. Consider again the hypothetical increase in road accidents in a given town. A retrospective search for causes will disclose half a dozen plausible ones. Experimental studies, by contrast, begin with a specific interpretation and then determine whether it is congruent with externally derived data. Frequently, causal relationships seem to be established on nothing more substantial than the premise that any related event occurring prior to the phenomenon under study is assumed to be its cause – the classical post hoc, ergo propter hoc fallacy. 2 Overlooked is the fact that even when we do find a relationship between two variables, we must recognize the possibility that both are individual results of a common third factor rather than the first being necessarily the cause of the second. As we have seen earlier, there is also the real possibility of reverse causation, e.g. that a heart condition promotes obesity rather than the other way around, or that they encourage each other. The point is that the evidence simply illustrates the hypothesis; it does not test it, since hypotheses cannot be tested on the same data from which they were derived. The relationship noted may actually exist, but it is not necessarily the only relationship, or perhaps the crucial one. Before we can accept that smoking is the primary cause of lung cancer, we have to rule out alternative hypotheses. Further, a researcher may find that watching television correlates with poor school performance. Now, it may be there is a causal effect here: watching television causes poor school performance; or there may be reverse causality: poor school performance causes students to watch more television. However, there may be a third explanation: students who, for whatever reason (e.g. ability, motivation), do not do well at school also like watching television; it may be the third variable (the independent variable of ability or motivation) that is causing the other two outcomes (watching a lot of television or poor school performance). We must not conclude from what has just been said that ex post facto studies are of little value; many of our important investigations in education and psychology are ex post facto designs. There is often no choice in the matter: an investigator cannot cause one group to become failures, delinquent, suicidal, brain-damaged or dropouts. Research must of necessity rely on existing groups. On the other hand, the inability of ex post facto designs to incorporate the basic need for control (e.g. through manipulation or randomization) makes them vulnerable from a scientific point of view and the possibility of their being misleading should be clearly acknowledged. Ex post facto Chapter 12

268 EX POST FACTO RESEARCH designs are probably better conceived more circumspectly, not as experiments with the greater certainty that these denote, but more as surveys, useful as sources of hypotheses to be tested by more conventional experimental means at a later date. Occasions when appropriate Ex post facto designs are appropriate in circumstances where the more powerful experimental method is not possible. These arise when, for example, it is not possible to select, control and manipulate the factors necessary to study cause-and-effect relationships directly; or when the control of all variables except a single independent variable may be unrealistic and artificial, preventing the normal interaction with other influential variables; or when laboratory controls for many research purposes would be impractical, costly or ethically undesirable. Ex post facto research is particularly suitable in social, educational and – to a lesser extent – psychological contexts where the independent variable or variables lie outside the researcher’s control. Examples of the method abound in these areas: the research on cigarette-smoking and lung cancer, for instance; or studies of teacher characteristics; or studies examining the relationship between political and religious affiliation and attitudes; or investigations into the relationship between school achievement and independent variables such as social class, race, sex and intelligence. Many of these may be divided into large-scale or small-scale ex post facto studies, for example, Stables’ (1990) large-scale study of differences between students from mixed and single-sex schools and Arnold and Atkins’s (1991) smallscale study of the social and emotional adjustment of hearing-impaired students. Advantages and disadvantages of ex post facto research Among the advantages of the approach are the following: Ex post facto research meets an important need of the researcher where the more rigorous experimental approach is not possible. In the case of the alleged relationship between smoking and lung cancer, for instance, this cannot be tested experimentally (at least as far as human beings are concerned). The method yields useful information concerning the nature of phenomena – what goes with what and under what conditions. In this way, ex post facto research is a valuable exploratory tool. Improvements in statistical techniques and general methodology have made ex post facto designs more defensible. In some ways and in certain situations the method is more useful than the experimental method, especially where the setting up of the latter would introduce a note of artificiality into research proceedings. Ex post facto research is particularly appropriate when simple cause-and-effect relationships are being explored. The method can give a sense of direction and provide a fruitful source of hypotheses that can subsequently be tested by the more rigorous experimental method. Among the limitations and weaknesses of ex post facto designs the following may be mentioned: There is the problem of lack of control in that the researcher is unable to manipulate the independent variable or to randomize her subjects. One cannot know for certain whether the causative factor has been included or even identified. It may be that no single factor is the cause. Aparticularoutcomemayresultfromdifferent causes on different occasions. When a relationship has been discovered, there is the problem of deciding which is the cause and which the effect; the possibility of reverse causation must be considered. The relationship of two factors does not establish cause and effect. Classifying into dichotomous groups can be problematic.

268 EX POST FACTO <strong>RESEARCH</strong><br />

designs are probably better conceived more<br />

circumspectly, not as experiments with the greater<br />

certainty that these denote, but more as surveys,<br />

useful as sources of hypotheses to be tested by more<br />

conventional experimental means at a later date.<br />

Occasions when appropriate<br />

Ex post facto designs are appropriate in circumstances<br />

where the more powerful experimental<br />

method is not possible. These arise when, for<br />

example, it is not possible to select, control<br />

and manipulate the factors necessary to study<br />

cause-and-effect relationships directly; or when<br />

the control of all variables except a single independent<br />

variable may be unrealistic and artificial,<br />

preventing the normal interaction with other influential<br />

variables; or when laboratory controls<br />

for many research purposes would be impractical,<br />

costly or ethically undesirable.<br />

Ex post facto research is particularly suitable<br />

in social, educational and – to a lesser<br />

extent – psychological contexts where the independent<br />

variable or variables lie outside the researcher’s<br />

control. Examples of the method abound<br />

in these areas: the research on cigarette-sm<strong>ok</strong>ing<br />

and lung cancer, for instance; or studies of teacher<br />

characteristics; or studies examining the relationship<br />

between political and religious affiliation and<br />

attitudes; or investigations into the relationship<br />

between school achievement and independent<br />

variables such as social class, race, sex and intelligence.<br />

Many of these may be divided into<br />

large-scale or small-scale ex post facto studies, for<br />

example, Stables’ (1990) large-scale study of differences<br />

between students from mixed and single-sex<br />

schools and Arnold and Atkins’s (1991) smallscale<br />

study of the social and emotional adjustment<br />

of hearing-impaired students.<br />

Advantages and disadvantages of ex post<br />

facto research<br />

Among the advantages of the approach are the<br />

following:<br />

<br />

Ex post facto research meets an important need<br />

of the researcher where the more rigorous<br />

<br />

<br />

<br />

<br />

<br />

experimental approach is not possible. In<br />

the case of the alleged relationship between<br />

sm<strong>ok</strong>ing and lung cancer, for instance, this<br />

cannot be tested experimentally (at least as far<br />

as human beings are concerned).<br />

The method yields useful information concerning<br />

the nature of phenomena – what goes with<br />

what and under what conditions. In this way,<br />

ex post facto research is a valuable exploratory<br />

tool.<br />

Improvements in statistical techniques and<br />

general methodology have made ex post facto<br />

designs more defensible.<br />

In some ways and in certain situations the<br />

method is more useful than the experimental<br />

method, especially where the setting up of the<br />

latter would introduce a note of artificiality<br />

into research proceedings.<br />

Ex post facto research is particularly appropriate<br />

when simple cause-and-effect relationships are<br />

being explored.<br />

The method can give a sense of direction and<br />

provide a fruitful source of hypotheses that can<br />

subsequently be tested by the more rigorous<br />

experimental method.<br />

Among the limitations and weaknesses of ex post<br />

facto designs the following may be mentioned:<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

There is the problem of lack of control in<br />

that the researcher is unable to manipulate<br />

the independent variable or to randomize her<br />

subjects.<br />

One cannot know for certain whether the<br />

causative factor has been included or even<br />

identified.<br />

It may be that no single factor is the cause.<br />

Aparticularoutcomemayresultfromdifferent<br />

causes on different occasions.<br />

When a relationship has been discovered, there<br />

is the problem of deciding which is the cause<br />

and which the effect; the possibility of reverse<br />

causation must be considered.<br />

The relationship of two factors does not<br />

establish cause and effect.<br />

Classifying into dichotomous groups can be<br />

problematic.

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!