RESEARCH METHOD COHEN ok

290 EXPERIMENTS AND META-ANALYSIS
used method of investigation, bringing together
different studies to provide evidence to inform
policy-making and planning. Meta-analysis
is a research strategy in itself. That this is
happening significantly is demonstrated in the
establishment of the EPPI-Centre (Evidence
for Policy and Practice Information and Coordinating
Centre) at the University of London
(http://eppi.ioe.ac.uk/EPPIWeb/home.aspx),
the Social, Psychological, Educational and Criminological
Controlled Trials Register (SPECTR),
later transferred to the Campbell Collaboration
(http://www.campbellcollaboration.org), a
parallel to the Cochrane Collaboration in
medicine (http://www.cochrane.org/index0.htm),
which undertakes systematic reviews and metaanalyses
of, typically, experimental evidence in
medicine, and the Curriculum, Evaluation and
Management (CEM) centre at the University of
Durham (http://www.cemcentre.org). ‘Evidence’
here typically comes from randomized controlled
trials of one hue or another (Tymms 1999; Coe
et al.2000;ThomasandPring2004:95),withtheir
emphasis on careful sampling, control of variables,
both extraneous and included, and measurements
of effect size. The cumulative evidence from collected
RCTs is intended to provide a reliable body
of knowledge on which to base policy and practice
(Coe et al. 2000).Suchaccumulateddata,it
is claimed, deliver evidence of ‘what works’, although
Morrison (2001b) suggests that this claim
is suspect.
The roots of evidence-based practice lie
in medicine, where the advocacy by Cochrane
(1972) for randomized controlled trials together
with their systematic review and documentation
led to the foundation of the Cochrane
Collaboration (Maynard and Chalmers 1997),
which is now worldwide. The careful, quantitativebased
research studies that can contribute to the
accretion of an evidential base is seen to be a
powerful counter to the often untried and undertested
schemes that are injected into practice.
More recently evidence-based education has
entered the worlds of social policy, social work
(MacDonald 1997) and education (Fitz-Gibbon
1997). At the forefront of educational research
in this area are Fitz-Gibbon (1996; 1997; 1999)
and Tymms (1996), who, at the Curriculum,
Evaluation and Management Centre at the
University of Durham, have established one of the
world’s largest monitoring centres in education.
Fitz-Gibbon’s work is critical of multilevel
modelling and, instead, suggests how indicator
systems can be used with experimental methods
to provide clear evidence of causality and a ready
answer to her own question, ‘How do we know
what works’ (Fitz-Gibbon 1999: 33).
Echoing Anderson and Biddle (1991), Fitz-
Gibbon suggests that policy-makers shun evidence
in the development of policy and that practitioners,
in the hurly-burly of everyday activity, call upon
tacit knowledge rather than the knowledge which
is derived from RCTs. However, in a compelling
argument (Fitz-Gibbon 1997: 35–6), she suggests
that evidence-based approaches are necessary in
order to challenge the imposition of unproven
practices, solve problems and avoid harmful
procedures, and create improvement that leads
to more effective learning. Further, such evidence,
she contends, should examine effect sizes rather
than statistical significance.
While the nature of information in evidencebased
education might be contested by researchers
whose sympathies (for whatever reason) lie outside
randomized controlled trials, the message from
Fitz-Gibbon will not go away: the educational
community needs evidence on which to base
its judgements and actions. The development
of indicator systems worldwide attests to the
importance of this, be it through assessment and
examination data, inspection findings, national
and international comparisons of achievement,
or target setting. Rather than being a shot
in the dark, evidence-based education suggests
that policy formation should be informed, and
policy decision-making should be based on the
best information to date rather than on hunch,
ideology or political will. It is bordering on the
unethical to implement untried and untested
recommendations in educational practice, just
as it is unethical to use untested products and
procedures on hospital patients without their
consent.