Systematic Review - Network for Business Sustainability

Systematic Review - Network for Business Sustainability Systematic Review - Network for Business Sustainability

23.07.2013 Views

Study methodologies Figure 9 shows that 60 per cent of included studies adopted a cross-sectional approach; the remaining 40 per cent of studies were longitudinal (11 per cent), historical (10 per cent) or did not have a discernible perspective (19 per cent). As Figure 10 shows, the single preferred methodology was qualitative (46 studies, 46 per cent), and the next most preferred method, quantitative (24 studies, 24 per cent). A large proportion of studies (22 studies, 22 per cent) did not make explicit their methodology — though they mostly reported on single case histories. Figure 9 STUDY TEMPORAL PERSPECTIVES (NUMBER OF PAPERS) QUALITY APPRAISAL The role of quality appraisal of studies in systematic reviews in MOS is contested. The role for quality appraisal in systematic review was originally conceived as a filtering mechanism to exclude possibly biased studies or other factors that might affect the “truth” of the conclusions. MOS consists of diverse types of evidence, characterized by different epistemological and ontological positions. As a result, the purpose that quality appraisal serves and its position in systematic review have been re-assessed. Quality appraisal has not been widely reported in many of the systematic reviews published in peer-reviewed MOS journals, Figure 10 STUDY METHODOLOGIES (NUMBER OF PAPERS) Innovating for Sustainability 80

which speaks to the contestation around its use, the difficulty of execution and the confusion about the role of such appraisals in studies that draw on evidence not derived from randomized controlled trials. For this review, we assessed the quality of studies not to filter out studies, but to describe the range of quality across included studies in order to reflect on the strength of evidence underpinning the findings. As we have already noted, most studies informing this review are qualitative. No standardized criteria exist for the appraisal of qualitative studies. Researchers whose systematic reviews have included qualitative and mixedmethod studies have adopted a variety of approaches, Table 9 HIERARCHY OF EVIDENCE FOR PRACTICE IN QUALITATIVE RESEARCH (Source: Daly et al., 2007) from not applying quality criteria and thus keeping all relevant data, to seeking objectivity by applying checklists. We assessed the quality of studies on three dimensions: generalizability, technical accomplishment and strength of evidence. Generalizability refers to the extent to which findings apply beyond the immediate context of the study. To this end, we used Daly et al.’s (2007) hierarchy of evidence for practice in qualitative research. This hierarchy emphasizes research’s ability to provide evidence-for-practice or evidence-for-policy. It consists of four levels (see Table 9) and we put each of the studies in one of these. STUDY TYPE FEATURES EVIDENCE IMPLICATIONS FOR PRACTICE Generalizable studies Sampling focused by theory and the literature, extended through analysis to capture diversity of experience. Analytic procedures comprehensive and clear. Located in the literature to assess relevance to other settings. Conceptual studies Theoretical concepts guide sample selection, based on analysis of literature. May be limited to one group about which little is known or a number of important subgroups. Conceptual analysis recognizes diversity in participants’ views. Descriptive studies Sample selected to illustrate practical rather than theoretical issues. Record a range of illustrative quotes including themes from the accounts of ‘‘many,’’ ‘‘most,’’ or ‘‘some’’ study participants. Single case study Provides rich data on the views or experiences of one person. Can provide insights in unexplored contexts. Clear indications for practice or policy. May offer support for current practice, or critique with indicated directions for change. Weaker designs identify the need for further research on other groups, or urge caution in practice. Well-developed studies can provide good evidence if residual uncertainties are clearly identified. Demonstrate that a phenomenon exists in a defined group. Identify practice issues for further consideration. Alerts practitioners to the existence of an unusual phenomenon. Innovating for Sustainability 81

Study methodologies<br />

Figure 9 shows that 60 per cent of included studies<br />

adopted a cross-sectional approach; the remaining<br />

40 per cent of studies were longitudinal (11 per cent),<br />

historical (10 per cent) or did not have a discernible<br />

perspective (19 per cent).<br />

As Figure 10 shows, the single preferred methodology<br />

was qualitative (46 studies, 46 per cent), and the next<br />

most preferred method, quantitative (24 studies, 24 per<br />

cent). A large proportion of studies (22 studies, 22 per<br />

cent) did not make explicit their methodology — though<br />

they mostly reported on single case histories.<br />

Figure 9<br />

STUDY TEMPORAL PERSPECTIVES<br />

(NUMBER OF PAPERS)<br />

QUALITY APPRAISAL<br />

The role of quality appraisal of studies in systematic<br />

reviews in MOS is contested. The role <strong>for</strong> quality<br />

appraisal in systematic review was originally conceived<br />

as a filtering mechanism to exclude possibly biased<br />

studies or other factors that might affect the “truth”<br />

of the conclusions. MOS consists of diverse types of<br />

evidence, characterized by different epistemological<br />

and ontological positions. As a result, the purpose that<br />

quality appraisal serves and its position in systematic<br />

review have been re-assessed. Quality appraisal has<br />

not been widely reported in many of the systematic<br />

reviews published in peer-reviewed MOS journals,<br />

Figure 10<br />

STUDY METHODOLOGIES (NUMBER<br />

OF PAPERS)<br />

Innovating <strong>for</strong> <strong>Sustainability</strong> 80

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!