manual for social impact assessment of land-based ... - Forest Trends

TOOLBOX AREA 2: DATA COLLECTION AND ANALYSIS METHODS
T5 General Data Collection Methods
T5.1 Introduction
The HOW to measure methods can be subdivided into general and more specific data collection
methods. General data collection methods refer to standard tools like household surveys, case
studies, key informant or focus group discussions, community mapping, and a range of other PRA or
RRA type methods. More specific methods are those designed for impact assessment or M&E - all of
them are participatory to some extent. Since the general data collection methods are well-known
and covered in various manuals (see Section T5.4), it is unnecessary to explain them in detail here,
although a brief overview and some observations on their use in the context of SIA are provided
below.
T5.2 Overview of General Data Collection Methods
The first thing to note is that data collection methods should not be seen as mutually exclusive. It is
not a case of using PRA or household surveys or key informant interviews, but is rather one of an
appropriate combination of methods, and in the right sequence, in order to obtain reliable
estimates. A mix of qualitative and quantitative analysis is necessary for SIA. For example, some
methods are more suitable for analyzing process-based or qualitative indicators, while others are
more suited to quantitative indicators (see Box T1). In general, qualitative and participatory research
methods have become more popular than sample surveys and quantitative analysis in SIA, e.g., in
the micro-finance sector. This is because they are better for identifying intangible, negative or
unforeseen outcomes; assessing social and institutional change (e.g., Box T2 suggests a simple
approach for social capital); capturing local stakeholder perceptions; exploring social and livelihood
complexities, including causative links; and capturing equity, gender and temporal issues. But there
are some caveats to participatory data collection methods:
• participatory research methods are subject to bias and subjectivity 5
, and may be less
effective for measuring indicators based on SMART objectives;
• qualitative participatory research can be expensive for local people (in terms of their
opportunity costs) and research teams, e.g., the ‘Participatory Assessment of Livelihood
Impacts’ study based on the SLF and PRA methods, required “highly analytical and skilled
study teams” (Ashley & Hussein, 2000);
• an authoritative source (Guijt, 1999) advises that genuinely participatory M&E is expensive
and time consuming, and urges caution in using this suite of methods unless the benefits are
very clear.
5
For example, research by Richards et al (2003) found that even ‘best practice’ PRA based estimates of
household income 5 are prone to major bias problems.
Social Impact Assessment of Land-Based Carbon Projects (1.0) – Part II | 35