manual for social impact assessment of land-based ... - Forest Trends

T4 Matching Methods
T4.1 Introduction
Using the experimental or quasi-experimental approach is the classical or traditional way of tackling
the ‘attribution’ challenge of impact assessment. The essence of this approach is making
comparisons, statistical if possible, between 'control' and 'treatment' (or project) groups - hence the
term ‘matching methods’. Control groups or individuals are non-participants (in the project) with
similar observable characteristics (age, income, education, gender, etc.) to the project participants.
While we have classified the quasi-experimental approach as an impact assessment framework, it
should be noted that it does not per se provide a basis for selecting indicators unlike the previously
described approaches. It is rather a framework for data collection and analysis that tackles the
attribution problem.
T4.2 Description of Methods
Experimental methods (or ‘randomized experiments’)
The difference between experimental and quasi-experimental methods is in the way that the control
and treatment communities (or other stratifying units) and households are selected. In an
experimental approach, control and treatment (participants) respondents are selected using
statistical sampling methods. This allows econometric and other statistical analysis using the
‘difference of differences’ method – subject to bias tests, any differences in the outcomes or results
between control and treatment groups are attributed to the project. This is a cross-sectional
comparison – therefore no baseline or starting conditions study is necessary, although one is always
desirable since it provides a second basis for comparison. Other advantages of matching method
approaches is that they can pick up negative or unexpected impacts, and show whether they are due
to the project or not, which can prevent a project being falsely blamed for them.
But the experimental methods approach suffers from various problems (Richards, 2008 based on
various sources):
• The high cost associated with the sample size and expertise needed;
• While the ‘observable characteristics’ may be similar, it is difficult to know how similar are
the ‘unobservable characteristics’ (attitudes to risk, personal goals, entrepreneurship skills,
etc.) without further research – differences in either type of characteristic increase bias, and
reduce the reliability of the ‘estimators’;
• Where controls are close to the project area, ‘spillover effects’ can blur the distinction with
participants, e.g., the controls might modify their behavior or activities based on observing
participants or obtaining project information;
• Where the controls are further away, this increases the costs and the likelihood that they
will have different characteristics even though selected randomly (e.g., due to differences in
market access, influence of other projects, etc.)
Social Impact Assessment of Land-Based Carbon Projects (1.0) – Part II | 32