Measuring Impact - Nicva

60 Measuring Impact – Case-studies of impact assessment
What works:is there a hybrid of the different approaches 8.4
The components of a successful impact assessment might constructively use a mix of the different
approaches used in the case-studies. When combined with data collected for other reasons (e.g.
monitoring and evaluation), they could work as ‘triangulation tools’ for a more grounded, realistic
assessment of impact. A hybrid approach therefore might employ the following:
• use of systems maps of stakeholders;
• use of one-to-one interviews with randomly selected representatives from each
stakeholder group, but particularly providers and beneficiaries;
• use of existing approaches, many of which offer lists of indicators which can be used
as they are or adapted;
• use of focus groups to develop interview or questionnaire themes;
• development of context-specific themes or dimensions of impact;
• the development, in conjunction with stakeholders, of ways in which to represent
impact in terms of a selection of these dimensions.
It is clear that any hybrid approach needs more thought on how to capture longer-term and
negative impacts. The former may simply be a question of resources and building impact
assessment into organisational planning; capturing negative impact may be a reflection of the
skills and abilities of the interviewer.
Moving forward:what still needs to be done and why 8.5
Our recommendations for future work centre around two broad issues: mapping types of impact
to particular voluntary sector functions or sub-sectors, and further development of contextspecific
tools for impact assessment.
Further investigation still needs to be undertaken to produce what ultimately might be a typology
of impacts. This would build on Table 8.1, with each cell populated by function-specific casestudies.
Interestingly, there may be another, sub-sectoral dimension. By being able to map types of
impact to voluntary and community sector functions, organisations might then have a much
better starting point than we had when thinking about the impact of their specific organisation.
In order to complete the case-studies for this table, work needs to be undertaken to refine and
test the hybrid approach loosely outlined above. This would include the development of specific
tools for non-researchers, including sample interview schedules and generic stakeholder maps.
Our limited resources mean that the approach would of course need to be tested in a much wider
range of environments. Finally, we recognise that much more work needs to be done with regard
to the presentation of findings: it remains the case that policy change is easier to implement when
the evidence base is predominantly quantitative.
The beginning of this report argued that impact assessment need not be complex or expensive.
We believe we have gone some way to demonstrate the former, although the question of
resources still has to be fully addressed. The development of a more comprehensive ‘how to’
guide will go some way to addressing this, and will therefore be the focus of our future work in
this area. In the meantime, we welcome the feedback and comments of those who are also
working in this area.