Measuring Impact - Nicva

12 Measuring Impact – Case-studies of impact assessment
Finally, a number of studies have tried to take into account the different perspectives of stakeholders
when assessing impact. A number of frameworks have been developed to address
this, including the New Economics Foundation’s Social Auditing.
Common themes 2.3
Common themes can be identified among existing tools and approaches. Perhaps the most
persistent is the involvement of the community and other stakeholders in the evaluation
process. Another is the idea that the voluntary sector differs fundamentally from the public
and private sectors, and any tool for impact assessment should reflect and capture this difference
or added value. Social capital is another important issue for the sector. Organisations are
thought to both make use of, and contribute to, social capital. While a number of
approaches 1 attempt to capture this, the field needs further development.
There is general agreement that not very much is known about the impact of the sector as a
whole, or sub-sectors and clusters of organisations. This is being rectified in part by a number
of recent studies that have attempted to address this. 2 However, these are not UK specific.
The range of methodologies being used to measure impact in the voluntary sector has also
been studied, though again this field is not UK oriented. 3
Another common feature is the development of toolkits, methodologies or guidelines for inhouse
use, with the recognition that they may be of interest to a wider audience. Examples
include Connexions’ A little Book of Evaluation, New Economics Foundation’s Prove it! and
the previously mentioned First Steps in Impact Assessment by Lloyds TSB Foundation.
Problems with existing approaches 2.4
No tool can claim to assess all aspects of impact. As a result, approaches are tailored to
specific types of impact or organisation. Nevertheless, there are still particular types of impact
that cannot be adequately identified or assessed using current tools. For example, no tools
exist to adequately assess unanticipated impacts. Most approaches establish a framework for
assessment at the beginning of the process, which might preclude the observation of unanticipated
impact during the measurement process. Unanticipated impacts include the negative
aspects of an organisation’s impact, rarely captured using any of the existing tools.
Similarly the short time scales of most approaches, and the need to be able to report impact
soon after a project is completed, rarely allow for the assessment of long-term impact. Also,
the more time that is allowed to pass between an activity and the assessment of its impacts,
the more the impact of the activity will be diluted by external factors. Finally, collective
impact (impact achieved by a cluster of organisations or projects) is difficult to measure using
any of the existing tools as the measurement process involves the co-operation of more than
one organisation or project.
1 For example Morrissey et al (2001) Building an Indicator Template for Evaluating Community-Based
and Voluntary Activity in Northern Ireland.
2 Salamon et al (2000) The Nonprofit Sector: For What and For Whom.
See also Independent Sector (2002) Balancing the Scales.
3 Independent Sector (2001) Outcome Measurement in Nonprofit Organizations.