Measuring Impact - Nicva

16 Measuring Impact – Case-studies of impact assessment
Some forms of impact can be adequately described by quantitative measures. These might
include financial and physical capital, and even some aspects of human capital. However,
other forms of impact, especially the impact on social and cultural capital, cannot be
adequately described by quantitative measures alone and thus require a more qualitative
approach. Consequently, the studies presented in the next four chapters draw more strongly
on qualitative approaches.
Placing organisations within a broader context 3.4
One of the central themes underpinning the approaches in this report is the role of voluntary
organisations within a wider system or network of stakeholders. Voluntary organisations are
increasingly engaged in intricate webs of organisational partnerships, and understanding how
organisations add value to these partnerships will increasingly be necessary. Impact assessment
in such a complex setting requires a flexible, holistic and multi-perspective approach.
This report explores approaches that look at an organisation’s impact as part of a wider
system or network. Three of the studies presented here have drawn on basic systems principles
(the idea that an organisation operates as part of a network or system) and the use of
system diagrams or maps. This has been the framework for the collection and interpretation
of information. Our thinking is based loosely on ‘systems theory’, which argues that:
“A system is a set of interconnected variables, change in any one of which will affect all
the others... variables should not be analysed in isolation but rather for their
interrelation as part of a system.” (Seymour-Smith, 1986)
The basic principle of this theory is that it looks at the world holistically. The immediate
world is broken down into various parts which could be people, activities or organisations.
The analysis then focuses on the way in which each of these parts interacts with the others.
As an impact assessment tool, a systems approach seems to offer a number of advantages for
researching impact in the voluntary sector. 6 Organisations are embedded within complex
webs of relationships with other organisations, people and groups. Key stakeholders have
very different views on what they see as the significant issues of concern. Its holistic approach
assumes that impact is in part dependent on interrelationships with other organisations in the
system. It also recognises the synergies that occur as a result of such interdependencies. Visual
representations of networks and relationships also give a clearer sense of broader impacts.
There are however, a number of limitations to this approach. These include the difficulties in
defining the boundaries of a system, and the fact that the relationships between organisations
and groups in the system may change.
6 Also see Williams (2002) for a discussion of the strengths and weaknesses of systems approaches.