Measuring Impact - Nicva

Impact assessment involving users 39
Methodology 6.2
A three-stage plan was developed as the framework for the study. Three members of the
refugee community were trained to act as research assistants. These were either ex-service
users of EMEC or had some personal experience of the recent arrivals community. 12
The three principal components of our approach are identified in the following sections.
6.2.1 Stage one: Understanding the organisation and its context
A key issue for many voluntary and community organisations is that the environment within
which they operate is suboptimal (for example, providing services where market failure has
occurred). So, identification of the operating context is key to understanding how the organisation
makes a difference. It makes sense then, to begin with a mapping exercise to identify the context in
which the organisation operates and the other organisations and stakeholders involved.
This mapping exercise is based on consultation with key stakeholders, including project
workers, partner organisations and other service providers. A ‘snowball’ approach was used
to identify these stakeholders, starting from an initial set of contacts provided by EMEC and
other Scottish Council for Voluntary Organisations (SCVO) contacts. Preliminary face-to-face
interviews were then carried out up to one level beyond the initial contacts, i.e. initial stakeholders
plus a selection of stakeholders identified by them.
The research co-ordinator conducted interviews with ten project co-ordinators from voluntary
organisations identified on the stakeholder map, five counselling and managerial staff
from EMEC and an associated project. At each interview the developing network map was
shown to the interviewees for comment and amendment. The size, proximity and direction of
relationships, as represented by arrows, were subjective and based on stakeholders’ perceptions
of the nature of the relationships and the interrelation between activities.
Contextual information from the other stages also informs this mapping exercise. A comparison
between these stakeholders’ perspectives helps to map the organisation and its web of
relations and partners. Figure 6.1 (overleaf) shows the system map as developed during the
project. It shows the wider system in which the organisation is situated, and is based upon
organisational networks.
The diagram illustrates that voluntary and community sector organisations like EMEC are
involved in a range of partnerships with public, private and academic sectors as well as joint
initiatives with other voluntary organisations. This kind of mutually inclusive activity appears
to be mediated by a series of networks (white circles in Figure 6.1) and joint-partnership projects
(grey circles). Interviews with the service providers pointed to an extensive system of
client referrals between the voluntary organisations on the map. The model suggests a distinct
role, whereby voluntary organisations such as EMEC appear to fill in gaps in service provision
rather than trying to compete with each other.
12 See Tomlins et al, for study using ethnic minority research assistants.