Measuring Impact - Nicva
Measuring Impact - Nicva
Measuring Impact - Nicva
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
Assessing the impact of regeneration projects 35<br />
Critique of methodology 5.4<br />
A number of lessons were learnt as a result of this approach for identifying different impacts.<br />
Some concerned the use of a qualitative approach, whereas some were specific to the regeneration<br />
partnership.<br />
5.4.1 Qualitative approaches<br />
In a study where interviews are being used to establish the range of impacts resulting from<br />
particular activities, it is sometimes difficult to know to what extent the full range of possible<br />
impacts has been successfully captured. Similarly, we will never know whether the full range<br />
of stakeholders has been identified.<br />
Another problem with this type of approach is that it is not always possible to know whether<br />
interviewees are simply quoting from their organisation’s plans or objectives, and giving the<br />
‘accepted line’ as regards particular types of impact. However, these issues can be addressed<br />
by careful use of interview techniques.<br />
5.4.2 Issues of unanticipated,collective and long-term impact assessment<br />
Interviewees admitted that they don’t actually know what the impact of their work is. This<br />
perhaps explains why, when asked about impact, people often talk about intended impact and<br />
what they hope to achieve. There is an admission too, that knowledge of a project’s impact<br />
often comes from hearsay, the sources of which may not be reliable. Sometimes the very nature<br />
of a project means that some elements of its impact will not be directly visible to those involved<br />
in delivering the project. Another problem is that assessing impact is particularly difficult when<br />
the activity’s aim is to slow down a negative effect or prevent something from happening.<br />
It was hoped that this approach would prove a successful means of capturing unanticipated<br />
(including negative), collective and long-term impacts. As a whole the approach worked well,<br />
but there were some limitations, particularly with regard to long-term and collective impact.<br />
People found it hard to talk about the possible future impacts of a particular activity or<br />
organisation. There was a general tendency, when talking about the future, to say things like<br />
“well, what we hope to achieve is…” and “what our plan states is…”. In the absence of any<br />
anecdotal evidence to illustrate the long-term or future impact of a project, people tend to fall<br />
back on quoting the planned impacts, or objectives.<br />
Identifying collective impact also proved difficult. An assumption was made that the projects<br />
would consider themselves to be part of a larger whole or entity and share common aims. It<br />
seemed however, that while projects certainly shared some common objectives and aims, most<br />
objectives were organisation-specific. As a result, establishing collective impact in terms of<br />
common aims was not possible. It was more relevant to explore the existence of collective<br />
impact generated by the chosen projects and other funders or partners. These included development<br />
agencies and other partnerships. Here, relationships and shared objectives tended to<br />
be strong, and project managers had a clear sense of how they worked together to achieve<br />
things that would not otherwise have been achieved.