Social Impact Investing

SOCIAL IMPACT INVESTMENT: BUILDING THE EVIDENCE BASE
6.6. Challenges in SII data collection
6.69 Scoping and sizing the global SII market is an enormous challenge. There is no consensual
definition across different geographies and, above all, market players and researchers involved in SII. This
implies that most market estimates are not directly comparable. Data is not being collected in a
standardised and systematic way.
6.70 Most importantly, the lack of a consensual definition creates enormous challenges in identifying
the target population — i.e. what exactly is being measured. For example, the scope of SII demand cannot
possibly be identified as long as the boundaries of SII delivery organisations are blurry. Sampling schemes
avoid the need to capture information about the whole population by focusing on a representative subset.
However, sampling requires a clear definition of the population along with a number of key observable
characteristics. So, in the case of SII, even if a sampling frame is properly devised and a robust sampling
methodology is used, it results in biased estimates because it is unclear what the population of interest
should be in the first place. In other words, it is not possible to find a subset of the populations that,
according to some observable characteristics, is representative of the population. Therefore, estimates of
SII scoping exercises are usually biased towards certain sectors, instruments or investor types.
Accordingly, extrapolating total market sizes based on limited and unrepresentative samples should be
regarded with caution. Chapter 4 of this report provides a structure that helps defining SII, the basis for
engaging in data collection efforts.
6.71 In most measurement exercises, a common error is to exclude data that could be relevant (Type I
error). In the case of SII, focusing on a narrower scope can result in significantly incorrect measurements.
For example in Clark et al. (2013) only a selected number of intermediaries were analysed. The coverage
in the series of GIIN\JP Morgan investor surveys (Saltuk et al., 2011; 2013; 2014) has been increasing,
which reveals that some important investors might be (or have been) left out.
6.72 However, the early stage of SII market development can potentiate another type of measurement
errors: including data that is not relevant for SII (Type II error). For example, applying the working
definition described in Chapter 4, some delivery organisations can be wrongly included as SII investees.
As discussed in that section of the paper, some certified B-corps may not be included in the potential
demand for SII, because some of the certified companies would not meet the other necessary eligibility
criteria for SII. Another example can be found in the ASSB dataset in the UK, which samples social
enterprises from all existing firms with less than 250 employees (see Section 6.3).
6.73 The lack of a statistical definition of SII means that data is embedded in other broader data
categories. Disentangling what is the subset of information that corresponds to SII can be extremely
challenging. For example, in top-down approaches very strong assumptions are usually required, as
discussed earlier in this section. In addition there are several SII data layers (e.g. social need granularity)
which add substantial complexity to the analysis. A more granular approach may require data collection
efforts that are more resource intensive and might entail additional confidentiality issues. The trade-off
between level of detail and comprehensiveness of the information should be taken into account when
deciding to collect data. As an example, for the purpose of analysing SII transactions, survey respondents
might include investor platforms or the investors themselves. While focusing on investors provides a
greater level of detail, it multiplies the number of respondents, thus increasing data treatment and survey
management needs.
6.74 Finally, a common challenge in data collection is clarifying why, what and for whom data should
be collected. This is particularly important because different goals require different data types (and data
sources). Having a clear goal for data collection and well-defined data requirements is crucial. Two types
of measurement objectives were identified in the OECD SII Expert Meeting in London. On the one hand, it
100 © OECD 2015