Dr. Saneeya Qureshi and Dr. Richard Hazenberg  from the Institute for Social Innovation and Impact at the University of Northampton have written an article making a case for the European Commission’s (EC’s) ‘Groupe d’Experts de la Commission sur l’Entrepreneuriat Social’ (GECES)’ Standard for social impact measurement.
The motivation for this blog post is a result of the increasing trend for social enterprises seeking to evidence their social impact and enhance their effectiveness. However, this trend is accompanied by challenges that arise from the plethora of avenues that can be taken to measure social impact. This post recognises that that there are no rigid set of indicators in a ‘one-size-fits-all’ strategy to measure social impact. However, the implementation of a broad framework on a regional level would enhance the comparability of various social impact measurement methods.

Organisational context

To begin with, there are a variety of motives behind why organisations seek to measure social impact. Reasons include amalgamating evidence for current and potential stakeholders, funders and beneficiaries; allowing for organisational learning and optimal allocation of resources; and successfully achieving the ‘social aspect’ that constitute social enterprises. The manner in which social impact is measured, is essentially an institutionally individualistic one, as there is no right or wrong answer, and the decision has to be shaped by key personnel within (and externally) to the organisation.

Past experience with social impact measurement indicated that the three main influencing factors are:

  • Motivational factors: why is your organisation engaging in SI measurement; what do you want to get out of it; and what impact do you want it to have?
  • Readiness factors: how soon can you engage in SI measurement?
  • Capacity factors: what is your institutional capacity regarding staff expertise, time, financial resources and external contacts/networks?

In the last decade, there have been a number of recommendations for various tools put forward as possible indicators for social impact measurement based on the three factors described above (McLoughlin et al., 2009; Barraket and Yousefpour, 2013; Hazenberg et al., 2014), as well as critiques of certain approaches, most notably ‘Social Return on Investment’ (SROI) (Gibbon and Colin, 2011; Millar and Hall, 2013; Wilson and Bull, 2013; Banke-Thomas et al., 2015). The goal of impact measurement, however, should not be to develop complicated algorithmic evaluations which differ from organisation to organisation, thereby precluding a common base for comparability. It is with this lack of a common standard and with the objective of the formation of a single, consistent social impact measurement standard across the European Union (EU), that the European Commission (EC) formed the GECES sub-committee on social impact measurement.

Development of the GECES Standard

As a result of the Single Market Act II outcome recommending that appointment of a group of experts on social entrepreneurship, GECES was set up on 5th June 2012 by the EC Group. Members are either experts designated by EC Member States, or persons selected by the Commission via a tendering procedure (including experts, CEOs, university professors, etc.) who have a mandate of five years. The Group’s key objectives include developing a rigorous and systematic methodology to measure the socio-economic benefits created by social enterprises; evidencing how the money invested in social enterprises yields high savings and income; and agreeing upon a European methodology which could be applied across the European social economy.

The GECES Standard, as recommended in the GECES Report was officially adopted by the EC on 3rd June 2014. In order to attain a standard that is equally applicable to the range of social enterprises that exist in the region, as well as the interventions that they deliver, GECES divides measurement into four key elements:

  • A common process of measurement designed to give the account of the intervention, its outcomes and how it achieves them, also known as the five stages for social enterprises to measure their social impact – these shall be examined in depth later in this article
  • Disclosure standards which are certain universal and mandatory characteristics that define measurement disclosure (reporting) that is of acceptable quality
  • The choice of Frameworks
  • The choice of Indicators

The latter two elements are not mandatory, although they are expected to meet the 80% rule: in that at least 80% of cases, similar interventions will be seeking similar outcomes. These elements must be:

  • reported against and communicated effectively and regularly to stakeholders
  • agreed between the social enterprise and the funder
  • selected by the social enterprise for their appropriateness in relation to the intervention concerned, the outcomes targeted to arise from it, and the stakeholders affected by it
  • regularly reviewed for appropriateness and updated or changed as needed

The GECES Standard is therefore essentially based around a commonality of approach – rather than method – in the measurement of social impact. It is a pan-European standard on impact measurement, which aims to enable social ventures to maximise their full potential, being based on a widely recognised flow, known as the Impact Value Chain (also known as the ‘Theory of Change’ or ‘Logic Model’).

 

The basic principles of social impact

Theory of Change refers to the means (or causal chain) by which activities achieve outcomes, and use resources (inputs) in doing that, taking into account vari­ables in the service delivery and the freedom of service-users to choose. It forms both a plan as to how the outcome is to be achieved, and an explanation of how it has occurred (explained after the event).

In terms of impact in the social sector, the theory of change, as illustrated in Figure 1 below, shows an organisation’s path from inputs and needs; through to activities, outcomes and impacts. It describes the change that an organisation intends to make through its activities, and the various steps involved in ensuring that the change occurs (Kail and Lumley, 2012). The flow chart below illustrates how an optimally implemented theory of change can reveal whether an organisation’s activities make sense, given their social impact aims (or vice versa) and whether there are things that the organisation could do better to achieve its social impact aims. The key here is for an organisation to understand the reasons why it wants to measure social impact in the wider context, and not simply for the sake of informing the value provided to funders and beneficiaries.

theory of change_Saneeya

Figure 1: Key terms within the Theory of Change as applied to social impact measurement (Source: The GECES Report)

The Theory of Change as illustrated above is therefore a useful tool for planning as well as reflection. However, the process is not as simplistic in implementation. Consideration must also be given to extraneous variables that impact the parameters described in Figure 1. It is recommended that social impact measurement should take into account three further adjustments. These three factors (illustrated in Figure 2 below) ensure a more robust reflection of social outcomes as measurements, both long-term and short-term, as they enable the adjustment of the impact for effects that would have happened anyway (deadweight), for external consequences (displacement), and for effects declining over time (drop-off).

All reporting of social impact measurement should include appropriate and proportionate evidence regarding details of not only how the Process (Figure 1 above) has been applied, and what has been done in each of the five stages; but also a clearly explained account of the effects of the intervention with regard to the qualitative aspects of deadweight, displacement and drop-off (Figure 2 below).

social_impact_saneeya

Figure 2: The three further adjustments to be considered in evaluating impact based on outcomes

It is worth mentioning that during the course of social impact measurement, whilst benefitting the robustness of the evaluation, taking into consideration these three additional adjustments (deadweight; displacement; drop-off) also has a resultant increase in costs and complexity with regard to the capture of additional data.

The 5 stages for social enterprises to measure their social impact

The GECES Report informs the Standard agreed upon by the EC and sets out five easy-to-understand steps for social ventures to implement impact measurement:

managingImpact_Saneeya

Figure 3: The five stages of social impact measurement (Source: The GECES Report)

The five stages, as illustrated in Figure 3 above, are:

  1. Setting Objectives: identify the goals of the various parties in seeking measurement and of the service being measured – what is it intended to do and how? This will establish target beneficiaries, outcomes, activities and the theory of change.
  2. Analysing Stakeholders: identifying who gains and who gives what and how? What is their level of engagement with, control over, and contribution to achieving the desired objectives and the outcomes and impacts that come with them? It is essential for organisations to appropriately assess the relative importance of different stakeholders and their needs in setting the measurements required
  3. Measuring Results: setting relevant parameters by which the social enterprise will plan its intervention, and how the activity achieves the outcomes and impacts most needed by its beneficiaries and stakeholders. This link from activity to impact is the social enterprise’s ‘theory of change’. It will decide this, and establish measurement most appropriate to explaining the theory of change and the achieved impacts, and will then agree it with major stakeholders.
  4. Measuring and Valuing Impact: measuring, validating, valuing and assessing whether the targeted outcomes are actually achieved in practice, whether they are apparent to the stakeholder intended to benefit, and whether they are valuable to that stakeholder.
  5. Monitoring and Reporting: reporting, learning and improving as the services are delivered and the measurements of their effectiveness emerge, so these results are reported regularly and meaningfully to internal and external audiences.

Each of the five stages of the process outlined above is relevant to and involves stakeholders at all levels – ranging from investors to service-users. Indeed, this multi-stakeholder approach to the design, delivery and evaluation of social interventions is a process that was recommended by Hazenberg et al. (2014) when examining employability interventions with unemployed graduates. It is essentially through the dynamics of the involvement and engagement of all stakeholders that the balance is achieved and maintained between the overriding need to deliver measurable social impact as against the need for a profitable operation of the social enterprise that can meet investor expectations.

How does the UK fit into this?

The contention here is that there is currently no gold standard or best practice model of how to go about social impact measurement for social enterprises not only in the UK, but also across Europe too. This means that there are a large number of uninformed consumers of social impact evaluation being asked to make important decisions with little information on which approach to use in a saturated marketplace. This information asymmetry between consumer and provider can (and often does) lead to negative market outcomes, which can result in social enterprises (and other organisations) paying for expensive social impact evaluations that are of limited validity and hence value. The GECES Standard provides a way of reducing this uncertainty by providing a common approach and guide that social enterprises can use.

In terms of applicability to the UK, in addition to the elements mentioned above, other aspects of the GECES Standard also set benchmarks for accountability and commissioning themes. These include shared value via the delivery of social benefits to stakeholders; integrated reporting relating to the six capitals (financial capital, manufactured capital, intellectual capital, human capital, social and relationship capital and natural capital); and social value in commissioning. Considered in tandem with the UK Public Services (Social Value) Act 2012 and the Lord Young Review 2015, both of which advocate a broad and consistent approach to impact measurement, the GECES framework offers a robust guide to ‘best practice’ in social impact measurement.

 To conclude it should also be noted that there are growing grounds for optimism relating to challenging the current status quo in social impact measurement in the UK: currently E3M and its founding partners have coordinated a working group with the intention of bringing the GECES discussion to the wider public. A policy paper based on initial meetings has been circulated and the E3M partners are assessing ways forward in working with the key UK stakeholders in impact measurement to take forward the next steps recommended within the policy paper (Clifford and Hazenberg, 2015). Time will tell as to how successful this initiative, and ultimately the widespread adoption of GECES, actually is.

 

 

References:

Banke – Thomas, A.O., Madaj, B., Charles, A., Van Den Broek, N., (2015) Social Return on Investment (SROI) methodology to account for value for money of public health interventions: a systematic review, BMC Public Health, 15, pp. 582-595.

Barraket, J. and Yousefpour, N., (2013) Evaluation and Social Impact Measurement Amongst Small to Medium Social Enterprises: Process, Purpose and Value, Australian Journal of Public Administration, 72(4), pp. 447-458.

Clifford, J. and Hazenberg, R. (2015) E3M-led review for social impact measurement strategy: Aligning the needs and requirements for social investment, commissioning for social value and effective social enterprise. [Online] Available from:  http://e3m.org.uk/wp-content/uploads/2015/03/E3M_Impact-Policy-Paper_March-2015.pdf [Accessed 21st  September 2015]

Gibbon, J. and Dey, C., (2011) Developments in Social Impact Measurement in the Third Sector: Scaling Up or Dumbing Down? Social and Environmental Accountability Journal, 31(1), pp. 63-72.

Hazenberg, R., Seddon, F. and Denny, S., (2014) Programme Recruitment & Evaluation: The effect of an employability enhancement programme on the general self-efficacy levels of unemployed graduates, Journal of Education & Work, Taylor & Francis (DOI: 10.1080/13639080.2014.900165).

Kail, A. and Lumley, T. (2012) Theory of Change: The beginning of making a difference. April 2012. [Online] Available from:  http://www.thinknpc.org/publications/theory-of-change/ [Accessed 21st  September 2015]

 McLoughlin, J., Kaminski, J., Sodagar, B., Khan, S., Harris, R., Arnaudo, G., and Mc Brearty, S., (2009) A strategic approach to social impact measurement of social enterprises: The SIMPLE methodology, Social Enterprise Journal, 2(2), pp. 154-178.

Millar, R. and Hall, K., (2013) Social Return on Investment (SROI) and Performance Measurement: The Opportunities and Barriers for Social Enterprises in Health and Social Care, Public Management Review, 15(6), pp. 923-941 DOI:10.1080/14719037.2012.698857

Wilson, D. and Bull, M., (2013) SROI in practice: the Wooden Canal Boat Society, Social Enterprise Journal, 9(3), pp. 315-325.