Here comes the science bit: Aniston meets SROI

We have a provocation piece up today about social impact on Social Enterprise Live…Here it is in full:

Social impact measurement. Words to send a chill down the spine of many, and a shiver of excitement down some (you know who you are). For some, an utter turn-off, for others a near-impossibility, and for even more something to ignore or fear.

And yet, absolutely crucial. Whether you look at business guru Michael Porter’s recent writing on Shared Value, the growth of impact investing, the Social Value Bill, venture philanthropy, social impact bonds and more, the ability to measure social value and the impact of an organisation is absolutely central.

And, for the social entrepreneurs and social enterprises, not only crucial for communicating and proving their worth to those external agencies and investors, but also for improving their work, learning from their activity, motivating themselves and their team, and mobilising support.

But (and there are quite a few buts… ):
– there are more tools to choose from than in an over-excited DIY enthusiast’s garage;
– there is as much agreement on approaches as on when the Big Society Bank might open;
– there are those who think attribution is vital, and those who think it just doesn’t make sense;
– simply not enough impact measurement is going on at a grassroots, practitioner level;
– government says it’s important but has often summarily ignored it when making decisions about which organisations it should back (or save).

The debates have become stuck as well: SROI is critiqued for its cost and its ‘boiling down’ to a ratio, but its broad principles and international partnerships overlooked; multiple agencies work with individuals (as they should) but all claim the outcomes; individual sectors can’t agree standards or shared approaches; reporting and monitoring is mistaken for transparency and accountability; the different ‘camps’ have become entrenched; practitioners don’t have the necessary resources. And so on…

And the large grey, big-eared pachyderm in the lounge is the growing view that such impact and evaluation reports are just glorified marketing: solely aimed at an external audience and at arming the communications department with a handful of key statistics. There are those who point to the fact that evaluation is located in the marketing department, not in research. This means that an increasing number consider them the social enterprise equivalent of Jennifer Aniston saying, “Here comes the science bit” on the L’Oréal adverts: figures that give a veneer of credibility to the marketing of a product.

It’s time to shift the debate on, work out what’s next, untrench the entrenched (or something like that), work together (cue shocked gasps), take this stuff seriously (and implement improvements internally), and, most importantly, avoid the superficial attractions of glossy hair and prove that we are really, genuinely worth it.

Should be fairly simple.

This entry was posted in Social Impact. Bookmark the permalink.

2 Responses to Here comes the science bit: Aniston meets SROI

  1. Fergus Lyon says:

    An interesting post and I look forward to what comes from your discussions tomorrow. Three points come to mind from our research at TSRC.
    While lots of organisations I am talking to may be initially motivated by the marketing element, the process of measuring of impact is also a powerful tool that can really change their organisations. It requires good leadership to bring the two together.

    Seconldly, approaches such as SROI have the potential to be empower organisations and find ways of measuring what matters to them. However, it is this empowered approach to developing their impact measurement that makes comparing organisation against another impossible as they are measuring different things. We are now getting ‘ratio inflation’ as competing organisations use their discretion to make a decision of what to include or exclude. However, a common approach to measurement would mean organisations loose the control they have over what they measure. I am now interested to see how commissioners can use social impact measures without being too prescriptive. It is balancing act but I would be interested to hear what is happening around the country.

    Finally, the challenge for those of us looking at different ways of measuring impact is how to ensure these approaches gain greater legitimacy and are trusted. Auditing the measurement process is becoming more common but this adds another layer of costs and there appears to be little pressure on organisations to be audited at present. Can we find cheaper ways for organisations to measure their impact while keeping up the quality?

  2. “simply not enough impact measurement is going on at a grassroots, practitioner level”

    There is a lot of impact measurement going on. Unfortunately, at the field level, there is a serious lack of resources, in particular money to collect large bodies of data, and to hire experts like public health professionals, statisticians, etc. This does not mean that no information is being solicited, or gathered. There are thousands upon thousands of documents literally sitting in files worldwide, detailing impact or lack thereof, that only made it into the public sphere as footnotes to reports for bilateral and multilateral donors, whose report formats do not require impact reporting.

    Nor could they, because the funding cycle is too short to monitor or evaluate for impact in many projects–not to mention the fact that as I wrote above, they certainly don’t want to pay for anything but the most bare-bones monitoring systems.

    Because organizations compete for monies, with cost-effectiveness as a criteria, and bilateral donors knowing that the life of the project is not enough to measure long-term impact.

    It is a complex problem, as you suggest. The reason many different metrics are in play is not confusion, but the fact that positive, non-monetized change in the lives of people and societies looks different in each person and society. It is also relative–what might be a small change to one person, would be a change of great magnitude to another.

    “There are those who point to the fact that evaluation is located in the marketing department, not in research.”

    But it is not in fact located in marketing in most development organizations, or even amongst large and medium-sized donors.

    Fergus has some interesting points. The World Bank and UN have known about the difficulty of comparing apples to oranges in social projects for decades. As clunky as these huge organizations may be, they have already gone through this problem many times, and have funded some of the largest impact studies in the world on social issues. Social enterprises could learn something from looking at where they are now, such as with the Millennium Development Goals.

    “Can we find cheaper ways for organisations to measure their impact while keeping up the quality?”

    No. Social studies are notoriously complicated, social change takes a long time to effect, and it requires a lot of background knowledge as well as sophisticated knowledge of economics, anthropology, and the field in question (such as medicine, agriculture, women’s issues, etc.). Businesses spend a lot on tracking profits. Social enterprises will need to spend even more on tracking social change. The key is to build on overall efficiency and local knowledge (like in business) and to integrate learning and evaluation into the day-to-day running of the business and organization. That reduces duplication and thereby reduces costs.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s