February 2013

Attribution or Contribution

15 Feb 2013

Daniel Robinson

Daniel is an evaluation expert and is passionate about supporting socially motivated ventures in evidencing their social impact as they grow.

Most people involved in philanthropy and social investment want to know if their efforts are making a positive difference, but seeking to demonstrate attribution (linking changes observed to a specific intervention) can be far from straightforward, and might even be the wrong question to ask.

Imagine, if you will, that you have commissioned a project to address high levels of unemployment in a particular community. Measurements have been made throughout, and now the project is drawing to a close, you look at the final results. Good news, employment is up against the baseline position at the start of your project, and appears to have turned around from what was previously a negative downward trend. However, your celebrations are short-lived when someone points out that shortly after the start of your project, and entirely independent of it, a new factory was opened down the road and that was responsible for creating hundreds of new jobs. Your project may (or may not) have achieved anything.

Read More

Measuring whether or not a positive social change has taken place is the first challenge. Determining if the change that occurred was down to your intervention is the next, and is at the root of questions around attribution. One approach to answer this might be rigorous experimentation, i.e. RCTs as they are specifically intended to test whether changes observed can be attributed to a specific intervention (e.g. a new medical treatment), or whether they would have happened anyway.

There is no small debate at present around the preeminence that should be given to RCTs in determining social policy and philanthropic activity, and whether the worlds of medicine and social policy are analogous? Another blog post perhaps. What is vital, is that in the pursuit of an improved understanding of attribution and the effectiveness of specific interventions, funders do not fall into the trap of believing that their projects and efforts, operating in isolation, will bring about lasting social change. Questions of attribution should lead to humility, but can often lead to hubris.

A useful corrective to this might be to talk more of contribution than attribution. That is, first to recognise that as funders our interventions might be necessary to effect social change, but are rarely sufficient (in other words they contribute towards the end results, but there are other things beyond are direct control that are also in important in this respect). Second, in terms of social and behavioural change we are looking at interventions in complex systems, which can raise some interesting challenges around demonstrating that our intervention was the direct cause of the changes that were observed. Seeking to analyse contribution also offers the opportunity to evaluate the difference an organisation has helped bring about, when the level of experimental design needed to demonstrate attribution might be neither practical nor feasible.

Another potential benefit of looking at our contribution to a particular challenge is that it fits with the growing recognition that working collaboratively and developing multi-level interventions are essential if governments and social enterprises are to bring about lasting social and behavioural change. It is widely recognised that the “complex nature of most social problems belies the idea that any single program or organisation, however well-managed and funded, can single-handedly create lasting social change.” (Hanley-Brown et al. 2012). If we are to work together, collaboratively to tackle profound social challenges, then it seems that we also need to ensure that our approaches to monitoring and evaluation recognise that our interventions are often attempting to contribute towards a certain outcome, but will never achieve it on their own.

I suspect that this might be where approaches such as Theories of Change (ToC), including the use of outcomes chains, explicitly articulated assumptions and risks, and emphasis on collaborative development and testing, might be useful. Working collaboratively with multiple stakeholders (inc. end users and a range of expert perspectives) can help you to identify, and be held accountable for, your specific contribution to addressing a particular social challenge. In terms of the question of attribution, this approach would have a number of consequences. First, organisations, including funders, would need to be more specific (and humble) about the changes they are seeking to attribute to their interventions. Second, even then, there would always need to be a constant appreciation of the contribution of other organisations, and wider environmental factors, to the changes observed. Third, attribution will need to be something that is shared and validated by the community of stakeholders that brought about the changes that have been observed.

Read Less