Monitoring, Measuring and Communicating Impact

Social innovation labs help address and tackle complex problems. However, evaluating the change we bring about for complex problems is extremely challenging. This is because:

• A complex problem changes as you try to solve it;

• The causes and drivers are interdependent;

• It is filled with uncertainties and unknowns; and

• It requires multiple new solutions which is impossible to know which will work best.

This makes attribution and causality very difficult. That is, if you’re measuring change of something in a changing environment (among other characteristics of a complex problem), it is difficult to attribute any cause and effect to your actions. It also involves many stakeholders with different values and priorities – which makes it difficult to agree on what change to bring about (and the scope of that change), as well as how to measure it.

This requires adaptive management, where you change and adapt your methodology as you go, rather than use a traditional cause-effect methodology. While the field of evaluation of social innovation labs is still very nascent, we clearly need to move beyond traditional methodology to find new frameworks to evaluate the efforts of labs in this emergent, unpredictable environment. As Michael Quinn Patton explains:

Not all forms of evaluation are helpful. Indeed, many forms of evaluation are the enemy of social innovation. This distinction is especially important at a time when funders are demanding accountability and shouting the virtues of “evidence-based” or “science-based” practice.

– Michael Quinn Patton (2016)

We suggest focusing on what you can learn from each prototyping cycle, and reintegrating that feedback into your prototyping design process (where each feedback loop gives you critical information about the prototype). In this way, you can both learn about what is working and not working, but reintegrate this back into the prototyping process.

For this reason, it’s important to think of your intervention or solution as a source of data, so that you can collect it for feedback. This means when you’re designing an intervention, you can already start to think about how you will encourage end users to help collect data. In doing so, you need to think about how the data might be useful to them as well as for your team.