Introducing Measuring Impact by Design

This month, the Impact and Innovation Unit is proud to launch our first ever impact measurement guidance, which we call Measuring Impact by Design. This document is the product of a year’s worth of collaborative effort, in which we worked with experts and potential users of this guide to compile a plain-language explanation of some key concepts related to impact measurement. It introduces some of the main ways in which we can measure impact for the projects we will be delivering under Impact Canada.

At its core, Impact Canada is about doing things differently to improve results for Canadians. Pay-for-success is a key means by which we do this, and as you will have seen, we’ve already launched a series of challenge prizes in diverse areas to this end. Pay-for-success can take many forms, and we have some exciting work planned to develop social impact bonds as well.

Measuring Impact by Design was written to equip ourselves and our partners with a general guideline for rigorous impact measurement. The basic premise of pay-for-success is that government only pays when a desired result is achieved. In this context, we need to be proactive and systematic in our measurement efforts, and use designs that can attribute results to our programs, ruling out other possible causes as much as possible. To this end, Measuring Impact by Design places a strong emphasis on experimental and quasi-experimental methods for measuring impact.

Fundamentally, this is a guide for the non-specialist. It is written for program and policy experts who are developing Impact Canada pay-for-success projects, who might not know what methods of impact measurement are available to them, what the advantages and disadvantages of various methods are, or which are best suited to which contexts. The goal is not to turn its readers into specialists, but rather to equip them with a deeper appreciation of the science of impact measurement, so that they can be critical consumers of it in their own pay-for-success work.

This is just the beginning. Over the coming months, watch for our impact measurement blog post series, which will dive a little deeper into the key messages of Measuring Impact by Design. We’ll be writing a little more on topics like randomisation and impact measurement, the relative advantages of experiments and quasi-experiments, why we don’t use a traditional evidence hierarchy, what the relationship between impact measurement and pay-for-success is, and how we are working with Statistics Canada to measure the impact of our own work on prizes and challenges.

So stay tuned! There is much more to come. In the meantime, we hope Measuring Impact by Design is useful to you. We love feedback, so please do not hesitate to reach out to us.