By Chris Barnett

In this third part of our blog series on ‘hot debates in impact evaluation’, we set out a number of areas in which we have ambitions to further our work at the Centre for Development Impact (CDI). These are areas that chime with broader debates, but we have yet to do significant research on. Here goes…

Power behind statistics: the potential of participatory impact assessments

In Jeremy Holland’s recent book on ‘Who Counts?’ there is a potential win-win of creating the statistically credible numbers required by many organisations, but doing it in such a way that it involves and empowers citizens at the same time. Of course, participatory research and action has a long tradition at the Institute of Development Studies, while the University of East Anglia is one of the birthplaces of the ‘democratic evaluation’ paradigm (both organisations are partners of CDI). At CDI, this interests us, having hosted seminars on participatory statistics, as well as jointly with the STEPS Centre on participatory impact pathway analysis. We are looking at ways to do further work in this area.

What does it all add up to? Alternatives ways to synthesizing and using impact evidence

Data science versus evaluation? Perhaps the emergence of ‘data science’ makes evaluation irrelevant; with masses of data (‘big data’) now available, data science offers a tantalising future where we can gain invaluable insights at little cost. Of course, apart from the data itself – even though it is now more widely available than ever - there are significant challenges in effectively making use of any type of evidence for social policy. This is something that Nancy Cartwright and Jeremy Hardie explore so eloquently in their book on ‘Evidence-Based Policy’.

So, while we have established ways of synthesizing evidence (such as the Systematic Review), these approaches often discard important perspectives and evidence. How we weigh-up evidence for social policy is important – synthesizing experiments as if society was a science lab has its limits. At CDI, we are presently exploring alternatives ways, like realist synthesis, as well as how to empower citizens in the deliberation of evaluation evidence as part of the policymaking process.

The SDGs: Linking evaluation across the natural and social sciences

As we move closer to the launch of the Sustainable Development Goals (SDGs), then evaluations of impact will need to consider both the societal and environmental dimensions. At the moment, a lot of discussion seems to be focused on the sheer number of SDG indicators – with grave concerns about the evidence demands of the SDGs (see Morten Jerven’s blog, for example). But a bigger challenge for evaluators is how to consider the timing, spatial and scale dimensions of impact – all of which likely to operate in different ways as the social system interacts with the natural (eco)system. So, while evaluators have traditionally drawn from the social sciences, we may now need to draw more heavily from the natural sciences, and better understand the interaction between the two. How then might this be best achieved?

I hope that in this series of three blogs I have given you a taster of the breadth and depth of our work at CDI. Do look at our new website. If there are areas, such as those above that you wish to collaborate with and fund, please do get in touch.

Chris Barnett is Director of the Centre for Development Impact, and Director of Itad. 

Partner(s): Institute of Development Studies