Impact evaluations are central to keeping our development edge sharp

Although Millennium Villages Project had good intentions, it did not live up to its potential. Photo credit: Millennium Villages Project

A good deal of our work at ONE is focused on pressuring world leaders to support government funding for poverty-fighting programs in the world’s poorest places. Central to this fight is demonstrating that these programs are effective and deliver clear, sustainable results. This leads to real and important questions, such as how do we ensure that the funding is going to the right projects and how do we know that they’re working?

Experts across the academic and development practitioner world have devoted their careers to conducting “impact evaluations,” a formalized process that examines the effectiveness of interventions in a range of different environments. I had the opportunity to meet and speak with Michael Clemens from the Center for Global Development, where he leads the Migration and Development Initiative as a senior fellow. In addition to his current research on the effects of international migration on people from and in the developing world, Michael also devotes a large portion of his time to advocating for rigorous impact evaluation of aid projects.

Michael’s research is particularly relevant to our upcoming campaign on transparency in 2013, which is fundamentally focused on ensuring that scarce resources are spent effectively and produce accountable results.

Impact evaluations assess the results of a development project by comparing the outcomes with what would have happened had the project never been implemented. Michael suggests that independence, consistency and transparency are the three most important components of a well-designed and rigorous impact evaluation. These three factors are important because they make sure that assessments of projects are objective, well-defined and clear.

Michael pointed to the Millennium Villages Project as an example of a program that would have greatly benefitted from a more comprehensive and rigorous evaluation. A joint project between Columbia University and the United Nations, the program is being implemented in communities across the African continent to improve agriculture, education, health, water, sanitation and infrastructure simultaneously.

To date, the assessment of its success has been handicapped due to the fact that it violated each of the three core principles stated above: it was carried out internally by individuals affiliated with the program (not objective), the goals of the project were not consistent (not well-defined) and the direct results of the program are presented in an unclear fashion (not transparent).

However, the Millennium Villages Project may be learning from its past mistakes. Its newest effort in northern Ghana, largely driven by DfID’s involvement (the UK’s premier development agency), will include much more rigorous and independent evaluation of development outcomes over time. That is a clear sign of progress. And it is critical for identifying whether this approach is truly game-changing or something that is channeling scarce development resources away from much higher impact and more sustainable interventions. That is something that all hard-headed poverty-fighting advocates need to know: what approaches we should fight for and what approaches we should cut loose.

Impact evaluations are a critical part of development work because they ensure that the money invested in a particular program is generating real results in terms of sustainable and long-term growth. Progresa, a pro-poor intervention initiated by the Mexican government in 1997, is a great example of how a strong impact evaluation can provide non-partisan legitimacy and credibility.

Offering cash transfers to the rural poor on the condition that their children consistently attend school, health clinics, and pláticas (seminars on health and nutrition); Progresa (now called Oportunidades) has served as a model for others to emulate.

Photo caption: Oportunidades has been successful in fighting poverty in Mexico. Photo credit: UNESCO.  

Following an evidence-based pilot phase, it was later scaled-up and then continued by successive Mexican governments on a national basis. It has also helped spawn a variety of similar programs across the hemisphere, including those in Honduras, Colombia and Brazil.

Here at ONE, we’re constantly advocating for the biggest bang for the development buck. We are searching for the most effective ways to tackle the root causes of poverty and for supporting the most demonstrable and sustainable solutions. While they may not be relevant in every setting, better and more rigorous impact evaluations are sure to help the cause. The work of Michael and others committed to evidence-based approaches is something to support and keep a watchful eye on.

Find out more about Michael’s work here. Read the full version of Michael Clemens’ and Gabriel Demombynes’ paper on impact evaluations here.