Only by gathering evidence-based knowledge about what works, Torero argued, can aid agencies make a credible case for more donor support.
“It’s important to identify which interventions make sense and which ones don’t, and why,” he said.
Objective and systematic
Torero’s rapid-fire explanation of impact evaluation covered a lot of ground, as did the ensuing question-and-answer period. (Watch the full seminar here.) Following are a few of the presentation’s main themes.
- Causal links. Over the past decade – and especially amidst global recession and fiscal constraints – governments and other donors have increasingly demanded proof of the impact made by development policies and programmes. To meet these demands, aid organizations must clearly identify the causal pathways between interventions and results, and systematically gauge the impact of their work.
- Before and after. Objective impact evaluation hinges on a difficult task: determining what would have happened if the project in question never existed. This requires a comparison of indicators for two separate groups – beneficiaries and non-beneficiaries – both before and after the intervention.
- Control groups. Controlled trials are one approach to making such comparisons. The trials compare randomly selected members of a beneficiary group with their counterparts in a non-beneficiary control group. There are other methods of comparison, as well. No approach is perfect, but each can be helpful in addressing the issue of attribution – that is, finding out whether changes in the status of beneficiaries are attributable to a particular intervention.
- Outcomes vs. impacts. When choosing indicators to be used for evaluation, it’s important to recognize the distinction between the outcomes and the impacts of a development project. A rural electrification initiative, for example, might result in improved indoor air quality, as households use fewer polluting fuels for cooking, lighting and heating. In this case, better air quality is the outcome. From a human development standpoint, however, the relevant impact could be a reduction in respiratory diseases and deaths among children under five.
- Design for success. To ensure an accurate assessment and to reduce costs, the impact evaluation should be designed at the inception of a project, not imposed after the project is already operational. Optimally, interventions can be evaluated at the pilot stage and then brought to scale on the basis of a rigorous analysis of lessons learned early on.
Evaluation is, of course, simply a means to an end. For IFAD, the ultimate goal is to make a tangible, positive impact on the lives of poor smallholder farmers in the developing world. As Torero put it: “We’re not looking for a silver bullet. We’re trying to get people out of poverty on a sustainable basis.”
Missed the seminar? Watch the recording.