by Bernadette Mukonyora and Marzia Perilli at the side event of the 36th GC - Measuring impact: Understanding pathways to rural transformation.
Aid effectiveness – a high priority. In his opening remarks of the pre-Governing Council seminar on Measuring Impact –Understanding Pathways to Rural Transformation, Carlos Sere, Chief Development Strategist of IFAD underlined the increased attention by the international community to measuring and reporting on the results of development interventions. IFAD is not the only organization giving itself measurable targets. Similar to IFAD’s 9th Replenishment commitment to lift 80 million people out of poverty, the European Union has set a goal of 2025, to reduce the number of children who are malnourished by 7 million while the New Alliance for Food Security and Nutrition aims to raise 50 million people out of poverty over the next 10 years through sustained and inclusive agricultural growth.
Projects in IFAD have always been evaluated. What’s new? A lot has been done on impact evaluation both within IFAD and outside but a lot more still needs to be done. Alessandra Garbero, Econometrician in SSD, outlined the challenges for evaluating impact on poverty in IFAD, particularly the lack of adequate baseline data and the complexity in the nature of IFAD projects. The novelty for IFAD will the design of impact evaluations into the project designs; theory-based impact evaluations with clearly mapped out causal chains; the use of credible counterfactuals; and, finally the use of panel structure to track beneficiaries over time in order to clearly attribute impact. In two word: more rigour.
Are rigorous impact evaluations worth it? Philip Davies, Head 3ie presented 8 reasons why we should measure impact, i) effectiveness, ii) attribution, iii) efficiency, iv) successful implementation, v) service orientation, vi) adaptation, vii) accountability, viii) democracy. Ultimately, rigorous impact evaluations help institutions understand the following: do their interventions work?, how to ensure implementation effectiveness?, identifying diversity of effectiveness i.e for whom are interventions effective? And when are they effective? It also helps to understand experiential effectiveness and resource effectiveness.
A toolbox of instruments exists for IFAD to tap into for rigorous impact evaluations. The Living Standards Measurement Study (LSMS) is another tool which offers many opportunities to inform IFAD’s theory of change agenda, said Gero Carletto, Lead Economist in the Development Economics Research Group at the World Bank.
The journey towards institutionalizing a learning agenda for IFAD does NOT begin at the end of the project. It was stressed that monitoring and evaluation methods should be included at the design stage, suggesting the need to harmonize operational and evaluation functions in IFAD. Jerry Clavesillas, IFAD Project Director in the Philippines, spoke about the importance of partnerships and coordination between the evaluation function and the operations and the ensuing success.
Partnerships are vital, IFAD cannot do it alone. Thomas Elhaut, Director of the Statistics and Studies for Development division pointed out the importance of donor harmonization. IFAD is working in partnership with a number of institutions, such as the USAID, DFID, BMGF, 3ie, WB, DANIDA, IFPRI, and many others, on a joint learning agenda which has the ultimate objective of harmonizing the results monitoring and impact evaluations efforts across institutions.
The other key factors for success… Ambassador of Mexico, Miguel Ruiz Cabañas Izquierdo left food for thought on the ‘other issues’ that need to be factored into the discussion. What do the beneficiaries think? Determine the viability and the commitment of stakeholders and adopt a ‘phased approach’ to interventions with short, mid and long term goals. Projects should also be evaluated independent of the implementers. Another issue is the second phase to successfully undertaking the impact evaluation i.e. what to do with the results? Institutions should also evaluate how good they are at ‘up taking’ the lessons learnt.
IFAD will require significant resources and identify partnerships to take this forward. Significant efforts towards partnerships would go a long way in ensuring the necessary coordination. A key challenge is that many development partners and governments are implementing different monitoring and evaluation systems which makes it difficult to harmonize results and ensure measurability.
What next? It is imperative that we learn fast as this is a challenge in the world today. To do so, diversity of approaches appears to be the way as well as partnerships and coordinated efforts. We need to invest in the public good of information in a coordinated manner. Finally, Carlos Sere also stressed the importance of this being the “beginning of a conversation” and encouraged the over 100 participants to engage with the “team” with ideas and approaches to further develop our action plan…all hands on deck to contribute to this huge challenge.