Scoping workshop – setting the stage for impact evaluation

By Line Kaspersen

It is common practice to have stakeholder workshops when projects close; to validate project completion report findings and communicate them to stakeholders. In Uganda we are trying something new; as part of the increased corporate focus on impact evaluations, a team of 4 statisticians have come to support the Community Agricultural Infrastructure Improvement Programme (CAIIP-1) Project Facilitation Team (PFT) to ensure high quality on the other hand. On the other hand, IFAD will be able to develop technical concept notes on how best to analyse the impact of infrastructural projects. For CAIIP-1 we will produce a vigorous impact assessment of this successful project!
What is happening is that several different activities are on-going:
A snapshot of how complex the impact evaluation can get.
  • A Results Impact Monitoring System (RIMS) study
  • An impact evaluation study 
  •  A Project Completion Report - PCR
  • Updating M&E databases – PFT
Each of the first three activities above is being handled by a different consultancy firm.  Having several firms doing different things at the same time created a challenge: How can we maximise the outcome of this work with so many stakeholders on board?
Bearing in mind that we need to indicate numbers of people pulled out of poverty, working with statisticians gives us a great opportunity to report confidently on this. In Uganda, we have linked the IFAD statisticians to the three different firms doing the RIMS, PCR and impact evaluation studies through the PFT.
In a one day stakeholder workshop, the PFT and the three firms came together to discuss how to: 
  • Harmonize the objectives and TORs of each consultancy firm (could one firm focus on community questionnaires and one on households?)
  • Develop a common sampling methodology prior to data collection (first we need to discuss which control-groups are valid? What do we do when we don’t have a baseline?)
  • Agree on the theory of change - what exactly we mean by impact? And on who?
  • Review the available documentation and understand it well  
The IFAD statisticians are expected to provide the following support to ensure better quality studies, which can serve as a dependable point of reference in regard to impact.
  • Technical support for data collection – developing questionnaires and training enumerators
  • Bringing in external data-sources and GIS resources
  • Specific cost-benefit analysis expertise
  • Support for coordination of the parties


Jesse Atkinson said…
This meeting to coordinate impact evaluation efforts up front is a great idea. We have seen the challenge of retrospective evaluation and now are ready to plan ahead for better results and linkages between consultancy firms. I support this idea being applied in LAC region as well.
Line Kaspersen (Uganda) said…
Dear Jesse, thanks for your feedback. It is very key to plan out the whole strategy for impact evaluation, and to work closely with the consultancy firms to make sure that the questions we want answered are answered with good statistical analysis behind it. We will attempt to share as much as possible from our experiences.