Scoping workshop – setting the stage for impact evaluation

By Line Kaspersen

It is common practice to have stakeholder workshops when projects close; to validate project completion report findings and communicate them to stakeholders. In Uganda we are trying something new; as part of the increased corporate focus on impact evaluations, a team of 4 statisticians have come to support the Community Agricultural Infrastructure Improvement Programme (CAIIP-1) Project Facilitation Team (PFT) to ensure high quality on the other hand. On the other hand, IFAD will be able to develop technical concept notes on how best to analyse the impact of infrastructural projects. For CAIIP-1 we will produce a vigorous impact assessment of this successful project!
What is happening is that several different activities are on-going:
A snapshot of how complex the impact evaluation can get.
  • A Results Impact Monitoring System (RIMS) study
  • An impact evaluation study 
  •  A Project Completion Report - PCR
  • Updating M&E databases – PFT
Each of the first three activities above is being handled by a different consultancy firm.  Having several firms doing different things at the same time created a challenge: How can we maximise the outcome of this work with so many stakeholders on board?
Bearing in mind that we need to indicate numbers of people pulled out of poverty, working with statisticians gives us a great opportunity to report confidently on this. In Uganda, we have linked the IFAD statisticians to the three different firms doing the RIMS, PCR and impact evaluation studies through the PFT.
In a one day stakeholder workshop, the PFT and the three firms came together to discuss how to: 
  • Harmonize the objectives and TORs of each consultancy firm (could one firm focus on community questionnaires and one on households?)
  • Develop a common sampling methodology prior to data collection (first we need to discuss which control-groups are valid? What do we do when we don’t have a baseline?)
  • Agree on the theory of change - what exactly we mean by impact? And on who?
  • Review the available documentation and understand it well  
The IFAD statisticians are expected to provide the following support to ensure better quality studies, which can serve as a dependable point of reference in regard to impact.
  • Technical support for data collection – developing questionnaires and training enumerators
  • Bringing in external data-sources and GIS resources
  • Specific cost-benefit analysis expertise
  • Support for coordination of the parties