Measuring impact in rural Vietnam: IFAD tests a new approach
|Children walk to school in Hoang Su Phi, Viet Nam, in 2002. |
Since then, the country has halved poverty overall.
This past week, staff and partners from the IFAD country office in Hanoi joined colleagues at headquarters in Rome by video link. They convened to discuss a newly expanded approach to monitoring and evaluating the impact of IFAD-supported projects in Viet Nam. It was the third in a series of research seminars co-sponsored by IFAD and the International Food Policy Research Institute (IFPRI), which has collaborated, as well, on developing this new approach.
The discussion was not without controversy. Still, there was a tangible sense of urgency, on all sides, about stepping up IFAD’s ability to accurately measure results on the ground – not only in Viet Nam but worldwide.
Thomas Elhaut, IFAD’s Director of Statistics and Studies for Development, moderated the seminar. He began by placing the Viet Nam experience in a broader context: namely, the ambitious target set during the latest replenishment of funding from IFAD member states. The target calls for lifting 80 million rural people out of poverty by 2015.
Without improved evaluation, Elhaut suggested, IFAD will not have the evidence base to design and implement projects that can help reduce poverty on such as large scale. Nor, in the end, will it have the data to credibly report on progress toward the goal of 80 million.
From Hanoi, Atsuko Toda of IFAD and Nicholas Minot of IFPRI went on to describe the approach being tested in Viet Nam and why, in their view, it may hold promise for other countries. The methodology builds upon IFAD’s existing results and impact management system, or RIMS, which has been in use since 2003. By expanding the standardized RIMS household survey, this approach – dubbed “RIMS-plus” – aims to capture more detailed data that is relevant to specific project aims. In another innovation, it uses a control group of households, located outside the target area, to better assess which results are attributable to a project.
Toda noted that the expanded questionnaire goes beyond the main objectives of the RIMS survey, which were to measure household assets and child nutrition. The RIMS-plus survey, she said, collects information on many additional indicators to diagnose constraints faced by farmers, including limits on access to rural markets, extension services and credit.
Minot added that the designers of the expanded questionnaire have still kept it relatively brief compared to most other household surveys; as a result, it should not overburden either enumerators or respondents.
Regarding the control groups, Minot explained that they would be selected from populations who live near project areas and whose backgrounds and economic status are similar to those of project beneficiaries. “It helps us to adjust for changes in the standard of living outside the project area,” he said. For example, a drought in the same province where a project is under way could cause an overall drop in rural incomes, which would have to be factored into any analysis of the project’s impact.
A third participant in the video link, Nguyen Ngoc Ahn of the Hanoi-based Development and Policies Research Centre, outlined the cost implications of both the extended questionnaire and the use of control groups. The questionnaire increases the amount of time needed to complete each survey and the training needed by enumerators, he said. The control groups add 300 households to the standard RIMS sample of 900 for large projects.
Costs of customization
At this point, some sceptical voices in the audience at IFAD headquarters questioned whether the new methodology’s value justified its added expense. Specifically, they asked whether the control groups could truly mirror the beneficiary households, given the complexities of rural poverty – including gender dynamics. In addition, they wondered whether the benefits of customized household surveys (more project-specific information) outweighed the advantages of standardized questionnaires (lower cost and consistent data across multiple projects).
In response, Toda noted that some of the expenses in Viet Nam represented the one-time cost of researching and developing a new evaluation tool. And while Minot acknowledged that the control groups could not match every household parameter, he maintained that they would provide valuable comparisons with project beneficiaries nonetheless.
On the question of standardized vs. customized surveys, Minot proposed a compromise: “We can use standardized modules but let each country decide which modules are most relevant for them,” he said. Such an arrangement would allow for a degree of customization without adding excessively to costs, he said.
As the seminar wrapped up, Thomas Elhaut praised the presenters in Hanoi for advancing IFAD’s thinking on the design and analysis of agricultural projects, even as various questions remain to be resolved. To some extent, those questions will be answered as the RIMS-plus experiment proceeds in the uplands and deltas of rural Viet Nam, where smallholder farmers need the tools to build a sustainable future.
Watch the recorded webcast of the IFAD-IFPRI seminar: