Monitoring and evaluation is an essential part of results-based management approaches to development assistance. Impact evaluations provide a powerful way to determine what works
and what does not. They focus on observed outcomes and use methods designed to establish causal links from interventions to outcomes. When conducted in comparable and consistent ways across countries, impact evaluations can provide useful benchmarks for program design and monitoring. The World Bank is taking advantage of advances in data and methodology to expand its use of impact evaluations, often through collaboration with governments and the research community, and to apply the results in its operational work.
Role and value of impact evaluations
The World Bank’s Development Research Group has been at the frontier of impact evaluation. For a decade or so it has experimented with new methodologies and has tested them on a variety of programs. It has built significant expertise in this area and has gained the recognition of many academics in the field. It is hoped that contributions from the KCP will help to continue and expand this work.
In the past, impact evaluations were fundamentally constrained by lack of data and by the technical challenges of developing a counterfactual (i.e., what would have happened without the project or program being evaluated). But over the past few years, significant improvements in both these areas
have made impact evaluations easier to implement on a systematic basis: microeconomic data gathered through household or community-level surveys are more available and a range of evaluation techniques—from randomized experiments to quasi-experimental techniques—has been developed to construct the counterfactual.
Much work remains to be done, however. Currently, most evaluations focus on interventions in the human development policy area and for a limited number of countries. Methodologies need to be developed or further tested and refined for other critical interventions such as investments in infrastructure for water supply and roads, or governance reforms. Remaining data limitations as well as the complexity of the policy questions faced by the development community demand further research and innovation on sound and practical approaches to impact evaluation. Much is to be
gained, too, from undertaking more evaluations of projects in areas for which methodologies have already been established. Comparative analyses of results from such evaluations will provide critical inputs to both results-based management efforts and evidence-based policy advice offered to World
The knowledge generated by impact evaluations is an international public good that can benefit the whole development community. Without international financing and leadership, the status quo of insufficient and inadequate evaluation of the effectiveness of development programs is likely to continue.
The World Bank is well placed to help countries implement sound impact evaluations of their programs as well as to facilitate global learning on effective development interventions, based on such evaluations.
Role and value of impact evaluations
Impact evaluations generate knowledge on which sorts of programs create substantive results and which do not, and under what circumstances. Such information is critical not only for the policymakers directly in charge of the program evaluated, but also for others who may be considering
adapting its approach for use in their own circumstances.
Moreover, when comparable interventions are evaluated across countries and regions, the results may serve three additional purposes. First, they provide the empirical basis to determine reliable goals for performance, both at the project and country level (such as what reduction in malnutrition can be expected from a child health program). Second, they help assess the relative effectiveness of alternative designs of development programs under different country contexts and settings (such as the effectiveness of conditional cash transfer programs in rural versus urban settings, or under centralized versus decentralized management). Third, impact evaluations are also a fundamental research tool. Appropriately designed, they elucidate not only the overall connection between interventions and outputs but also channels of causation. They permit the rigorous testing of hypotheses and thus the accumulation of fundamental knowledge. It is this last function that implies their relevance to the KCP and which will guide the evaluation supported by the Trust Fund.
Over the past few years, impact evaluation activities have been boosted and have become better coordinated across the Bank through the Development Impact Evaluation (DIME) initiative led by the Chief Economist’s office. Under this initiative, the Bank is supporting the systematic evaluation of programs on a number of strategic priority themes that are selected on the advice of, or demand from, operational teams. A series of coordinated but independent program evaluations are being undertaken in education (school-based management, teacher contracting, role of information in enhancing school performance), early childhood development (nutrition, pre-school), social protection (conditional cash transfers, school-to-work transition), health (HIV/AIDS treatment and prevention, malaria control programs, pay-for-performance models) and infrastructure (urban slum upgrading programs, rural electrification, rural roads).
For additional information, please visit DIME website.