Click here for search results

Newsletter

Finance and Private Sector Development Impact Evaluations

Evaluation of development projects is key to understanding and quantifying impact and also for learning and improving project content and delivery for future implementation. Impact evaluation entails the use of rigorous statistical methodology to identify the causal impact of development projects. While there are many different evaluation methods to choose from, the common theme among all of them is establishing a credible counter-factual -- i.e. what would have happened in the absence of the development project. Randomized evaluations, where individuals or groups of individuals are randomly selected to receive a development intervention and another set of individuals or groups are randomly selected not to receive the treatment (but are still observed and surveyed), allows for the cleanest and most rigorous inference. FPD staff are involved in many impact evaluations that cover a range of topics, from financial literacy, risk and insurance, to microenterprises and informality among SMEs.    


 

The purpose of this note is to summarize some of our existing and ongoing impact evaluations in the areas of finance and private sector development. We are interested in evaluating the impact of interesting policy questions, and use a range of modern evaluation tools such as randomized experiments, difference-in-difference, propensity-score matching and other evaluation designs to answer the questions of interest.

The summary below includes details of impact evaluations designed to:

EXISTING AND ONGOING IMPACT EVALUATIONS

Impact of Innovations to Induce Greater Use of Formal Financial Services

Financial Literacy and Use of Financial Services
Country: Indonesia
Methodology: Randomized Experiment
Researchers: Shawn Cole and Bilal Zia
Access to financial services is increasingly recognized as critically important to the micro-foundations of economic development. While much attention has been focused on access to credit, very little work has been conducted on what determines individuals in developing countries to open and maintain bank savings accounts. Savings accounts are likely the first and most obvious way in which household participation in the formal financial sector begins. This research focuses on providing basic financial literacy training about bank savings accounts to households in a group-class setting. We employ randomized evaluations to answer the following questions: (a) Does financial literacy training influence the willingness of households to open bank accounts, and save? (b) Do peer effects significantly influence household decisions about financial product consumption? And (c), what impacts do (a) and (b) have on household welfare?
Status: Ongoing

Impact Evaluations of Changes in Business Inputs

Returns to Capital in Microenterprises
Countries: Sri Lanka, Mexico
Methodology: Randomized Experiment
Researchers: David McKenzie, Chris Woodruff and Suresh de Mel.
The rapid expansion of microfinance services is based on the belief that microenterprises have productive investment opportunities and can enjoy high returns to capital if given the opportunity. However, measuring the return to capital is complicated by unobserved factors such as entrepreneurial ability and demand shocks, which are likely to be correlated with capital stock. Randomized experiments in Sri Lanka and Mexico have been used to overcome this problem and measure the return to capital. We accomplish this by providing cash and equipment grants to a panel survey of small firms, and measuring the increase in profits arising from this exogenous (positive) shock to capital stock. We find the average real return to capital to be around 6 percent per month in Sri Lanka and at least 20 percent per month in Mexico, substantially higher than the market interest rate. Returns are found to be highest for the most credit constrained.
Status: Paper forthcoming in the Quarterly Journal of Economics; working paper on Mexico available

Gender and Microenterprise Returns
Country: Sri Lanka
Methodology: Randomized Experiment
Researchers: David McKenzie, Chris Woodruff and Suresh de Mel.
Many microfinance organizations lend predominately or almost exclusively to women. Many of the justifications are economic in nature, and are based on the assumption that female entrepreneurs are more credit constrained and use resources more efficiently. If so, then the return to capital should, at the margin, be higher in female-owned enterprises than in male-owned enterprises. We provide the first test of the hypothesis using the randomized experiment on returns to capital described above. The startling finding is that while returns to capital average 9 percent per month for male firms, the average female firm has a zero or negative return from our treatments. The low returns do not appear to be a result of females taking the grants out of the business and spending them on household investments. Nor are they due to differences in ability of male and female owners. Part of the effect is due to females working in different industries than males, but we find female returns to be lower than male returns even for females working in the same industries as men.
Status: Preliminary paper available

Microenterprise Recovery after a Disaster: Experimental Evidence from the Sri Lankan Tsunami
Country: Sri Lanka
Methodology: Randomized Experiment, and Difference-in-Differences
Researchers: David McKenzie, Chris Woodruff and Suresh de Mel
We use quarterly panel data, coupled with experiments which gave aid in-kind and in the form of cash transfers to investigate the factors which determine how quickly small firms can recover from the shock of the December 2004 tsunami. In particular, we will examine whether recovery occurs faster for firms with more access to capital, for owners with higher education and ability, and for industries less focused on local demand. This will have implications for the appropriate interventions to be pursued in other disaster situations.
Status: Preliminary paper available


Impact Evaluation of Microfinance Contracts and Operation

Testing the Impact of Joint Liability Contracts
Countries: Philippines, Bolivia
Methodology: Randomized Field Experiment
Researchers: Xavier Gine, Dean Karlan
Group liability is often portrayed as the key innovation that led to the explosion of the microcredit movement, which started with the Grameen Bank in the 1970s and continues on today with hundreds of institutions around the world. Group liability claims to improve repayment rates and lower transaction costs when lending to the poor by providing incentives for peers to screen, monitor and enforce each other’s loans. However, some argue that group liability creates excessive pressure and discourages good clients from borrowing, jeopardizing both growth and sustainability. We worked with a bank in the Philippines to conduct a field experiment to examine these issues. We randomly assigned half of the 169 pre-existing group liability “centers” of approximately twenty women to individual-liability centers (treatment) and kept the other half as-is with group liability (control). We find that converting to individual liability from group liability but keeping other aspects of group lending such as weekly repayment meeting does not affect the repayment rate, but leads to higher outreach by attracting new clients.
Status: Two working papers available (one using one-year results and another using 3-year results for the Philippines. Bolivia is still work in progress)

What Works in Microfinance Contracts?
Country: Peru
Methodology: Framed Field Experiment
Researchers: Xavier Gine, Dean Karlan, Jonathan Morduch, Pamela Jakiela
Microfinance has been heralded as an effective way to address imperfections in credit markets. From a theoretical perspective, however, the success of microfinance contracts has puzzling elements. In particular, the group-based mechanisms often employed are vulnerable to free-riding and collusion, although they can also reduce moral hazard and improve selection. We created an experimental economics laboratory in a large urban market in Lima, Peru and over seven months conducted eleven different games that allow us to unpack microfinance mechanisms in a systematic way. We find that risk-taking broadly conforms to predicted patterns, but that behavior is safer than optimal. The results help to explain why pioneering microfinance institutions have been moving away from group-based contracts. The work also provides an example of how to use framed field experiments as a methodological bridge between laboratory and field experiments.
Status: Working paper available

The Impact of an Incentive Scheme to Credit Staff
Country: Pakistan
Methodology: Field Experiment
Researchers: Xavier Gine, Ghazala Mansuri
The availability of highly motivated and well trained field staff has been recognized as one of the key constraints to the rapid expansion of CDD programs. Given the complex and multi-faceted nature of program objectives, the incentive structure facing facilitators may also be quite important since there may be important trade-offs across program objectives. We are working with the Rural Support Program in Pakistan, a nation-wide initiative to implement an incentive scheme for their field staff in line with the twin objectives of the institution. The incentive contracts will reward (i) the quality of the loan portfolio and (ii) the quality of the COs. Strong financial performance is crucial for the sustainability and scaling up of NRSP operations, while cohesive COs are crucial for effective social mobilization. While incentives for credit performance are more conventional in micro-credit programs, an unstated hypothesis is that a more cohesive and active community organization may ultimately improve credit performance in addition to generating broader collective action capacity within the community.
Status: Work in progress

Impact Evaluations of Provision of Credit and Insurance to Enterprises

Impact of Subsidized Exporter Credit on Firm Outcomes
Country: Pakistan
Methodology: Natural Experiment, Difference-in-Difference
Researcher: Bilal Zia
The provision of subsidized credit to exporting firms is widespread in emerging markets. To what extent are such incentives useful in alleviating financial constraints and promoting firm growth? In addition, are more financially constrained firms allocated a greater share of credit? This paper combines an exogenous shock to the supply of subsidized credit with unique loan-level data from the export sector in Pakistan to identify the impact of such financial incentives on real firm outcomes. The removal of subsidized credit causes a significant decline in the exports of privately owned firms, while the exports of publicly listed and corporate group firms are unaffected. Publicly listed firms make no significant adjustments to their balance sheets, and only their profits are reduced, indicating that they are financially unconstrained. Nearly half of all subsidized loans are assigned to such firms, implying a substantial misallocation of credit. The analysis also shows that productivity differences cannot explain the heterogeneous effects across firms.
Status: Paper forthcoming in the Journal of Financial Economics

Getting Credit to High Return Microenterprises: The Results of an Information Intervention
Country: Sri Lanka
Methodology: Randomized Experiment, and Difference-in-Differences
Researchers: David McKenzie, Chris Woodruff and Suresh de Mel
We conducted an information intervention, in which these microenterprises with high average returns to capital were invited to a meeting with a local development bank, and given information on loan options available and how to apply for credit. We then investigate the determinants of which firms show up for this meeting, finding they are positively selected in terms of returns to capital. The intervention succeeds in getting a large fraction of the firms to apply for loans. However, relatively few of those applying are granted loans. We use our rich panel dataset of information on these firms to look at the screening decision of the firms, and identify the main constraints in operation which prevent more microenterprises from obtaining finance.
Status: Preliminary paper available

Impact of Changes in Business Skills

The Effect of Mentorship on SME Growth in Mexico
Country: Mexico
Methodology: Randomized Experiment
Researchers: Miriam Bruhn, Dean Karlan and Antoinette Schoar
This project evaluates whether providing mentorship services to micro, small and medium size enterprises can help them grow in size and improve their performance. A state government agency in Puebla, Mexico, will provide 150 randomly chosen SMEs with mentors from local consulting firms that are specialized on working with SMEs. During the first year of the program the mentors will work with the firms to draw up a business plan. During the second year of the program the business plan will be put into action. In addition to focusing on making changes in the firms per se, the program also focuses on developing the entrepreneurial skills of the firms’ principal decision maker.
Status: Ongoing

Constraints to Female Entrepreneurship: Ideas or Capital?
Country: Pakistan
Methodology: Field Experiment
Researchers: Xavier Gine, Ghazala Mansuri
This study uses a randomized experimental design to assess the impact of a program to provide business training to poor rural women to assist them in identifying business opportunities in their local environments, understanding markets for inputs and outputs, obtaining basic entrepreneurial skills, such as book keeping, and finally assisting them with their needs for financial capital through micro enterprise loans. The study is being conducted with the Pakistan Poverty Alleviation Fund (PPAF) a World Bank funded apex organization in Pakistan which supports NGOs working with the poor, and the National Rural Support Program (NRSP), the largest community based development program in rural Pakistan. The key outcome of interest is the extent to which such training and credit access improves the economic opportunities of women. To see this we will measure new business start ups and business expansions, changes in business profitability and women’s labor market activity. In addition, however, we will examine the ways in which such an intervention can impact other aspects of a woman’s wellbeing, including her ability to influence household decisions regarding the allocation of resources, investment in children, and her own mobility.
Status: Work in progress

Management Practices and Firm Performance: A Randomized Management Intervention
Country: India
Methodology: Randomized Experiment
Researchers: David McKenzie, Nick Bloom, Benn Eifert, Aprajit Mahajan
Recent surveys suggest that a large tail of SME firms in developing countries are extremely poorly managed. A pilot study will conduct a randomized management consulting intervention directed at upgrading management practices in medium-sized Indian manufacturing firms. The intervention will be a ten-week management transformation aimed at significantly upgrading operations and human resources management. The project will assess the effectiveness of such an intervention, and attempt to determine the barriers to firms purchasing such services for themselves.
Status: Pilot in preparation

Impact Evaluations to Foster Technology Adoption in Agriculture

Marketing, Credit and Technology Adoption
Countries: Kenya
Methodology: Randomized Field Experiment
Researchers: Xavier Gine, Dean Karlan and Nava Ashraf.
In much of the developing world, many farmers grow crops for local or personal consumption despite export options which appear to be more profitable. Thus many conjecture that one or several markets are missing. We report here on a randomized control trial conducted by DrumNet in Kenya that attempts to help farmers adopt and market export crops. DrumNet provides smallholder farmers with information about how to switch to export crops, makes in-kind loans for the purchase of the agricultural inputs, and provides marketing services by facilitating the transaction with exporters. The experimental evaluation design randomly assigns pre-existing farmer self-help groups to one of three groups: (1) a treatment group that receives all DrumNet services, (2) a treatment group that receives all DrumNet services except credit, or (3) a control group. After one year, DrumNet services led to an increase in production of export oriented crops and lower marketing costs; this translated into household income gains for new adopters. However, one year after the study ended, the exporter refused to continue buying the cash crops from the farmers because the conditions of the farms did not satisfy European export requirements. DrumNet collapsed as farmers were forced to sell to middlemen thus defaulting on their loans. Eventually they switched back to local crops. The risk of such events may explain, at least partly, why many seemingly more profitable export crops are not adopted.
Status: Working paper available

Insurance, Credit, and Technology Adoption
Country: Malawi
Methodology: Randomized Field Experiment
Researchers: Xavier Gine, Dean Yang
The adoption of new agricultural technologies may be discouraged because of their inherent riskiness. This study implemented a randomized field experiment to ask whether the provision of insurance against a major source of production risk induces farmers to take out loans to invest in a new crop variety. The study sample was composed of roughly 800 maize and groundnut farmers in Malawi, where by far the dominant source of production risk is the level of rainfall. We randomly selected half of the farmers to be offered credit to purchase high-yielding hybrid maize and improved groundnut seeds for planting in the November 2006 crop season. The other half of the farmers were offered a similar credit package but were also required to purchase (at actuarially fair rates) a weather insurance policy that partially or fully forgave the loan in the event of poor rainfall. Surprisingly, take up was lower by 13 percentage points among farmers offered insurance with the loan. Take-up was 33.0 percent for farmers who were offered the uninsured loan. There is suggestive evidence that the reduced take-up of the insured loan was due to the high cognitive cost of evaluating the insurance: insured loan take-up was positively correlated with farmer education levels. By contrast, the take-up of the uninsured loan was uncorrelated with farmer education.
Status: Working paper available

Impact of Formalization and Changes in the Business Environment

License to Sell: The Effect of Business Registration Reform on Entrepreneurial Activity in Mexico
Country: Mexico
Methodology: Difference-in-Difference
Researcher: Miriam Bruhn
Several cross-country studies and Doing Business reports have argued that simplifying business registration regulation can increase economic activity. This paper investigates this claim in a within-country setting. It estimates the effects of a reform in Mexico that reduced the complexity of business registration procedures. The identification strategy in this paper relies on the fact that the reform was implemented in different municipalities at different points in time. The number of registered businesses in eligible industries is shown to have increased after the reform. Employment in these industries also increased. Furthermore, the results show a decline in prices and in the income of incumbents, suggesting that the entrance of new firms led to an increase in competition. The panel structure of the labor market survey also allows the paper to determine that the increase in registered businesses came from former wage earners setting up new businesses. Informal business owners were not more likely to register their businesses after the reform.
Status: Working paper available

Does It Pay Firms to Pay Taxes? The Impact of Formality on Firm Profits
Country: Bolivia
Methodology: Instrumental Variables
Researchers: David McKenzie, Yaye Seynabou Sakho
A new survey of micro and small firms is used to identify the impact of tax registration on business profitability, using the distance of a firm from the tax office where registration occurs, conditional on the distance to the city center, as an instrument for registration. Proximity to the tax office provides firms with more information about registration, but is argued to not directly affect profits. We find tax registration leads to significantly higher profits for the firms that the instrument affects. However, we also find evidence of heterogeneous effects of tax formality on profits. Tax registration is found to increase profits for the mid-sized firms in our sample, but to lower profits for both the smaller and larger firms, in contrast to the standard view that formality increases profits. We show that owners of large firms who have managed to stay informal are of higher entrepreneurial ability than formal firm owners, in contrast to the standard view (correct among smaller firms) that informal firm owners are low ability.
Status: Working paper available

Do Reorganization Costs Matter for Efficiency? Evidence from a Bankruptcy Reform in Colombia
Country: Colombia
Methodology: Natural Experiment, Before -After
Researchers: Xavier Gine, Inessa Love
The paper studies the effect of the length of reorganization on the efficiency of bankruptcy laws. We develop a simple model that predicts that in a regime with lengthy reorganizations and thus high reorganization costs, the law fails to achieve the efficient outcome of liquidating unviable businesses and reorganizing viable ones. We test the model using the Colombian bankruptcy reform of 1999 using data from 1,924 firms filing for bankruptcy between 1996 and 2003. We find that the pre-reform reorganization proceedings were so inefficient that the bankruptcy system failed to separate economically viable firms from inefficient ones. In contrast, by streamlining the reorganization process, the reform improved the selection of viable firms into reorganization. In this sense, the new law increased the efficiency of the bankruptcy system in Colombia.
Status: Working paper available

Impact of Liquidity Booms in Emerging Markets

Dollars Dollars Everywhere, Not a Dime to Lend: Credit Limit Constraints on Financial Sector Absorptive Capacity
Country: Pakistan
Methodology: Natural Experiment
Researchers: Asim Khwaja, Atif Mian, and Bilal Zia
The events of 9/11 led to an unexpected surge of capital into Pakistan. This provides a rare opportunity to understand the micro-level causes preventing emerging economies from effectively utilizing liquidity booms. We show that despite the surge of capital, an aggregate demand boom, and sharply falling cost of capital, banks were remarkably sluggish in increasing firm credit. Consequently Pakistan became a net exporter of capital. Using quarterly loan-level data covering the entire banking sector and all borrowing firms in the economy, we show that backward looking pre-9/11 credit limit constraints imposed by banks are largely responsible for the limited absorptive capacity of the banking sector. Banks are unable to extend credit in sync with firm demand, particularly for smaller firms and those facing more stringent collateral requirements. Our evidence provides important clues for why emerging markets often find it difficult to attract and retain capital. We estimate the economy wide cost of this limited absorptive capacity to be 2.3% of GDP.
Status: Working paper available

Impact Evaluation of Micro-insurance Products

Patterns of Rainfall Insurance Participation in Rural India
Country: India
Methodology: Field Experiment
Researchers: Xavier Gine, Robert Townsend, James Vickery
Insurance markets are growing rapidly in the developing world. As part of this growth, innovative new products allow individual smallholder farmers to hedge against agricultural risks, such as drought, disease and commodity price fluctuations. These financial innovations hold significant promise for rural households. We describe the contract design and institutional features of an innovative rainfall insurance policy offered to smallholder farmers in rural India, and present preliminary evidence on the determinants of insurance participation. Insurance takeup is found to be decreasing in basis risk between insurance payouts and income fluctuations, increasing in household wealth and decreasing in the extent to which credit constraints bind. These results match with predictions of a simple neoclassical model appended with borrowing constraints. Other patterns are less consistent with the “benchmark” model; namely, participation in village networks and measures of familiarity with the insurance vendor are strongly correlated with insurance takeup decisions, and risk-averse households are found to be less, not more, likely to purchase insurance. We suggest that these results reflect household uncertainty about the product itself, given their limited experience with it.
Status: Working Paper available and work in progress

Health Insurance Take-up and Market Efficiency
Country: Philippines
Methodology: Field Experiment
Researchers: Xavier Gine, Dean Karlan and Jonathan Zinman
Health risk is an important source of income fluctuations for low-income households in developing countries. With complete and frictionless insurance markets, protecting households from health shocks would be fairly unproblematic but in developing countries, these markets are typically incomplete or altogether absent. Insurance providers face great challenges in trying to reduce the health risk faced by their clients due to moral hazard and adverse selection. Thus, one might conclude that transactions costs and information problems loom too large for insurance providers to make inroads in a cost-effective way. Yet, microfinance institutions around the world are starting to offer health insurance to their clients. We are working with the Green Bank of Caraga, a rural bank operating in the central and southern Philippines and The Philippine Health Insurance Corporation (PhilHealth) to address several research, policy, and programmatic questions related to expanding access to health insurance. Using randomized field experiments, our goal is to identify the presence or absence of adverse selection, to estimate the impact of access to insurance on household health and risk-taking, to measure potential crowd-out of informal insurance arrangements, and to evaluate alternative strategies for microfinance institutions interested in offering micro-insurance products.
Status: Work in Progress

Impact Evaluation of Experiments on Data Collection from Firms

How Accurate is Recall Data: Evidence from Coastal India?
Country: India (Tamilnadu)
Methodology: Randomized Experiment, multiple measurement
Researchers: Xavier Gine, Monica Martinez-Bravo
Lacking baseline data, researchers have to rely on recall data to assess impact and changes in behavior. We use an unusually rich dataset kept by informal lenders of daily catches, loans and changes in fishing gear for 300 boatowners in 7 villages of Tamilnadu from 2002 to 2007. Using survey data we measure the timing of purchase of fishing gear, past monthly catches and loans from the boatowner as well as their spouse. Two versions of the questionnaire were randomly assigned to respondents, to understand whether the timing of where in the survey these questions were asked affected response accuracy. Using the administrative data we can assess the direction and magnitude of the biases of the recall data.
Status: Ongoing

How Can We Collect Profit Data from Small Firms that Don’t Keep Records?
Country: Sri Lanka
Methodology: Randomized Experiment, multiple measurement
Researchers: David McKenzie, Chris Woodruff and Suresh de Mel
A myriad of potential problems plague the measurement of profits in small and informal firms. This work conducts experiments to measure the importance of these problems, and to draw recommendations on how to best collect profit data. We randomly allocated some firms ledger books, to keep diary records of firm revenue and expenses. We find short run effects on reported revenues and expenses, but not profits. The firms did not keep books long enough to see longer term effects. We also examine whether firms deliberately underreport revenues. The majority of firms think that revenues are underreported, with a median level of underreporting of 30 percent, a level confirmed by monitoring of sales.
Status: Working paper available; paper forthcoming in the Journal of Development Economics


Last updated: 2014-01-29

Library

WPS6689Unilateral facilitation does not raise international labor migration from the PhilippinesBeam, Emily; McKenzie, David; Yang, Dean2013/11
WPS6407Harnessing emotional connections to improve financial decisions : evaluating the impact of financial education in mainstream mediaBerg, Gunhild; Zia, Bilal2013/04
WPS6368Using administrative data to evaluate municipal reforms : an evaluation of the impact of Minas Facil ExpressoBruhn, Miriam; McKenzie, David2013/02
WPS6145Business training and female enterprise start-up, growth, and dynamics : experimental evidence from Sri Lankade Mel, Suresh; McKenzie, David; Woodruff, Christopher2012/07
WPS6141Soft skills or hard cash ? the impact of training and wage subsidy programs on female youth employment in JordanGroh, Matthew; Krishnan, Nandini; McKenzie, David; Vishwanath, Tara2012/07
More...



Permanent URL for this page: http://go.worldbank.org/6F5AD32N10