September 16, 2008, Stuti Khemani
The push toward community monitoring of public service delivery in development projects is based on the notion that local oversight will raise the social accountability of public service providers and thereby improve the quality of services. Diverging results from evaluations of two community monitoring programs in health and education suggest that local monitoring does not guarantee better service delivery.
Two evaluations of ambitious efforts to improve public services show strikingly different impacts. Both projects attempted to mobilize communities to achieve this. One project on health in Uganda succeeded, while another on education in India had no impact on public schools even as a private initiative outside the public school system was able to improve the reading skills of students.
The push toward community monitoring of public services in development projects is based on the notion that local oversight raises the social accountability of public service providers. The diverging results from the two evaluations suggest that local monitoring does not guarantee better services.
In Uganda community monitoring led to improved quality of health delivery
In 25 randomly selected communities local NGOs organized meetings of residents and health service providers about the quality of care in public clinics. Quality of care was measured previously through user surveys about the quality of service received at the clinics and was compiled in “citizen report cards.” In another 25 randomly selected “control” communities, no such discussion took place.
In the communities where the meetings were held, absenteeism by providers decreased and the quality of service (measured by wait time and quality of care) improved. Ultimately, immunization rates rose and child mortality rates dropped in these communities, suggesting that mobilizing the community to monitor providers more actively can improve services. In Uganda, this oversight could be done through the beneficiary-control institutions called Health Users Management Committees (HUMCs).1
In India community monitoring did not lead to improved quality in public education
A leading education-focused local NGO mobilized communities about learning in schools in 195 randomly selected villages in the state of Uttar Pradesh. It facilitated information sharing about the quality of schools and how to improve them through agencies like the Village Education Committee (VEC). The VECs operate much like the HUMCs for health services in Uganda.
Reading test volunteers in Uttar Pradesh. Photo by Shekhar Shah.
Village volunteers prepared “report cards” on the reading ability of children; these were shared with teachers, local government representatives, and residents in village-wide meetings. An evaluation found no difference in community participation, teacher effort, or learning outcomes in public schools between the villages where the meetings took place and 85 randomly selected “control” villages where no meetings were held.However, in 65 villages (among the 195) where a local NGO held additional classes to improve reading skills outside the public school system, reading scores increased. Children who could only decipher alphabet letters at baseline attended the reading classes and were more than twice as likely to read words and stories when surveyed one year later (see fig.). A large number of local youths volunteered for training in the use of the new reading tool, and they held more than 400 reading classes across 55 villages that involved almost 7,500 children. 2,3
A large number of factors may affect the performance of community monitoring
The success of this private initiative outside the public school system compared with the failure to improve the public schools through the village meetings suggests that collective public action does not guarantee better public services. VECs in India and HUMCs in Uganda were largely inactive at baseline, and both had only indirect influence, mainly in the form of complaints from parents and users about non-performing teachers or health workers. Action against them would have to be taken by a senior bureaucrat.
So what accounts for the different outcomes in the two countries?
Variation in NGO activism in the interventions. In Uganda, the local NGOs seemed more active in pressuring the health care providers to improve performance than the education NGO in India. The facilitators in Uganda directly negotiated with the dispensary staff before involving the villagers, and the villagers who became involved were hand-picked by them. Because the community served by each health clinic is almost ten times larger than a village in India, the activist role for external facilitators may have been more scalable. In India the focus was on community-led engagement.
Differences between health and education services. Poor health services are more directly observed by users than poor teaching which can remain invisible to parents. As a result, the users of health clinics could be more easily spurred to demand better services they did not know they were entitled to. In contrast, young pupils need advocates. Monitoring the quality of teaching requires parental visits to the classroom to observe teacher effort and performance, and one parent’s complaint may go unheeded unless it sparks complaints from more parents.
Differences in country political economy. Teachers are often organized, politically powerful, and therefore resistant to social or bureaucratic reproach. Larger political obstacles can constrain local collective action. Mobilizing citizens in the Indian political economy context may be more successful by providing information for comparisons and benchmarking purposes across jurisdictions, thereby putting public pressure on politicians to improve the quality of services.4
The contrasting results in Uganda and India underscore the importance of continuing to experiment with information interventions in different socio-political contexts.
1. Martina Bjorkman and Jakob Svensson. Forthcoming. Power to the People: Evidence from a Randomized Field Experiment of Community-Based Monitoring in Uganda. Quarterly Journal of Economics.
2. Abhijit V. Banerjee, Rukmini Banerji, Esther Duflo, Rachel Glennerster, and Stuti Khemani. 2008.Pitfalls of participatory programs: evidence from a randomized evaluation in education in India. World Bank Policy Research Working Paper 4584, April.
3. Abhijit V. Banerjee, Rukmini Banerji, Esther Duflo, Rachel Glennerster, and Stuti Khemani. 2006.Can information campaigns spark local participation and improve outcomes ? A study of primary education in Uttar Pradesh, India. World Bank Policy Research Working Paper 3967, July.
4. Stuti Khemani. 2007. Can Information Campaigns Overcome Political Obstacles to Serving the Poor? In The Politics of Service Delivery in Democracies: Better Access for the Poor, ed. Shantayanan Devarajan and Ingrid Widlund. Expert Group on Development Issues, Ministry for Foreign Affairs, Sweden. (Download the report)
STUTI KHEMANI is a Senior Economist in the Development Research Group (Human Development and Public Services Team). Her research interests include the political economy of public policy choices, and institutional reforms for development. She is currently working on how institutions of decentralization and local monitoring may strengthen political incentives for quality services.