Highlights | Notes | 2010 Publications
The Director’s Office conducted research on poverty and inequality, focusing on monitoring the developing world’s progress against poverty, understanding why some countries are more successful than others in reducing poverty, and evaluating specific anti-poverty policies.
Your new composite index has arrived—“Please Handle with Care”
A host of indicators are used to track development. The World Bank’s annual “World Development Indicators” presents hundreds of such indicators. The United Nation’s “Millennium Development Goals” are defined using a long list of indicators. Faced with so many indicators there is an understandable desire to form a single composite index. However, it is not always clear how this can be done. Here researchers can help, in both constructing better indices and in flagging cautions for users.
For some of the composite indices found in practice, economic theory provides useful clues as to how the index should be constructed. This is not the case for another type of composite index now becoming popular. For these indicators neither the menu of the primary series to be aggregated nor the aggregation function is pre-determined from theory and practice, but are “moving parts” of the index—key decision variables that the analyst is essentially free to choose, largely unconstrained by economic or other theories intended to inform measurement practice.
Probably the most famous example of such “mashup indices” is the “Human Development Index” (HDI) published each year in the United Nations Development Program’s Human Development Report. The HDI is a composite of life expectancy, schooling (literacy and enrollment rates), and log GDP per capita at purchasing power parity. It adds up attainments in these three dimensions, with equal weight (one-third each). Similar in spirit to the HDI is the “Multidimensional Poverty Index” (MPI) developed for the 2010 edition of the Human Development Report. The World Bank has contributed its own mashup indices, notably the “Worldwide Governance Indicators” and the “Ease of Doing Business Index.”
While the country rankings based on these and other mashup indices often attract media attention, the details of how the composite index was formed—the variables and weights—rarely get critical scrutiny. The current enthusiasm for more mashup indices needs to be balanced by clearer warnings for, and more critical scrutiny from, users. Any new mashup index should be able to answer the following four questions: What is it measuring? What tradeoffs are embedded in the index? How robust are the rankings” And: How should the index be used by policy makers?
Greater clarity is needed about what exactly is being measured
And more attention needs to be given to the tradeoffs embodied in the index—how much of one component of the index is being given up for more of another component. In most cases the tradeoffs are not identified in the most relevant space for users to judge, and in cases where it can be derived from the data available it has been found to be questionable—implying, for example, unacceptably low valuations of life in poor countries. Indeed, the value of life implicit in the HDI ranges from a remarkably low level in the poorest countries to a very high level in rich countries. Thus the HDI embodies some ethically troubling tradeoffs most users are unaware of.
While mashup indices have often contributed to public awareness about important development issues, their producers may need to be more humble. Their rhetoric is often in marked tension with reality. One is struck, for example, that the “multidimensional poverty indices” proposed to date actually embrace far fewer dimensions of welfare than commonly used measures based on household consumption.
One area for greater caution is the robustness of country rankings
The uncertainty about the components and their weights is not adequately acknowledged by mashup producers, and users are given little guidance to the robustness of the resulting country rankings. Today’s technologies permit greater openness about the sensitivity of country rankings to choices made about a mashup index’s (many) moving parts.
Policy relevance is another area of concern
It is unclear what can be concluded about “country performance” toward agreed development goals in the absence of an allowance for the (country-specific) contextual factors that constrain that performance. There are also potentially important “targeting applications,” though here policy
makers might be better advised to use the component measures appropriate to each policy instrument rather than the mashup index.
Theory needs to catch up
Arguably, mashup indices exist because theory and rigorous empirics have not given enough attention to the full range of measurement problems faced in assessing development outcomes. The lessons for measurement from prevailing economic theories only take us so far in addressing the data concerns of practitioners, including policy makers. Theory needs to catch up. It is also important to recognize that the theoretical perspectives relevant to measurement practice are not exclusive to economics— theory has to include the political, social and psychological sciences as well.
1. World Bank. 2009. World Development Indicators. Washington, DC: World Bank.
Available at http://data.worldbank.org/data-catalog/world-development-indicators.
2. United Nations. 2011. Millennium Development Goals.
Available at http://www.un.org/millenniumgoals/.
3. United Nations Development Programme. 2010. Human Development Report: The Real Wealth of Nations: Pathways to Human Development. New York: Palgrave Macmillan.
Available at http://hdr.undp.org/en/reports/global/hdr2010/.
For a definition and background information on the human development index see http://en.wikipedia.org/wiki/Human_Development_Index.
4. Multidimensional Poverty Index.
More available at http://www.ophi.org.uk/policy/multidimensional-poverty-index/.
5. World Bank. Various years. Worldwide Governance Indicators.
Available at www.govindicators.org.
6. World Bank. Various year. Doing Business: Measuring Business Regulations. Washington, DC: World Bank.
Rankings available at http://www.doingbusiness.org/rankings.
7. Ravallion, Martin. 2010. “Mashup Indices of Development.” Policy Research Working Paper 5432, World Bank, Washington, DC.
8. Ravallion, Martin. 2010. “Troubling Tradeoffs in the Human Development Index.” Policy Research Working Paper 5484, World Bank, Washington, DC.
9. Ravallion, Martin. 2011. “On Multidimensional Indices of Poverty.” Policy Research Working Paper 5580, World Bank, Washington, DC.