Study uses the gold standard, standardized patients, to measure quality of medical care.
Overall quality was low, and training, equipment and patient load didn’t have large effects on quality.
Results show the urgent need to systematically measure the quality of care.
WASHINGTON, Dec. 3, 2012 – When we talk about strengthening health care systems in developing countries, we often mean building hospitals, hiring more staff or stocking up on medications. But one topic has been noticeably missing: the quality of doctors and nurses.
That may seem baffling, given that the quality of medical professionals – the very people taking care of the sick – can mean life or death for patients, and that their salaries take up the bulk of a health system’s budget. And surely, governments, donors, development institutions and non-governmental organizations want the best results for their investment.
The problem arises in part because the quality of care is a hard thing to measure. Every patient is different, and even with the same broad category of diagnosis, they may warrant different treatment, which makes it difficult to compare health outcomes. In addition, patients may not be able to follow an optimal treatment plan, especially when it’s expensive.
In a new study led by World Bank economist Jishnu Das, those issues are resolved by using “standardized patients,” or local people who are recruited to memorize and present consistent symptoms of an illness during unannounced visits to multiple health care providers. The methodology is considered the gold standard in quality measurement.
The study, which is published today in the journal Health Affairs, recruited 22 such patients in India, trained them for at least 150 hours and had them make 926 visits to 305 providers in India. They presented symptoms for unstable angina (chest pain), asthma, and dysentery (intestinal inflammation) – which all come with well-known, textbook treatment plans. That makes it easier to compare treatment – and providers – across the board.
What they find is surprising. First, most providers – for example, 67% in rural Madhya Pradesh, one of India’s poorest states – don’t have medical degrees. Second, the overall quality is low. Each visit in the state averaged 3.6 minutes, 33% of the providers gave a diagnosis at all and just 12% of those were correct. Only 30.4% of the visits led to correct treatment, but 41.7% of the visits led to unnecessary or harmful treatment. The numbers are a little better in Delhi, one of India’s wealthiest states, with visits averaging 5.3 minutes and 48 percent of providers giving the correct treatment.
In addition, the study shows there aren’t big differences between trained and untrained doctors in areas such as sticking to clinical checklists, signaling the need for better medical education. Perhaps as a result of that, public clinics didn’t fare better in the study. In fact, when it comes to following medical guidelines, public providers were worse than their untrained, private counterparts.
The study could help policy makers make evidence-based decisions. In November, the government announced a five-year plan to triple health spending and improve the quality of health services. In 2010, India spent 4% of gross domestic product on health expenditures, compared with an average of 10% in developed countries. Out-of-pocket expenditures account for 70 percent of the nation’s overall health care costs.
“This study points to the need to think beyond the health infrastructure and medical qualifications, neither of which is closely correlated with the quality of care," says Adam Wagstaff, a health economist and research manager of the human development and public services team in the Bank’s Development Research Group. "The study also casts doubt on some commonly-held views about the quality differences between the public and private sectors. I hope that the study will stimulate a large and systematic effort to collect data on and monitor quality of care so that health systems can be held accountable for delivering value for money.”
The study is the latest product from a 10-year effort by Das and colleagues to look at how to best measure the quality of health care in several low-income countries. One paper uses hypothetical cases to evaluate the competence of doctors, showing that private providers in richer areas of Delhi are significantly better than those in poorer neighborhoods. Another paper shows that providers in private practice prescribe a lot of drugs that aren’t necessary. In the public sector, education subsidies and salaries don’t translate into better care at all. (Indians mostly prefer private hospitals over public facilities, because of understaffing and other concerns.)
The new research, funded by the Bill and Melinda Gates Foundation, was conducted jointly with the Center for Policy Research in Delhi, with collaborators from the World Bank, Johns Hopkins University, Duke University, the University of Toronto and Massachusetts General Hospital. “This study, conducted by our multi-disciplinary team, is unique in scale and scope, and the population-based sample of providers is representative of primary care facilities in rural Madhya Pradesh,” says Alaka Holla, a co-author and economist in the World Bank’s Human Development Network.
The research team hopes the vigorous methodology will help jumpstart a conversation on how to encourage doctors to provide better care. “Instead of assuming quality is higher in facilities with more equipment or better-trained doctors, countries should better integrate quality measures into existing health policy,” says Das, a senior economist in the Development Research Group. “And the quality of care can only improve after it is measured systematically and in multiple settings.”