A new study involving LSTM’s Department of International Public Health is shining a light on the problem of inaccurate health service monitoring and suggesting ways in which the issue might be better tackled.
The United Nations has continually highlighted the need for accurate statistics to assess and monitor the quality of health provision. Currently a range of sources are used, including information gathered by health workers during service delivery. However, there are recognised issues with this administrative information, including inherent biases, and no measure of its accuracy. Without accurate data, health policies and planned interventions can actually have detrimental impacts on communities, and efforts to achieve the Sustainable Development Goals, for example, are hampered.
In an earlier paper the study team, which includes LSTM’s Professor Joseph Valadez and colleagues from Harvard, had already demonstrated that decisions based on administrative data can be improved if combined with high-quality, randomly sampled survey data obtained over the same region. Following invitations to present additional results from World Bank, USAID and Countdown 2030, and having been asked about the new tool’s potential, the team set about showing how to extend these methods to situations where the randomly sampled survey data are only available for some, but possibly not all, proximal districts.
This new paper presents the novel approach by extending vaccine prevalence estimates from areas in Benin with both administrative and survey data to those with only administrative data. The study uses figures from a 2015 probability survey (n=1786) conducted in 19 of Benin’s 77 communes, together with administrative data collected by health workers from all of the districts during a national immunization day (n=2,792,803). By adapting their estimation technique the team extended its earlier technique combined-data estimation from 19 to 77 communes by estimating denominators using the survey data and then building a statistical model using population estimates from different sources to estimate denominators in adjacent districts. By dividing administrative numerators (tallies of children vaccinated on the day) by the model-estimated denominators they obtained extrapolated hybrid prevalence estimates. When the problem was framed using the Bayesian paradigm, all estimated prevalence rates fell within the appropriate ranges, in contrast to previous methods, and conveniently incorporated a sensitivity analysis.
LSTM lead Joseph Valadez said:
“By developing this approach we can better inform health policy-making and intervention planning, help to reduce waste and improve health in communities, and in doing so contribute to a priority arising from the Sixtieth World Health Assembly which underscored the importance of acquiring robust information to strengthen health systems and policies.”
The authors hope that this paper will change the policy dialogue for public health metrics. Their approach to assess health systems nationally has the potential to stimulate methodological and policy debates to affect funding choices of donors, the policy decisions of governments, and health programming for the near future.
Ocampo A, Valadez JJ, Hedt-Gauthier B, Pagano M (2022) How to estimate health service coverage in 58 districts of Benin with no survey data: Using hybrid estimation to fill the gaps. PLOS Glob Public Health 2(5): e0000178. https://doi.org/10.1371/journal.pgph.0000178