Abstract

Understanding the level of vaccine acceptance is crucial for the design and implementation of public health campaigns to achieve mass vaccination against COVID-19. Phone surveys have been the most frequent way to collect this information, yet they pose measurement challenges that could bias the results. Researchers are analyzing data from two large-scale surveys, one from the World Bank and one from IPA, to determine how sampling issues, especially selection and attrition biases, influence vaccination acceptance measurement in low- and middle-income countries (LMIC).

Policy Issue

During the COVID-19 pandemic, decision-makers across governments, international organizations, and the health industry needed fast and reliable data to inform their decisions. Due to restrictions on travel and interpersonal contact, most efforts to gather information rely on remote methods, which often required recruiting respondents remotely and conducting phone surveys. Once vaccinations became available, a particularly important need was to provide data on the demand for COVID-19 vaccination to national health ministries, the World Health Organization (WHO), pharmaceutical companies and vaccine administrators. 

Understanding a community or country’s overall vaccine acceptance rate is crucial for planning and implementing mass campaigns worldwide. But even in the best of circumstances, these values can be hard to measure. COVID-19 cases, hospitalization and mortality rates, as well as vaccine development, approval and side effects reporting, can lead to sudden changes in vaccine acceptance. Similarly, the acceptance of vaccines can also be influenced by respondents' perception of local and personal vaccine access. Given that remote phone surveys have clear operational advantages over face-to-face surveys in terms of cost, logistics, and deployment time, the question arises whether they can be used to reliably capture public opinion around vaccines. 

Context of the Evaluation

During the pandemic, massive efforts were launched to provide government and multinational organizations with timely data and evidence to influence decision-making. IPA for example launched a Research for Effective COVID-19 Responses (RECOVR) survey in Burkina Faso, Colombia, Mexico, Ghana, Philippines, Rwanda, Sierra Leone, Côte d’Ivoire, and Zambia. The objective of these phone surveys, conducted over a three round period, was to directly inform key government partners on the health, economic, and social ramifications of the pandemic. Similarly, the World Bank and partners conducted high-frequency panel phone surveys in Burkina Faso, Ethiopia, Kenya, and Nigeria. to track responses to and socio-economic impacts of COVID-19. Researchers leverage data from these two projects to identify to what extent the use of remote surveys correctly captures the levels of vaccine acceptance. 

Details of the Intervention

Researchers are compiling and analyzing data from a wide variety of phone surveys, currently including five from IPA’s RECOVR project and four from the World Bank high-frequency panel surveys, to determine how sampling issues, especially selection and attrition biases, influence vaccination acceptance measurement in low- and middle-income countries (LMIC).

Conducting surveys remotely entails some methodological challenges. For example, the selected respondents may not be representative of the population of interest, which could bias the results. This is known as sampling bias. Another common challenge is  attrition bias, in which certain types of respondents drop out of subsequent rounds more rapidly than others. Through this analysis, researchers intend to answer whether these factors influence the measurement of vaccine acceptance and whether remote surveys are an appropriate method for measuring vaccine acceptance moving forward.

Results and Policy Lessons

Research ongoing; results forthcoming.