Researchers at Northwestern University’s Global Poverty Research Lab are leveraging the Philippines Socioeconomic Panel Survey to measure the impact of skills-based referrals to recruit enumerators on data quality. 

Credible research relies significantly on the quality of data enumerators collect from households. However, evidence suggests that this data can vary based on the enumerator’s demographic characteristics, and their skills and behaviors when conducting interviews. Poor data quality can lead to bias and limited validity across contexts. The Philippines Socioeconomic Panel Survey (PSPS) will provide government actors and the Philippine public with important, publicly available data on the socio-economic development of over 13,500 households in Western Visayas over the next twenty years. Increased data quality can improve the causal inferences that can be drawn from the PSPS data and inform better policy. 

Researchers at the Global Poverty Research Lab at Northwestern University and IPA are conducting a randomized evaluation to measure the impact of recruiting enumerators through skills-based referrals on data quality. They will evaluate whether skills change the skill profile in the enumerator applicant pool and what is the return of specific skills (active listening and conscientiousness) on data quality. Active listening is hearing and responding to verbal speech and non-verbal cues, like body language and facial expressions. Conscientiousness is the ability to be hard-working, organized, proactive, and follow through with their commitments. 

Enumerator applicants will either be recruited with referrals for being active listeners, with referrals for being conscientious, or without referrals and serve as the comparison group. The intervention will be implemented in three phases: 

• Phase 1 (Business as Usual No Referrals): Recruitment will be conducted and advertised as usual, with an open application, without skills-based recruitment.
• Phase 2 (Conscientious or Active Listening Referral): Trusted enumerators who have worked with IPA Philippines previously will be randomly assigned and invited to refer candidates who they think possess the skill of active listening or conscientiousness. Applications by “business as usual” applicants who apply without referrals will continue to be accepted and reviewed.
• Phase 3 (“Friends of Friends” Recruitment): Applicants who applied without referrals in the “business as usual” phase, passed a screening quiz, and scored highly on an interviewer skills survey, will be invited to refer candidates based on the two skills.

The recruitment team will be blind to the referral status of a candidate (that is, if they were referred or not) to ensure that the outcome of their application remains unbiased. All referrals will go through the standard application, interviewing, and screening process. Being referred to the role will have no bearing on the outcome of a candidate’s application. All applicants have the opportunity to participate in this study and take a self-administered survey during their application, which researchers will use to measure their active listening and conscientiousness-related skills. To measure outcomes related to data quality, each enumerator’s survey-collected data will be linked to back-check data for the same household.

Results of this evaluation will be available in 2024.

Partner