Are People or Computers Better at Asking Difficult Questions about Intimate Partner Violence?
An estimated 835 million people worldwide experience intimate partner violence (IPV) in their relationships. However, the true extent of IPV prevalence remains debated because—although IPV statistics are staggeringly high in some regions—many experts theorize the numbers are still underestimates. When surveyors ask questions about embarrassing, sensitive, or morally charged subjects, sometimes people are hesitant to be honest. IPV questions are fundamentally about violence, sex, and degradation, subjects many people are reluctant to speak about candidly. For IPV, this hesitancy may be further compounded by cultural norms around “family privacy” and respondents’ legitimate fear of retaliation if their abusive partner discovers they were “gossiping” about their relationship with a researcher. However, despite the difficultly of asking these questions, accurate information is needed by governments and organizations to understand the scope of the problem and to design better programs to address IPV. The IPV Initiative at Innovations for Poverty Action is committed to improving measurement practices, both to get more accurate data on a pervasive problem and to provide greater safety and comfort for respondents.
In recent years Audio Computer-Assisted Self-Interview (ACASI) has emerged as a new method of conducting interviews. This method involves having respondents listen to questions on headphones on a tablet and entering their responses themselves, privately. It requires less surveyor training and could lead to an overall reduction in the “surveyor effect,” in which the surveyor influences the respondents’ answers. Some theorize this method could reduce the discomfort of the survey because the respondent never feels exposed to an outsider. There have been some early adopters of this method and guides for projects to use.
So what evidence do we have on the “best” ways to ask a woman about violence from her partner?
In Kenya, Oxford University researchers Mahreen Mahmud, Kate Orkin, and Emma Riley were tracking the effects of cash transfers and role model programs on women and their households. The study was not initially intended to be an experiment comparing ACASI to face-to-face interviews, but the team ended up adding questions to another study’s survey, which did not have the time to logistically manage surveying with ACASI. They as a result have data from two different methods in a comparable sample. They were able to compare ACASI to face-to-face interviews with 2,133 women using the World Health Organization (WHO)’s Violence Against Women survey. The incidence of moderate violence is 10 to 34 percentage points higher when reported using ACASI, across the different samples and questions. All told, the study finds a nearly 30 percentage point higher incidence of physical violence reports when asked using ACASI.
In the Dominican Republic, Seth Garz was investigating the impacts of Progresando con Solidaridad, a large government cash transfer and associated social intermediation services, on intimate partner violence among social protection program participants. They also became interested in how to best measure violence in household relationships. The IPV Initiative supported them to do a similar comparison as the team in Kenya, with the exact same questions, using a sample size of 2,392 households. In all questions, audio measures had substantially higher reported prevalence than the face-to-face interview.
Similarly, Claire Cullen ran a measurement-focused comparison asking the same violence questions using different survey approaches. In Rwanda, a sample of 2,700 women was asked two questions about violence. When they were asked using ACASI, results were also higher than face-to-face but the difference was not always statistically significant.
The IPV Initiative is supporting three additional studies that test the value of increased privacy during interviews. These studies should have results public within the next two years.
Our (current) conclusions
While the IPV Initiative is waiting for results from additional studies that test the differences in ways of asking the standard IPV questions, audio computer-assisted self-interviewing (ACASI) appears to get higher reported IPV prevalence rates. Based on qualitative information and feedback from research teams, we interpret the higher reported prevalence from ACASI as representing more honest answers about violence experiences than traditional face-to-face interviews. The implications of this research could be that the alarmingly high IPV rates that are cited most frequently may actually be under-reported since they were based on traditional face-to-face interviews.
Based on the findings that have emerged to date, the IPV Initiative suggests that researchers use ACASI or other survey tools that allow for complete respondent privacy when answering questions instead of face-to-face interviews. These findings also suggest that when surveyors ask the questions, respondents are less comfortable disclosing IPV.
At this point, we are unsure why people report higher rates of IPV when answering on a private device, as opposed to answering questions from a surveyor. Further research could help shed light on whether the difference is driven by respondents knowing there is no one is secretly “listening in” on the interview if it is easier to disclose information without the labor of managing emotions in front of a surveyor if they fear judgment from the surveyor or some combination of the above.
Even if the field moves more toward computer-assisted methods, it’s important to note that surveyors still need to be trained in the ethical practices around collecting data on IPV. Surveyors will still be responsible for administering the survey instructions, ensuring respondent clarity, and supporting respondents who choose to disclose extreme or dangerous situations. However, these findings suggest studies will get more accurate answers from respondents on these difficult topics by using a method that offers more privacy.