Writing in Pacific Standard he discusses a paper that appeared in a shortened version in Nature and a longer version in Brain and Behavioral Sciences, pointing out many of social scientists’ assumptions about human nature come from experiments drawing from on a relatively small subset of the world’s population:

 

 

This bias in sampling is the infamous “college student problem,” and if you ever had to participate in an experiment for credit in a college class, you’re familiar with it (of course, you might have been allowed to opt out and write a paper instead, but do you know anybody who actually did that?) College students are plentiful and convenient to where the researchers usually are, can be encouraged to volunteer, and as the punchline to the old joke goes, “there are some things a rat just won’t do.”

 

When Joe Henrich, who trained as an anthropologist but now has a job split between an economics and a psychology department, started taking classic economics experiments on the road, he found radically different results in small-scale societies around the world, calling the assumptions of universality into question. Among the indigenous Machiguenga in Peru, he tried the classic ultimatum game, in which two people work to split a sum of money. (One person gets to propose how the money should be divided and the other can only accept or reject the offer. If the second rejects, neither get any money.) Henrich found that the Machiguenga often agreed to splits that American players would reject as “unfair”. In other words, they played the game more “rationally” than American college students, as accepting any deal increases the well-being of both.

 

But this failure didn’t apply to only economic games, when Heinrich started cataloging the findings which failed to replicate around the world with experimental psychologists, Steven Heine and Ara Norenzayan they concluded, writing in Nature:

 

 

So how do you solve this problem of theories developed by WEIRD professors based on data from WEIRD participants failing to generalize outside the Western world? If you’re IPA, you take it to the streets. Streets in urban slums, rural villages and everywhere in between to see how the actual people affected by policies will behave when they’re implemented. We often find unexpected results, programs that sounded good but didn’t work, or that not everybody in our study responded identically. What we’ve learned is that you can’t know if what sounds good in a government or NGO office will work well without a real evaluation and actual people from the communities involved participating.

 

Within each study, we rely on our local staff to make sure we’re asking the right questions and pilot relentlessly. Our Uganda office spent a full year piloting the super savers program before the main study began.

 

IPA’s also trying to help other researchers get over their WEIRD issues. Most behavioral researchers can’t find another society, hop on a plane, run some studies there, and be back in time for their 3PM lecture, but if we’re going to create policies that work, it’s important to make sure that economic principles take into account real people. To that end, we’re supporting the Busara Center for Behavioral Economics to help researchers run studies in Nairobi, Kenya. The center has a lab adapted for the needs of behavioral researchers and the local population, which include touchscreen computers designed for non-literate participants to use, and a pool of 1,000 people signed up to participate who are paid for their time by MPesa mobile money transfers (more here). 

 

We like to think it’s important for economic principles to actually work, otherwise, you just end up with WEIRD policies.