How Smartphones Could Keep Psychology From Getting Too WEIRD

In 2004, I was a lab rat for about 15 minutes. A psychology professor at Juniata College, where I spent my freshman year, was conducting an experiment. I don’t remember what exactly he was studying, but it involved video games. He put up posters around campus and gathered a bunch of volunteers in a campus building basement to frag each other in several rounds of Unreal Tournament. I lost pretty quickly, but did my part. I think I got a game store gift card for my time.

Now ideally, if you want to learn anything useful about human brains and behavior, you try to get a large and diverse group of people to draw your conclusions from. But as Canadian psychologist Joseph Henrich and colleagues revealed in a 2010 paper in Behavioral Brain Sciences, a lot of psych studies are done the same way as the one I participated in.

That is, they test ideas by looking at small and homogeneous groups of volunteers brought to college campuses and research facilities, usually drawing those volunteers from the school’s student body or the local population. (The rest of the guys in my study were, like me, all white male undergrads who liked playing first person shooters.)

WEIRD Science

Henrich’s team looked at hundreds of studies in leading psychology journals, and found that 68 percent of the research subjects came from the United States, and 67 percent of those were undergraduate psychology students. Overall, 96 percent of the subjects came from Western industrialized countries that, together, make up only 12 percent of the world's population. Frequently, studies that claim to reveal something universal about the human brain or our behavior are really just extrapolating results from the same (relatively) small groups.

This kind of study-building method results in the overrepresentation of a population that the authors dub WEIRD: Western, Educated, Industrialized, Rich, and Democratic. Sure, we’re all human. We’re all working with more or less the same software in our skulls. But, the researchers say, culture and environment play a role in shaping how we use that software. There are important differences in the way my brain works versus, say, a rural farmer in China, versus a member of a hunter-gatherer tribe on an island in the South Pacific, when it comes to areas like “visual perception, fairness, cooperation, spatial reasoning, categorization and inferential induction, moral reasoning, reasoning styles, self-concepts and related motivations, and the heritability of IQ.”

“The findings suggest that members of WEIRD societies, including young children, are among the least representative populations one could find for generalizing about humans,” the paper continues. We, the WEIRD ones, are actually  “highly unrepresentative of the species,” but form the basis for so much of what we think we know about ourselves.

Henrich and his colleagues call for their fellow scientists to collect comparative data across culturally and geographically diverse populations before drawing conclusions about our species as a whole. But how do you do that? With shrinking funding and small staffs, it’s not always feasible, to conduct a study in your own lab and then go elsewhere to get a different sample, or even to try to attract a diverse sample to you. Researchers have tried to get volunteers from the far reaches of the globe to participate in web-based studies, but found that mice and keyboards and web page interfaces couldn’t provide the precision necessary for understanding the subtle details and changes of cognitive processes and behavioral responses.

Pick up the Phone

But now there’s a new way to bring non-WEIRD volunteers right to the researchers. The number of smartphone users worldwide is expected to top one billion by next year. The technology has found a home in almost every social group in every part of the world, Western and Eastern, educated and not, industrialized and agrarian, rich and poor, democratic, autocratic and theocratic. Not only are they everywhere, but they’re well suited to collecting scientific data. They can transmit and receive multiple types of media and commands, can transfer time- and location-coded data, and can time, down to the millisecond, stimuli display and touchscreen responses. They are, an international team of scientists suggested last year, ideally adapted to studying cognitive function and could be used as a “multi-dimensional scientific ‘instrument’ capable of experimentation on a previously unthought-of scale” that could reveal things about the human mind long hidden by smaller experiments.

Researchers could take advantage of smartphones to revolutionize research in cognitive science, the paper argues, but the studies and the technology have to come together in a way that makes it work. To see if smartphones could live up to their promise in a real-world study, Stephane Dufau, the lead author, and her team took their idea for a road-test, without ever leaving the lab.

An App for That

The results collected so far are similar to those obtained by running the test in laboratory conditions and match many of the known features of this type of data, indicating that an app-based study like this doesn’t introduce variables that affect the results.

Another team of American researchers launched a similar app-based study to look at age-related differences in cognition. They got 15,000 people to participate and their results replicated specific patterns and data found in lab experiments. This study did reveal some problems with the app-based data collection, though. One hindrance the researchers noted is the lack of ability they had to monitor the participants. Their app instructions recommended that users complete their tasks without distractions, but there’s no way they could tell if someone used the app while multitasking or in a noisy environment, which might affect their performance.

Since there’s no obligation or accountability for completing the tasks, there was also a higher participant dropout rate than in many lab studies. Still, the researchers say that the larger sample size that the app gave them access to compensated for the loss in data amount and quality.

These two studies suggest smartphones are a reliable way to collect culturally and geographically diverse data on an enormous scale. The smartphone, far from being just a gadget that lets you tweet from the bathroom, could be as important to scientific exploration as the microscope or the lunar lander. They could potentially allow for direct tests of the universality of cognitive theories and make our understanding of ourselves a little less WEIRD.

The Science XL study is ongoing, so if you want to take part, the app is free to download from iTunes AppStore.