How Statistics Fool Juries

Peter Donnelly is a statistician with a sense of humor. He starts his talk with a classic statistician joke: "How do you tell an introverted statistician from an extroverted statistician? The extrovert is the one who looks at the other person's shoes." But he's not all fun and games. In a 2005 TED Talk, Donnelly explains a little about with it's like to be a professional statistician, then launches into a fascinating explanation of how statistics are misunderstood by typical audiences. He gives examples for the audience to examine (a few multiple-choice questions), to prove his point -- and I'll admit, he got me. I didn't get the examples right, though I was pretty confident in my answers.

This reminds me of when I was on a jury room in a personal injury case. While my experience wasn't related to statistics, it was an issue of science which seemed like we should have been able to prove the right answer one way or another -- but we failed. The jury had an hour-long argument about physics, trying to determine whether a driver's upper body in a car would be pushed forward or backward when the car was hit from behind. (There was a key question regarding whether a specific injury could have been caused by the impact, or was a pre-existing condition.) We even built a model, but that failed to convince anybody of what the real-world behavior would be. Everyone on the jury insisted that his or her own mental model was correct (which tended to align with their gut feeling about the guilt or innocence of the defendant), and neither demonstrating the physics of the situation, nor thinking through it with a shared mental model made any difference. It was an interesting day, to say the least. (See also: whiplash.)

Anyway, Donnelly's talk is a great example of how attorneys (or really anyone) can exploit general misunderstanding of statistics in order to make an invalid point -- and most of us won't notice that anything is wrong. There's even a term for one specific statistical misuse, the prosecutor's fallacy, and Donnelly explains how it was exploited in the Sally Clark case. Definitely worth a look!