Misbehaving – Mad at Nate Silver About Election 2016? Why Behavioral Economics Suggests You Shouldn’t Be

For many, the aftermath of Donald Trump’s election has raised questions about the future of the United States. It has also led some to cast aspersions on analysts like Nate Silver, who used statistical models and polling data to make probabilistic predictions about the election’s outcome. Because these models consistently favored Hillary Clinton, and often heavily so (Nate Silver predicted that Clinton had a 71.4% chance of winning, and other estimates were as high as 99%), some have accused these poll analysts of ignorance or bias. Research suggests, however, that it may be the critics, and not the statheads, who are suffering from a bias -- more specifically, “outcome bias,” whereby we have a tendency to overweight the outcome of a decision (or model) when assessing its ex-ante quality.

In a 2014 paper, Sticking with What (Barely) Worked, researchers at Brigham Young University examined how outcome bias affects decision making. Specifically, they explored the way that coaches in the National Basketball Association (NBA) revise their starting lineup following a game. But before we dive into the research, let’s take a step back and look at a different sport: football. Perhaps a familiar example from the US’s biggest sporting event -- the Super Bowl -- can best illustrate outcome bias.

It has been called the “worst play call in NFL history.” Quick recap. It’s February 1, 2015. The New England Patriots lead the Seattle Seahawks by 4 points, with 20 seconds on the clock. But the Seahawks are driving, and are a mere 1 yard -- just 36 inches -- from scoring a game-winning touchdown. Everyone thinks the ball will be put in the hands of the team’s star running back, Marshawn Lynch. But rather than call a run play, the Seahawks decide to pass and the ball is intercepted by the Patriots defender Malcolm Butler. Game over.

“It should’ve been a run!” It seemed so obvious. Everyone knew it should've been a run. The Patriots players knew it. The coaches knew it. The 100+ million viewers around the world watching on their flat screens knew it. But did they, really? In hindsight, it’s easy to say that the Seahawks should’ve run the ball, because their decision to pass led to an interception. Surely, an unfavorable result (an interception and a loss) must have been caused by a poor decision. It just so happens that this chain of logic is wrong -- and a result of outcome bias.

How is outcome bias playing a role here? The intuition is that the end result doesn’t change the initial probability that a good (or bad) outcome would occur. For all we know, an unfavorable outcome could be the result of bad luck and not bad decision making, poor strategy, or a bad model. So in the case of the Seahawks, a run might have worked, say, 90% of the time…but a pass might have worked 95% of the time! What does the data say? ESPN's data shows that in the past 5 years, on plays where a team is one-yard away from scoring a touchdown, teams chose to run 71% of the time and scored on a little more than half of those plays (54%). A similar analysis reveals that passes led to a touchdown 48% of the time. It’d appear then that running the ball is the smarter play, right? Not quite. The data on Marshawn Lynch reveals that in the past 5 seasons, Lynch converted on only 45% of runs from the one-yard line...and only 20% that season! The takeaway is just because the unlikely bad outcome of the pass happened, it doesn’t necessarily mean the initial choice to pass was wrong.

Read the full article at Misbehaving.

Syon Bhanot, who joined Swarthmore's faculty in 2015, conducts research based on field experiments that use behavioral science concepts to influence decision making with a focus on pro-social behavior, environmental conservation, personal finance decisions, and issues around development, poverty, and welfare. A regular contributor to the Misbehaving blog, he is also an affiliated researcher with the Busara Center for Behavioral Economics and the Yale Applied Cooperation Team. Bhanot earned a B.A. in public policy and international affairs from Princeton University and an M.P.P. and Ph.D. in public policy from the Harvard Kennedy School. 

Jacky Ye '19 is from El Monte, Calif. and is a founder of the Human Library Project.