The Bayes-ics of Election Forecasting

It’s nationwide election time yet again. As of October 30th, more than 85 million Americans have already cast their ballot, a remarkable number considering total voter turnout for the 2016 election was 138 million. By the time you’re reading this, we may or may not yet know the winners, especially given the massive increase in mail-in voting this year. But that hasn’t stopped pollsters and psephologists — political scientists who study the quantitative analysis of elections and balloting — from trying to predict who they will be by using various election forecast models as a tool to do so.

How do these election forecast models work? A peek under the hood reveals a complex mess of sophisticated mathematics. However, at the core of it is a statistical framework that’s older than our government itself: Bayes’s theorem.

As we will see, Bayes’ theorem is especially suited for political analysis because it allows us to quantify how our beliefs about the probability of certain events occurring can change over time. (It’s found its way into physics, too).

Bayes’s theorem is named after Thomas Bayes, who conceptualized the idea in his notable work, “An Essay towards solving a Problem in the Doctrine of Chances.” Published posthumously in 1763, Bayes’s theorem was presented as a solution to an inverse probability problem, concerned with determining some unobserved variables in experimental contexts. For example, an inverse probability problem in astronomy could be: given that I have data describing the approximate location of a star in the sky, can I estimate its exact location? Today a problem like this would be described as “inferential statistics.”

The equation for Bayes’s theorem is as follows:

In words, we can read the equation as to calculate the probability of A given B, we multiply the probability of B given A by the probability of A, and divide by the probability of B.

To use Bayes’s theorem (or to make any kind of experimental calculation), we need to start by making several assumptions. These are known as our “priors,” our initial beliefs about the probability of something coming true. For example, in the context of elections forecasting, a psephologist might say: “Given that Candidate A has won their past five re-election races, I assume that they have an 80% chance of winning again.” We construct our best-guesses about what will happen based on our experiential knowledge of the past.

However, as events happen or we learn new information, our beliefs may adjust accordingly. Say that Candidate A recently decided to take an unfavorable stance on a particular issue. This may, in turn, affect their odds of winning the race. In response, the psephologist might decrease Candidate A’s probability of winning the race from 80% to 70%. This new probability is a “posterior,” an updated probability estimate based on new information.

Notice, however, that these probabilities are calculated based on individual beliefs. Our intuitions about what is or is not more likely to happen are inherently subjective, shaped by our personal experiences. This is one common criticism of Bayes’s theorem (often cited by those who adhere to a frequentist approach to statistics) — if one researcher’s priors differ from another, they can potentially reach conflicting conclusions. To mediate discrepancies, the Bayes factor is calculated as a quantitative measure of how likely one researcher’s hypothesis is compared to the other; it’s a numerical basis to adjust your own beliefs if your priors don’t align with another proposed hypothesis.

Bayesian hypothesis testing offers a structured way to quantify how our beliefs about things that could happen change as we learn new information; it’s a measure of our everyday thinking. This is precisely what makes it a useful approach to developing election forecast models. The history of election analytics stretches back to the 1960s, as computer programmers attempted to apply principles of the very-new field of data science to predict election outcomes — one example was called the “People’s Machine.” Ever since the practice of predicting opinions on the macro-scale has become deeply ingrained in our political processes.

One popular election forecast is published by FiveThirtyEight. Their forecast model accounts for many different variables, including national and state-wide polling averages, voting accessibility per state, economic uncertainties, and even major news events, such as debates or party conventions. Three of their important priors are (1) partisan lean index, a reflection of how a state voted in the preceding two elections; (2) state’s elasticity, or how likely they are to swing between parties; and (3) cost of voting index, a measure of how easy it is to vote in different states. As things have changed over the past two months since the forecast was released, the model updates itself to incorporate their estimated impact on the predicted election outcomes. The methodology is certainly complex, but Bayesian principles still apply.

Bayesian statistics is an important technique for modern election forecasting and a host of other empirical endeavors. It’s also adopted in many areas of physics, from cosmology to condensed matter and machine learning. The elegant statistics informing how we measure our intuitions about the world today and how it could be tomorrow are fascinating, and they frequently remind us that yesterday’s posteriors are today’s priors.