Our forecasting model for America’s Senate elections is trained on every race for a seat in the upper chamber of Congress since 1972, and makes use of data on elections going back to 1942. It calculates its predictions using three basic steps.

The first challenge for the model is to predict an accurate range of outcomes for the national popular vote for the House—the sum of all votes cast for Democratic or Republican House candidates, with an adjustment for seats where one party is running unopposed. (The national popular vote for the Senate is not particularly useful, because one-third of states do not have senators up for election, and states receive equal representation in the Senate regardless of their population.) To calculate this distribution, the model uses data on “generic-ballot” polling—when survey respondents are asked which party’s Congressional candidate they plan to vote for—as well as presidential approval ratings; the average results of special elections held to fill vacant legislative seats; and the number of days left until the election. Using a machine-learning technique called “elastic-net regularisation”, the model finds the combination of these variables that would have produced the most accurate predictions of elections it was not allowed to “see” in its training set.

With this distribution of plausible results for the overall national political environment in hand, the model drills down to the state level. Here, its first challenge is to predict each state’s “partisan lean”—the gap between the election result in each state and the overall national average. Some Senate races are never polled, so the model creates a starting benchmark estimate for each district based exclusively on “fundamental” factors like its historical voting record; whether or not an incumbent is running; the candidates’ fundraising, ideological positioning and past experience running for office; and—crucially—the nationwide popular vote for the House. Rather than assuming that a state’s partisan lean (e.g., seven percentage points more Republican than the national average) is fixed, the model estimates how voters might respond differently to their choices under different political contexts. The model also includes an adjustment for partisan polarisation—which we measure as the share of the electorate that voted in consecutive presidential elections for candidates from different parties—which enables it to distinguish how the relative impact of each variable has changed over time.

After calculating this starting expectation in each state, the model next updates its forecast to incorporate any polls that have been taken of the race. It weights surveys by their methodology—pollsters who call mobile phones using live interviewers and belong to organisations committed to transparency in research get more weight—and by how long before the election they were taken, and adjusts the results of polls sponsored by candidates or political parties to counteract their expected bias. In heavily polled races, the fundamentals-based forecast constitutes just a small share of the final blended average, but the model will continue to rely on it heavily in contests with sparse or unreliable polling. The more polling a race has, the more confident the model can be of its forecast—and thus, the narrower the distribution of potential outcomes around its central estimate will be.

The final step is to combine these three elements into a forecast, by randomly simulating a result in each race 10,000 times. To start the simulation, the model picks 10,000 values at random from its distribution of outcomes for the national popular vote. Most will cluster around the average—so, in a year where Democrats are most likely to get 53% of the vote, the bulk of simulated values will fall between 52% and 54%—but some simulations will produce highly improbable outliers. Each of these results represents one hypothetical national political context in which each Senate race will occur. In presidential-election years, the model also simulates which party will control the vice-presidency, which casts the tiebreaking vote in case of a 50-50 split, based on the simulated national popular vote for the House.

For each of the 10,000 simulations, the model then feeds the simulated national result down to the state level, and calculates a distribution of potential vote shares for each race in each simulation. In scenarios that are unusually good for the Republicans, all of these distributions will shift in the GOP’s direction, but not necessarily by the same amount. The model then proceeds to draw one value at random from each of the 350,000 resulting distributions. Even in simulations where the Democrats romp to victory nationally, Republicans will still pull off some surprising upsets in a few unexpected races.

From there, the model simply counts up the number of seats won by each party in each simulation. Those in which the Democrats win at least 51 seats, or 50 plus the vice-presidency, are recorded as a Democratic victory; the rest go to the Republicans. The probability of victory published on our site is simply the percentage of our 10,000 simulations won by each party.

Updated October 29^th, 2020:

In response to feedback from numerous readers, we have adjusted how our forecasting model for America’s Senate simulates the party affiliation of the vice-president, who casts tiebreaking votes in the event of a 50-50 split. Previously, the model estimated the national popular vote for the presidency solely based on the historical relationship between it and the national popular vote for the House of Representatives. The model's only interaction with our separate presidential forecast was to derive an estimate of which party had the advantage in the electoral college relative to the popular vote, and by what margin. Our revised version fully incorporates the presidential model, using its estimate of the national popular vote for the presidency as a starting point, and then adjusting it up or down in each simulation based on the simulated national popular vote for the House of Representatives.

This modification has a significant impact on our overall forecast. Previously, the model estimated that Democrats would hold the vice-presidency in just one-fourth of 50-50 ties; now, they get the tiebreaker over 90% of the time. With the chances of a 50-50 Senate currently at 11%, this increases the Democrats’ probability of control by seven percentage points, from 75% to 82%. To avoid confusion, we are replacing the historical time series of daily predictions displayed on our website with what the current model would have projected on each past day of the race.