During the 2012 election, Republicans who hated the daily onslaught of polling showing that Mitt Romney was headed toward a comfortable defeat turned to Dean Chambers, the man who launched the website Unskewed Polls. The poll numbers were wrong, he said, and by tweaking a few things, he could give a more accurate count. His final projection had Romney winning close to all 50 states.
Chambers has wisely abandoned the field of election forecasting, and this year says he thinks the various models predicting a Hillary Clinton victory are probably accurate. The models themselves are pretty confident. HuffPost Pollster is giving Clinton a 98 percent chance of winning, and The New York Times’ model at The Upshot puts her chances at 85 percent.
There is one outlier, however, that is causing waves of panic among Democrats around the country, and injecting Trump backers with the hope that their guy might pull this thing off after all. Nate Silver’s 538 model is giving Donald Trump a heart-stopping 35 percent chance of winning as of this weekend.
He ratcheted the panic up to 11 on Friday with his latest forecast, tweeting out, “Trump is about 3 points behind Clinton ― and 3-point polling errors happen pretty often.”
So who’s right?
The beauty here is that we won’t have to wait long to find out. But let’s lay out now why we think we’re right and 538 is wrong. Or, at least, why they’re doing it wrong.
The short version is that Silver is changing the results of polls to fit where he thinks the polls truly are, rather than simply entering the poll numbers into his model and crunching them.
Silver calls this unskewing a “trend line adjustment.” He compares a poll to previous polls conducted by the same polling firm, makes a series of assumptions, runs a regression analysis, and gets a new poll number. That’s the number he sticks in his model ― not the original number.
He may end up being right, but he’s just guessing. A “trend line adjustment” is merely political punditry dressed up as sophisticated mathematical modeling.
Guess who benefits from the unskewing?
By the time he’s done adjusting the “trend line,” Clinton has lost 0.2 points and Trump has gained 1.7 points. An adjustment of below 2 points may not seem like much, but it’s enough to throw off his entire forecast, taking a comfortable 4.6 point Clinton lead and making it look like a nail-biter.
It’s enough to close the gap between the two candidates to below 3 points, which allows Silver to say that it’s now anybody’s ballgame, because “3-point polling errors happen pretty often.”
That line in itself is disingenuous, though. For the polls to be wrong, there wouldn’t need to be one single 3-point error. All of the polls ― all of them, as Brianna Keilar would put it ― would have to be off by 3 points in the same direction. That’s happened before, but in 2012 the error favored President Barack Obama. In 2014, it favored Republicans. Errors are just as likely to favor Clinton as they are to favor Trump, and they would have to favor Trump. And we still haven’t accounted for the unique fact that one campaign has a get-out-the-vote operation, while the other doesn’t.
By monkeying around with the numbers like this, Silver is making a mockery of the very forecasting industry that he popularized. “The idea that she’s a prohibitive, 95 percent-plus favorite is hard to square with polling that has frequently shown 5- or 6-point swings within the span of a couple weeks, given that she only leads by 3 points or so now,”he told Politico recently. “[E]verything depends on one’s assumptions, but I think that our assumptions ― a Clinton lead, sure, but high uncertainty ― has repeatedly been validated by the evidence we’ve seen over the course of the past several months.”
I get why Silver wants to hedge. It’s not easy to sit here and tell you that Clinton has a 98 percent chance of winning. Everything inside us screams out that life is too full of uncertainty, that being so sure is just a fantasy. But that’s what the numbers say. What is the point of all the data entry, all the math, all the modeling, if when the moment of truth comes we throw our hands up and say, hey, anything can happen. If that’s how we feel, let’s scrap the entire political forecasting industry.
Silver’s guess that the race is up for grabs might be a completely reasonable assertion ― but it’s the stuff of punditry, not mathematical forecasting.
Punditry has been Silver’s go-to move this election cycle, and it hasn’t served him well. He repeatedly pronounced that Trump had a close to 0 percent chance of winning the Republican primary, even as he led in the polls. “Trump’s chances [are] higher than 0 but (considerably) less than 20 percent,” he wrote in November.
Silver was far from alone in being wrong. I said myself, at the time, that Trump had no chance of winning the primary. But I did so as a pundit, and, as a pundit, I am endowed by my creator the inalienable right to be consistently wrong and never apologize. But our polling model always forecast a Trump win, and our polling team, led by Natalie Jackson, stuck by that prediction, because their job is to follow the numbers. They were right, while pundits like me and Silver were wrong.