Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
United States Politics

Why You Can't Rely on Election Forecasts (nytimes.com) 376

Posted by msmash from the closer-look dept.
Zeynep Tufekci, writing at The New York Times: There's a strong case for ignoring the predictions. Why do we have models? Why can't we just consider polling averages? Well, presidents are not elected by a national vote total but by the electoral votes of each state, so national polls do not give us the information we need. As two of the last five elections showed -- in 2000 and 2016 -- it's possible to win the popular vote and lose the Electoral College. Models give us a way to process polls of various quality in 50 states to arrive at a forecast. There are two broad ways to model an event: using "fundamentals" -- mechanisms that can affect the event -- and probabilities -- measurements like polls. For elections, fundamentals would be historically informed lessons like, "a better economy favors incumbents." With polls, there is no theory about why they are the way they are. We just use the numbers they produce.

Electoral forecast modelers run simulations of an election based on various inputs -- including state and national polls, polling on issues and information about the economy and the national situation. If they ran, say, 1,000 different simulations with various permutations of those inputs, and if Joe Biden got 270 electoral votes in 800 of them, the forecast would be that Mr. Biden has an 80 percent chance of winning the election. This is where weather and electoral forecasts start to differ. For weather, we have fundamentals -- advanced science on how atmospheric dynamics work -- and years of detailed, day-by-day, even hour-by-hour data from a vast number of observation stations. For elections, we simply do not have anything near that kind of knowledge or data. While we have some theories on what influences voters, we have no fine-grained understanding of why people vote the way they do, and what polling data we have is relatively sparse.

Consequently, most electoral forecasts that are updated daily -- like those from FiveThirtyEight or The Economist -- rely heavily on current polls and those of past elections, but also allow fundamentals to have some influence. Since many models use polls from the beginning of the modern primary era in 1972, there are a mere 12 examples of past presidential elections with dependable polling data. That means there are only 12 chances to test assumptions and outcomes, though it's unclear what in practice that would involve. A thornier problem is that unlike weather events, presidential elections are not genuine "repeat" events. Facebook didn't play a major role in elections until probably 2012. Twitter, without which Mr. Trump thinks he might not have won, wasn't even founded until 2006. How much does an election in 1972, conducted when a few broadcast channels dominated the public sphere, tell us about what might happen in 2020? Interpreting electoral forecasts correctly is yet another challenge. If a candidate wins an election with 53 percent of the vote, that would be a decisive victory. If a probability model gives a candidate a 53 percent chance of winning, that means that if we ran simulations of the election 100 times, that candidate would win 53 times and the opponent 47 times -- almost equal odds.


This discussion has been archived. No new comments can be posted.

Why You Can't Rely on Election Forecasts More

Why You Can't Rely on Election Forecasts

Comments Filter:

  • There is nothing wrong with forcasts. (Score:5, Insightful)

    by I'mjusthere ( 6916492 ) on Monday November 02, 2020 @10:08AM (#60675024)
    As the summary states in a very long winded way, people do not understand probability. A three in ten chance of winning STILL means someone can win.

    Poll numbers are just a distraction and I almost wish for a simpler time when we didn't have this constant bombardment of those numbers.

    • Re:There is nothing wrong with forcasts. (Score:5, Insightful)

      by AmiMoJo ( 196126 ) on Monday November 02, 2020 @10:29AM (#60675136) Homepage Journal

      In any case, if you have not done so already go out and VOTE.

      This time it matters more than it has for decades. This time you have a real choice, both candidates are NOT the same.

      • Re: There is nothing wrong with forcasts. (Score:4, Interesting)

        by maybe111 ( 4811467 ) on Monday November 02, 2020 @10:48AM (#60675228)

        Hillary was not the same as Trump... which is why I voted for Trump.

        • Re: There is nothing wrong with forcasts. (Score:4, Insightful)

          by AmiMoJo ( 196126 ) on Monday November 02, 2020 @12:33PM (#60675920) Homepage Journal

          At the time people were saying they were the same... Both crooks.

          That was the argument, Trump may be corrupt and dishonest but Clinton is just as bad. Well that's come back to bite everyone in the arse now hasn't it?

    • As the summary states in a very long winded way, people do not understand probability. A three in ten chance of winning STILL means someone can win.

      That's not the whole answer though, otherwise no forecast, however lousy, could ever be said to be 'wrong,' so long as it gave a nonzero probability to every outcome.

      • Re: (Score:3)

        by Sique ( 173459 )
        There is some degree of wrong, and that's what the whole article is about. Forecasts that give a 60:40 chance are always right if used for a single event. If they are used on 100 events, and the actual outcome is 10:90, there is a good chance that the model underlying the forecast is wrong. If the weather forecast gives the rain probability with 30 percent for 100 different days, and it rains at 29 of them, and on 71 days, it stays dry, it's a pretty good forecast. But we only had 12 presidential elections

    • Re:There is nothing wrong with forcasts. (Score:5, Informative)

      by raymorris ( 2726007 ) on Monday November 02, 2020 @10:40AM (#60675190) Journal

      > three in ten chance of winning STILL means someone can win.

      Absolutely. As I recall, in 2016 538 said Trump had about 36% or 38% chance of winning. We means he could certainly win. Currently, Trump's chances stand at about 10%. Possible, but Biden is in a significantly better position than Clinton was.

      In 2016, the polls were off by 2%-3%. This year, if we assume pollsters haven't learned anything and they are off by 3% AGAIN, Biden still wins.

      • Re: (Score:3, Interesting)

        by laird ( 2705 )

        I'll point out that the polls measured public opinion at the time that they were taken, and that public opinion shifted in 2016 right before the election due to Comey's violating department policy and publicly announcing re-opening an investigation into Clinton.

        • Re: (Score:3)

          by Mitreya ( 579078 )

          I'll point out that the polls measured public opinion at the time that they were taken, and that public opinion shifted in 2016 right before the election due to Comey's violating department policy and publicly announcing re-opening an investigation into Clinton.

          That's very true, Nate Silver has an article concluding that while there were many other factors, that could have been enough to move the swing states.
          Also, the polls did not include the undecided voters (which was about 14%). The polls would remain accurate if the undecided voters broke evenly to Trump vs Clinton, but they did not.

          • The polls also failed to take into account education level, which we now know was a strong predictor of who was going to vote for Trump and who was going to vote for Clinton.

            This year, support for Biden appears to cross that educational divide for the most part, while support for Trump has largely remained confined to those without degrees.

            Actually, his base hasn't really expanded at all in 4 years. The percentage of those who approve of him is about the same as the percentage who will vote for him (via nat

      • Re: (Score:2)

        by Nidi62 ( 1525137 )

        > three in ten chance of winning STILL means someone can win.

        Absolutely. As I recall, in 2016 538 said Trump had about 36% or 38% chance of winning. We means he could certainly win. Currently, Trump's chances stand at about 10%. Possible, but Biden is in a significantly better position than Clinton was.

        In 2016, the polls were off by 2%-3%. This year, if we assume pollsters haven't learned anything and they are off by 3% AGAIN, Biden still wins.

        Think of it this way: if there was an activity that had a 30% chance of killing you, would you do it? How about a 10% chance? (consider that skydiving, an activity that many people consider dangerous and possibly lethal, has a .0006% chance of killing you).

        But yes, if we take the results of 2016-both the projections and the final results- and compare them with the current numbers to account for both the normal margin of error and the actual 2016 margin of error, Biden should still have enough to win. But

      • Re:There is nothing wrong with forcasts. (Score:4, Insightful)

        by PyRosf ( 874783 ) on Monday November 02, 2020 @11:31AM (#60675478)
        Your excluding the legal strategy of trying to make up enough lawsuits that favor your voters and disfavor democrat voters in highly targeted ways. They are dragging Houston to the supreme court because they limited the area to 1 drop off box, Houston responded with drive through voting, and now that is going to the court Today (Monday). They could easily have to toss over 100k votes on a whim because someone did not expressly authorize a voting system with extra Covid safty.
    • The purpose of polls is to gauge a candidates campaign performance over time. The trend of the candidates polling over time is a much better indicator of the possible outcome then comparing the absolute numbers for the candidates polls against each-other.

    • Actual votes are what counts. I lie to any pollster that bugs me for info. I tell them someone other than who I actually voted for....it's none of their business.

    • Also a government, who actually cares about its population would also use national polls to get an inkling on what is going on.

      Lets say (You can if you feel like swap the parties around too) you live in a solid Red District or State. However the polling sees that from 2016 - 2020 you see a 10% increase in interest in electing a democrat. Sure that area will still go Republican, however if you are smart you will see that the Demographics in that area are changing, or something is going on that people don

      • Re: (Score:3)

        by sl149q ( 1537343 )

        The Republicans govern by a poll.

        Every day they poll one person and change their policies to match his thinking of the day.

        Unfortunately, it is the same person every day.

      • Governments already do this. It's known as poll driven politics [irpp.org] and it is often criticized because, from one perspective, it means pandering to the masses when they are wrong, rather than doing what's logical and right. Politicians may seem ideological, but that's usually because polls tell them that their voters are ideological.

  • Too much assuming (Score:5, Insightful)

    by nospam007 ( 722110 ) * on Monday November 02, 2020 @10:11AM (#60675046)

    They assume that only 4% of people are dicks and giving the wrong answers on purpose.
    According to my environment, the number should be more like 40%.

    • Re:Too much assuming (Score:4, Interesting)

      by apoc.famine ( 621563 ) <apoc...famine@@@gmail...com> on Monday November 02, 2020 @11:19AM (#60675406) Journal

      It's not just that. People self-select into polls. Nobody is forced to talk to a pollster. I have a hard time believing that the people who pick up the phone and talk to a stranger about politics are really representative of broader society.

      • Re: (Score:2)

        by Nidi62 ( 1525137 )

        It's not just that. People self-select into polls. Nobody is forced to talk to a pollster. I have a hard time believing that the people who pick up the phone and talk to a stranger about politics are really representative of broader society.

        That's the one thing that makes the "shy/silent Trump voter" argument seem a little shaky. Why would you talk to a pollster in the first place if you were afraid to tell them who you really supported? I would think you would be more likely to simply hang up, especially since people are desensitized to hanging up on people with all the spam calls you get nowadays.

        • I immediately hang up or ignore any any unsolicited phone calls or visits to my home. I am friendly though with people I meet randomly like in a grocery check out or restaurant. It's not about giving misleading results, its about removing yourself from polls.

      • They weight by age, income and education and other factors to try to adjust for that.

        One of the mistakes of 2016 was not weighting by education which has become an important predictor of voting intention, but most pollsters have fixed that problem.

    • Generally between 5 and 25% on anonymous polling of other subjects.

  • Poll Response (Score:5, Interesting)

    by JBMcB ( 73720 ) on Monday November 02, 2020 @10:12AM (#60675054)

    A major factor is phone poll response rate. According to Pew, it's decreased from ~30% in 2000 to ~6% in 2018. Overall it's probably much less than that.

    Now that phone numbers are, basically, ephemeral, having no concrete tie to any specific person or place, it's *really* hard getting accurate location-tied information. I have three phone numbers on various devices that put me in three different counties. My wife's work-issued cell phone number places her on the other side of the state. People send in mailed polls at an even lower rate than phone polls. I've seen surveys that come with cash in the mail asking us to fill it in, promising a gift card if we do.

    As for on-line polls, there is essentially no verification of address unless you are buying something, so unless the poll is coming from an on-line store, it's also basically worthless.

    • Re: (Score:2)

      by AmiMoJo ( 196126 )

      Do you not have exit polls in the US? In the UK they ask people leaving the polling station which way they voted.

      It's imperfect because of course some decline to say and some lie, but it tends to produce fairly accurate predictions.

      • We have exit polls, but they're not going to help as much in this election because of all the mail-in voting. They also don't help with predictions ahead of time.

        • Re: (Score:2)

          by AmiMoJo ( 196126 )

          Ah that's true. There isn't much that can be done for mail in votes.

          Good that so many people have already voted though.

          • Re: (Score:2)

            by pairo ( 519657 )
            Polling those people over the phone works and is being taken into account by most polls.

            • We've just been discussing why phone polling is decreasingly valuable, mostly because of land line abandonment and also increased volume of spam calls leading people to not answer phone calls unless they recognize the caller.

              In fact, phone polling does NOT "work" any longer.

              • Re: (Score:2)

                by pairo ( 519657 )
                That's true, but that's taken into account. Whether the correct adjustment is made or not, that's debatable. But, both phone and in person polls _do_ work just as well as they did, they're just way more expensive to conduct. Yes, this year exit polls will be slightly less reliable, since you have basically a normal poll and exit poll, but, then again, exit polls aren't about finding out the winner early, they're about figuring out what went wrong with the assumptions made later on.
        • I walk right past the exit poll person just like I ignore calls from people I don't know.

      • Do you not have exit polls in the US?

        You do know what "forecast" means, right? :)

    • Re: (Score:2)

      by pairo ( 519657 )
      That's not very likely: https://fivethirtyeight.com/fe... [fivethirtyeight.com]

    • Any unsolicited call to my phone gets you nothing but a "Go fuck yourself, never call me again" anyway, and I'm not the only person who will rightfully bite the head off someone who bothers me with such archaic bullshit technology.

    • Re: (Score:2)

      by Mitreya ( 579078 )

      A major factor is phone poll response rate.

      From what I understand the polls were skewed by the undecided voters (about 14% in 2016). If the undecided voters broke evenly to Trump vs Clinton the results would be far closer to accurate.
      A pollster cannot be expected to predict votes of the undecideds in their estimates.

  • Lies, damn lies and statistics (Score:4, Insightful)

    by sinij ( 911942 ) on Monday November 02, 2020 @10:19AM (#60675086)
    Polls are all based on models. They can go wrong two ways - a) GIGO and b) bad models. Both are in play this election.

    GIGO (garbage in, garbage out) is substantial issue with collecting data in 2020. There are many problems of various severity - from people not answering the phones due to spam to people just not willing to admit voting for Trump. When was the last time you answered your phone from unknown number? Me too, just too many calls from "Microsoft Technical Support". Consensus is that Trump voters are under counted, the only question is by how much. Even if (that is a big IF) you have an accurate model, feeding it flawed data will result in flawed results.

    The second issue are inaccurate models used to predict the outcome in 2020. A lot of modeling is heavily based on socioeconomic and demographic data (e.g. income, race, age, location). Basically, "past performance" is bulk of modeling using by polls. 2020 is unique for many reasons, as such models based on past are a lot less accurate.

    Existing polling severely underestimates uncertainty of the results and should not be take at face value. We won't know the results until the votes are counted. Everything else is reading the tea leaves at this point.

    • Re:Lies, damn lies and statistics (Score:5, Informative)

      by Nidi62 ( 1525137 ) on Monday November 02, 2020 @10:30AM (#60675142)

      Existing polling severely underestimates uncertainty of the results and should not be take at face value. We won't know the results until the votes are counted. Everything else is reading the tea leaves at this point.

      Which is why a potential gameplan for one side is to, if they appear ahead at specific points in specific states, declare victory and try to stop (or at least discredit) further vote counting.

      https://www.axios.com/trump-cl... [axios.com]

      And remember: due to state law it is physically impossible to know on election night the correct vote totals for specific states, including battleground states like Pennsylvania. All indicators are that Democrats are leading in early/mail-in voting, while Trump will lead in votes cast day of. In Pennsylvania state law says that mail-in ballots cannot be processed and counted until 7am on election day (7 counties have said they won't even start counting them until the following day). Over 2.4 million mail-in ballots have been returned, which is more than 1/3 the total number of voters for the 2016 election.

      • Re: (Score:2)

        by sinij ( 911942 )
        Yes, this is awful plan. I am not onboard with any attempt to undermine integrity of elections. However, Trump is on record telling anyone who would listen that pushing for mass mail-in voting is a bad idea. You can't trust something this important to unreliable mail. This in turn trapped Democrats in no-win position of being all-in on mail-in voting.

        Estimates I read somewhere is that approximately 2% of mail in votes expected to be lost in the mail. As Democrat voters are a lot more concerned with preven

      • "All indications"?

        In Wisconsin, considered to be a battleground state by at least the Republicans, early voting has been heaviest in areas dominated by Republicans. That's still not "votes counted", because legally that doesn't start until November 3rd.

        Lots of people are doing comparisons of which polls got the most accurate results in 2016, and how they compare in 2020. One "mostly accurate" poll doesn't ask people who they're planning to vote for, because many lie; They ask, "How do you think your neighbo

        • Re: (Score:2)

          by Nidi62 ( 1525137 )

          Heck, I'm getting calls and texts to vote Democratic in Texas, and I've never lived there. I can understand that sort of thing in Chicago, but Texas? Maybe I'm participating in polls down there that I don't know about...

          I've been getting flooded with election texts from both sides, especially Republican, is my southeastern state (ironically none addressed to me, but to my father and sister-we all are on the same mobile account and number is the primary). And the airwaves are completely saturated with ads as well. I would assume someone gave your phone number out as theirs somewhere (or just miswrote/mistyped) and it's bad data, but the fact is that these traditionally red southern states are coming more and more into pla

        • You're just making up numbers while wishful thinking.

          https://electproject.github.io... [github.io] has the actual released mail ballot returns, and you can see that urban counties are returning at about the same rate as rural counties, even though processing is slower in larger counties.

          You also seem a little confused about people's ability to aggregate polls; if you thought the substantive comments were cherry picking one poll, you're an idiot you can't read.

          Did you know that the "over-sample" has already been correct

    • people just not willing to admit voting for Trump

      If you're voting for someone so heinous you're not willing to admit voting for them, you've got a bigger problem than the polls.

      • Re:Lies, damn lies and statistics (Score:5, Insightful)

        by sinij ( 911942 ) on Monday November 02, 2020 @11:05AM (#60675326)

        people just not willing to admit voting for Trump

        If you're voting for someone so heinous you're not willing to admit voting for them, you've got a bigger problem than the polls.

        Views like this, cancel culture and social media dragging is exactly why people are not willing to admit voting for Trump. People like you created this problem.

        • Re: (Score:2, Interesting)

          by drinkypoo ( 153816 )

          If you're voting for someone so heinous you're not willing to admit voting for them, you've got a bigger problem than the polls.

          Views like this, cancel culture and social media dragging is exactly why people are not willing to admit voting for Trump.

          Views like yours are why we're calling Trump voters snowflakes these days, when they were the ones who popularized the term. For the most part, they're white and fragile. The idea that you should own your decisions being too scary for Trump supporters only proves the point. They're cowards who lack the courage of their convictions, except when they get in large enough groups to harass a campaign bus.

          • Re: (Score:3)

            by Shotgun ( 30919 )

            Not wanting to put up with shitheads like yourself doesn't make one a snowflake. Not wanting to had your house tagged or your car keyed, or your business burned to the ground in the middle of the night doesn't make you a snowflake.

  • Comment removed (Score:4, Insightful)

    by account_deleted ( 4530225 ) on Monday November 02, 2020 @10:20AM (#60675090)
    Comment removed based on user account deletion

    • Re: (Score:3, Interesting)

      by Nidi62 ( 1525137 )

      No one in the right mind is going to admit to voting for Trump these days except among friends who are inclined to do the same.

      So you agree then that all of those people driving around in pickups flying Trump flags are out of their mind.

      • Those driving around trying to push campaign buses off of the road are likely out of their minds to be sure. The others going on the "drive around in trucks and SUVs with your banners waving" road rallies may or may not be. Though you'd have to be out of your mind to show up to one of those road rallies with a Prius, they'd assume you were a Democrat plant if you did that!

        I don't see the point of such road rallies myself. I don't see the point of going to a presidential rally either. I also don't see th

    • Re:Simple: there's an incentive to not talk about (Score:5, Informative)

      by Mitreya ( 579078 ) <[moc.liamg] [ta] [ayertim]> on Monday November 02, 2020 @12:14PM (#60675750)

      No one in the right mind is going to admit to voting for Trump these days except among friends who are inclined to do the same.

      Have you seen the videos of Trump supporter interviews? Granted, that's not all Trump supporters, but many are very vocal. I mean, his supporters would confront/derail Biden cars on the road [texastribune.org], so I don't think they would hesitate to admit voting for Trump.
      Also, if there are so many shy Trump voters, he should be able to win the popular vote too. (Yes, I realize it doesn't matter for the election purposes, but it would confirm this theory).

  • Forecast enough and the forecast will be wrong. People look at them and then do not bother to vote. That gave the UK the Brexit, that gave the US Trump and other catastrophes are on record too.

  • Ignore 99% of what we have been selling you for the past 4 years...

  • ... to be biassed in favor of the Democrats by 6-8%

    You can't trust the leftists, but you can trust them to behave like leftists. The people with university degrees in Sociology and such, who manage the pollsters, design the polls, and interpret the raw data, are overwhelmingly left leaning.

    The scenario of 2016 happens again in 2020, and will happen in 2024. At the last moment, the margin between the candidates will disappear or even reverse, "explained" by some late event that in reality did not change

    • Re: (Score:2)

      by Nidi62 ( 1525137 )

      ... to be biassed in favor of the Democrats by 6-8%

      You can't trust the leftists, but you can trust them to behave like leftists. The people with university degrees in Sociology and such, who manage the pollsters, design the polls, and interpret the raw data, are overwhelmingly left leaning.

      The scenario of 2016 happens again in 2020, and will happen in 2024. At the last moment, the margin between the candidates will disappear or even reverse, "explained" by some late event that in reality did not change votes. The pollsters will cover their asses and play the same trick next time.

      Yeah, damn all those radical leftists in the Republican party and at Fox News for putting out all those polls showing Biden in the lead! "Wait, what?" you ask? Yes, even Fox News has published polls showing Biden leading and as for the Republican party, well, both parties do internal polling all the time, especially in major elections such as this, to gauge how they are doing in the down ballot races (state level elections, House/Senate, etc). During a presidential election year these down ballot races te

    • Re:You can rely on the polls... (Score:4, Insightful)

      by Junta ( 36770 ) on Monday November 02, 2020 @11:33AM (#60675500)

      Showing Biden in the lead but within a margin of uncertainty seems like a good way to get Trump supporters to come out and vote? It's not a hopeless cause, but needs all the support needed to make it happen.

      I don't see how 'leftists' would want to present poll data that undermines them and causes them to say every 5 seconds 'don't rely upon the polls, Biden can easily still lose'.

      I will say that 2016 to some extent had some novel facets that couldn't be easily predicted, and especially 2020 has a lot of wildcards in the mix so who knows how badly mismatched polls will be from reality.

  • IIRC the only pollster that predicted a win for Trump in 2016 was the one that asked: "Who do you reckon your neighbour will vote for?" Many people didn't want to admit publicly that they were voting for Trump, and in the past 4 years that has only gotten worse. As Jonathan Pie said [youtube.com]: in this climate, people are afraid to confess their political preference... but they will still express their opinion in the polling booth.

  • I use the Weather Forecast too (Score:3)

    by RobinH ( 124750 ) on Monday November 02, 2020 @11:20AM (#60675412) Homepage
    I rely on the weather forecast to plan my weekend, even though it's often wrong. It's just better than random chance, which is why it's useful.

  • We have secret ballot, so people may deviate from what they claim they will/would vote.

    You also have a guess as to whether poll participants can and will actually vote. Particularly this year this is a wildly unknown territory given the pandemic situation, both people abstaining from voting out of fear on the one hand, on the other hand aggressive enablement of absentee ballots may make people vote who didn't bother before actually vote.

    Also have to guess about the people who decline to participate in polli

  • You can't rely on election forcasts (Score:5, Insightful)

    by rsilvergun ( 571051 ) on Monday November 02, 2020 @12:55PM (#60676074)
    because one side is cheating their balls off.

    There's been multiple cases [fark.com] of voter intimidation [reddit.com], an attempt to throw out 100k ballots [twitter.com] that's on it's way to a (packed) federal court and the President has said he will declare victory before the votes are counted [axios.com]

    And don't get me started on the rampant voter suppression. I'm sure it's just a coincidence that the company in charge of ballots in swing state Georgia is run by the opposition party [foxnews.com] and didn't deliver 100k+ ballots in districts that lean heavily against his party. Or that voter roll purgers disproportionately target minority and youth voters. Or that the average wait time to vote is 10 times higher if you're black [ajc.com]

    At a certain point it's time to call a spade a spade. The GOP can no longer win on the strength of their ideas (if they ever could) and are outright cheating to win. The only question is, are we as a nation OK with that.

    We'll know in about 3-5 days.

Slashdot Top Deals

A computer scientist is someone who fixes things that aren't broken.

Close