Summer Polls: Stabs in the Dark (or Worse)


    In any trend, there’s going to be a moment when you realize things are out of control. That cool little bar band hasn’t just gotten noticed — they’ve sold 20 million records and now their second album is coming out and everyone’s expecting it to be the second coming of The White Album.

    And so when FiveThirtyEight brought easy-to-understand statistical analysis to the national stage in the 2008 cycle, there was sure to be a phase of irrational exuberance. When people start pretending polls are photographs of facts, not lobs at the dartboard. And that a SurveyUSA poll in July, when people’s #1 issue is how am I gonna get all this sand out of my car, that claims 97% of voters have already made up their minds might be the definition of worthless.

    As Josh Marshall at TPM points out yet another example that polls are far from perfect — intentional or not, the possibility of bias at SurveyUSA:

    Not quite sure what to make of this yet. But SurveyUSA’s polls this year are leaning pretty strongly toward the GOP candidates relative to most other pollsters.

    What caught my eye was this result for the California Senate race which has Fiorina over Boxer by 5 points, even though all the other recent polls, including Rasmussen, show Boxer ahead by at least a few points and in some cases by almost 10 points. So for instance, the current TPM Poll Average for that race gives Boxer a 2.6 point edge. But take out SurveyUSA and the margin pops up to 5 points, which is a big difference when a lot pollsters are following this race.

    In the days before the Jim Webb-George Allen contest in 2006, the final SurveyUSA poll had Webb as a lock — up 8% with only 1% undecided. Webb won by less than half of 1% of the final vote.

    Polls can be horribly wrong even the week of the election. How much of a stab in the dark do you think they are in the middle of summer?

    • VA Blogger

      I would argue with you over the substance of your post, but there is none.

      You don’t trust any poll because one shows Carly Fiorina up? Or you don’t trust SurveyUSA, ranked as one of the most accurate pollsters, because they didn’t nail every race exact?

      Polling is not always exact, but it certainly is scientific. You can choose to not believe polls, but that doesn’t make them wrong or irrelevant.  

    • VaPolitico

      My favorite part of this post is how wrong you are regarding the final SurveyUSA poll.

      As you can see, that poll had a margin of error of + or – 3.7%.  For those of us who have taken statistics classes, that means either or both candidate’s numbers can be off by as much as 3.7%. I’m not sure what their confidence in the margin was, but I’m gonna guess in the 95-99% range.  

      You whine that the poll had Webb up 8% (52% – 44%) but he only won by less than half a percent, and thus the poll was wildly inaccurate.  In reality, Webb could have been as low as 48.3% and Allen as high as 47.7%, and the poll would still be considered very accurate.  48.3-47.7 = 0.6 <– This is considered as accurate a poll spread as the 8% you (and the poll) cited.  

      Now I know, I know, 0.6% is wayyyyyyy more than, say 0.4%, but still, Survey USA was pretty on-the-money with that race.  So what was wrong with polls near Election Day again?

      P.S.  You are right about midsummer polls.  Those are a joke and really only useful in helping political parties decide where to spend their money.

    • VaPolitico

      …my apologies.  The post was a result of years of pent-up anger at reading blogs where people don’t understand how margin of error properly works.

      Polls are really only valuable at taking a snapshot of current views on an upcoming election.  If the election were held today, Perriello would get crushed, for example.  Hopefully (and based on what happens two years ago), that margin will close dramatically and will even reverse itself.