So what were the polls saying in 1996 and 2000? (Answer: Most were wrong)

[pmbryant]

I seem to be failing in my resolution after the last election to ignore the major media polls. So to try to channel that wasted nervous energy into something moderately productive, I decided yesterday to take a look at how the major polls did in the last couple Presidential elections.

In 2000, the polls in the last two weeks prior to election day ranged from a tiny lead for Gore (+1/2, Zogby & CBS) to a large lead for Bush (+9-13, Rasmussen, Gallup). Most were in between. The final result was at the extreme end of that range, of course.

In 1996, the final polls ranged from a Clinton blowout (+18, CBS/NYTimes) to a much more modest victory (+8, Zogby). Most were in between. Again, the final result was at the extreme end of that range.

What lessons do we take from this? Trust Zogby, since he got it right both times? Or simply pick an extreme end of the polling range (e.g., large victory for Bush or modest but decisive victory for Kerry)?

Zogby is certainly worthy of paying close attention to given his track record nationally. But part of his success in the last two Presidential elections was due to luck, since statistical variation alone could cause him to miss the margin by 3-4 points, and that hasn't happened yet. (In 1996 he was dead on, and he was off by 1.5 points in 2000.)

The real lesson, in my opinion, is to simply not trust the polls unless they indicate a very large lead (over 10 points) for one candidate, and even then don't trust the actual margin. In 1996 and 2000, when taken as a whole, the major polls were significantly biased in one direction or the other (by 4-5 points overall). In 1996 towards Clinton; in 2000 towards Bush. No doubt they are still biased by a similar amount. Perhaps even worse, since by all reports there will be many more new voters than usual in this election.

In which direction is the bias?

I think we have a good feeling about the answer to that one.

But time will tell.

Keep up the good work, everyone. We are very close to restoring our country to greatness.

:bounce:

--Peter

edit: added 'answer' to subject, since original subject wasn't attracting much attention

[otohara]

1. Thanks

I was wondering about 96 just yesterday when my state went from Clinton to Dole....maybe this year will be another 92 for CO

[pmbryant]

2. My impression is that state polls are not very reliable

Many are reasonably accurate, but a significant number are way off.

I'm keeping my fingers crossed for Colorado, as well, though I think it is a longshot.

--Peter

[Beetwasher]

3. Kick

n/t

[WI_DEM]

4. In '96 Zogby was the most correct

since in the end Clinton did win by 8--49-41.

[pmbryant]

5. That was his big career break, because everyone else got it wrong

Overall, the major media polls have had a 4-5 point bias in each of the last two elections. I haven't looked into why that is.

Is it an incumbent bias? Almost certainly not. That would explain Clinton's exaggerated lead in 1996, but doesn't explain Bush's false lead in 2000. Perhaps pollsters overcompensating for their 1996 Dem bias explains that?

We'll see if the bias is showing up this year as well. I have a strong feeling that it is, and it is a bias in Bush's favor---that Kerry will do significantly better than most polls indicate. But it's just a feeling.

:shrug:

Peter

[pmbryant]

6. It turns out 1980 and 1992 weren't such great performances either

Here's a handy chart of historical Presidential polling: http://www.ncpp.org/1936-2000.htm

And here's my brief analysis of 1992 and earlier that I posted in another thread:

Using that chart, it appears a similar bias has appeared in many other previous Presidential elections as well:

In 1992, the six polls listed predicted a Clinton win of 6-12 points. He won by 6. Again, at the extreme end of the range.

In 1988, the five polls listed predicted a Bush win of 4-12 points. He won by 7. No clear bias, though 4 of the 5 overestimated the size of Bush's victory.

In 1984, the six polls listed showed a huge range: anywhere from 10 point Reagan win to 27 point. Again, no clear bias, but that range is much larger than the statistical error.

In 1980, the four polls listed all predicted a smaller Reagan victory than he got: from 1 to 6 points, when he won by 10. Not a great performance.

Before that, there are very few polls listed for each election, and it becomes very hard to detect any bias if there was one. No obvious one shows up. (Except for 1948 and 1952 when Gallup was 10 points off each time.)

So the 1992, 1996, and 2000 elections show this bias: where the final outcome was only correctly predicted by an "outlying" poll.

Before that, the track record is somewhat better (except for 1980), but that was a very different polling universe than exists today.

So I have every reason to believe that this bias still exists today.

But I also have every reason to believe that, indeed, this is a close race.