Exit Polling Methodology

[LilKim]

It has always been my understanding that the purpose of exit polling in the U.S. has never been so much to project a winner but more to allow a postmortem analysis of the election, i.e. to find out who voted for whom and for what reasons.

Therefore in order to collect such data, those doing the polling are very much guided by the demographics of the precinct or area they are stationed in. I'm guessing they might have in hand the latest census data of the different kinds of people and proportions that populate their precinct, so that by the time the polls close, they will have a rough grasp of whether they've sufficiently sampled all the relevant demographic groups. And in making sure they hit all their demographic targets, they likely oversample minority groups, as they want to be sure they have more data than they need, rather than less, since election day is the only time available for sampling.

So, knowing this, as the sample data comes in, the tabulators must reweight and adjust it so that it does actually reflect those known demographics. For example, a sampler with a ream of polling questions will know ahead of time that their precinct is 9% black, but in their polling "just to be sure" their sample ends up being 15% black. When their stack of questionaires is received back to be tabulated the sample of black people in it is reweighted back to 9% as it has to be if the exit poll it to be demographically accurate.

My overall point is that, by their very demographic driven nature, 'raw' unadjusted exit poll numbers are not at all an accurate reflection of the electorate, and in fact, as they stand, in agregate (rather than at a very specific demographic group level) are not even close to being a random sample.

Now just how 'raw' were the mid-election night exit poll numbers that showed Kerry ahead? Because it just seems absurd to me for a polling firm to even 'internally' release numbers that have not been reweighted at all. In such a state they wouldn't come close to approaching any kind of random sample and would be very misleading.

Of course, I may have this all wrong. In that case, I'd very much like to know just how exit polls are conducted.

Oh, and Merry Christmas!

[Patsy Stone]

1. Can't say much about the "How"

But I think that if Mr. Mitofsky would have spoken with the Democratic Members of the House Judiciary he might have been able to explain the methodology. I also think that if the media would release the raw data, more than a few pollsters and statisticians could check to make sure that it was done correctly.

I don't know about you, but the fact that anyone who could put this issue to rest isn't doing a damn thing about it, is the issue for me.

:(

Merry Xmas to you, too.

[Carolab]

2. Methodology

http://www.exit-poll.net/election-night/MethodsStatementStateGeneric.pdf (STATE)

http://www.mysterypollster.com/main/2004/12/nep_methodology.html (NATIONAL/REGIONAL)

[platinumman]

3. Sort of right.

Actually, the main purpose for which Mitofsky was to help the networks predict the winners of each state. They didn't want any premature calls again. As far as that goes, they were pretty successful. I think there were only one or two wrong calls by the occasional network. However, as you know, a lot of people who did not know the meaning of these raw figures called it wrongly. As they did not know the sampling procedures, they assumed that a 52% advantage to Kerry should translate into a win for Kerry, whereas of course it depends entirely on which precincts were picked.
The postmortem analysis (main issues, voter concerns, etc) was a secondary aim, but as it was a money spinner it almost certainly had a bearing on the choice of methodology used.

You are right that the population polled was not a pure random sample, although they would have done their best to make it so within their budget and other practical constraints. Their basic methodology is explained on their web page, which you can find by googling Mitofsky. There are two sampling stages. First they have to take a sample of precincts: in other words, they have to decide which geographical area to choose. Because of the nature of the context, it is next to impossible to sample these randomly, and they do not say that they did. Instead they chose precincts according to a certain formula, which has not been divulged.

You are right to be suspicious of the raw figures, because without knowing how the precincts were chosen, it is not possible to ascertain what they really mean. As there are so many tiny precincts in country areas, and huge precincts in urban areas, it is often unavoidable that there is a preponderance of urban areas in the sample. This would probably lead to a over-representation of Democrat voters, which would be repeated in many different states if the same formula was used. As is explained on Mitofsky's exit poll page, this can easily be taken into account when making a prediction.

The second stage of sampling would certainly be a correct random sample of the voters in the precincts chosen. Unless there is interference with the process, there is no reason to believe that this would result in a very accurate estimate of the actual result in those precincts. One does not need a large sample to expect to come to within 0.05% .
However, this would only be an estimate of the actual precincts measured, and as we don't know what precincts were chosen, we cannot yet judge how accurate the polls were. If they turn out to be more than a percentage point out: say they had Kerry at 53%, and the actual was 51.5% or something, I'd tend to think that there is something seriously wrong. (I'm only guessing here).

Anyway, we really can't do much but wait until we have more information regarding the first, non-random geographical sampling. Until then, any calculations are premature, and only serve to dilute the impact when it does come.

[anamandujano]

4. Whatever the methodology, you've got to wonder why

they've always worked until 2000 and they're used in other countries to make sure the vote counters are on the level.

[KittyWampus]

5. 10% Black Adjusted To 5% Still Doesn't Effect The Fact MORE PEOPLE

voted for Kerry and they changed that.

[davidgmills]

6. Don't Buy the post-mortem argument.

Networks don't pay the kind of money they pay for some postmortem analysis. The public just doesn't care that much about the postmortem balderdash.

They pay to get the election called as early as possible.