Polls and Regression to the Mean

[Jackpine Radical]

Any time you see a poll with extreme results (in comparison to other polls), and get excited about it, you are probably getting excited about sampling error.

For example, suppose most polls are showing a 5-point gap between the candidates; say, 51-46. Let us suppose that this is the actual population value, the "true proportion." Now suppose a given poll, just by error, gets a proportion of 53 for candidate O and 44 for candidate M: a 9 point spread. The next day, their most likely result is 51-46--but if their error is just as bad as the day before and in the opposite direction, they might get 49-48. So the two errors, one in each direction, add up to an apparent 8-point closing, while all along, the "true proportion" remains at 51-46, and the apparent closing is entirely due to error. This is one reason why a "poll of polls" will tend to be more accurate: Given enough polls, the errors will tend to average out and the results will be far more stable than any single poll. In effect, you're adding the sample sizes of all the different polls together and reducing the margin of error.

What I have written here only addresses the issue of stability. There is another concern, namely, validity: Do the polls measure what they purport to measure, i.e. a random sample of voters? Maybe they're getting biased samples. Maybe they're all contaminated by the same bias if they're structuring their samples on the same assumptions (maybe undersampling young voters or blacks because they haven't come out in large numbers in the past, but this year is different, or whatever). Short of holding an election, there is no way to know.

[patrice]

1. I used to teach this stuff because I taught AP highschool Psychology.

Because Corporate Media has as much influence as it does, Validity & Reliability and issues like regression from the mean are things that EVERY American should be required to at least be familiar with. It should be part of public education graduation requirements. Students should be able to construct examples and then WRITE a short answer about the significance to their daily lives.

[amborin]

2. yep, the 'mean of the means' will in theory approach the true pop mean

but, as you finally said: the only poll that matters is on election day

[Jackpine Radical]

3. The problem of course is that the population in question

may not be the one they think it is. They assume that they're providing an unbiased sample of the population of voters, but that may not be the case.