Exit Polls can be just as inaccurate as regular polls

[qdouble]

I've been seeing a ton of people in the media and on the web talk about how they can't believe that Trump got more support among certain groups of people as if exit polls are somehow 100% accurate, when they are not. They are based on samples, just like regular polls. Exit polls quite often over or under estimate certain groups and have to be adjusted as far as how they are weighted after all of the results are fully analyzed.

Given how people voted in several different ways this year, exit polling may be even more inaccurate than normal.

So claims about Trump getting more support among people of color may be completely false.

[mr_lebowski]

1. Specifically, the issue comes from not achieving true randomness in your sampling ...

If you attain randomness, and have a sufficient sample size, polls of this type are EXTREMELY reliable ... statistics are very well understood, and they are actually damn-near infallible mathematical rules.

It's just ... the randomness that's hard to do.

[qdouble]

2. From a statistical standpoint

Accuracy increases with the amount of samples in terms of the probability of correctness, however being wrong never becomes impossible unless you sample everybody...it just becomes less and less probable.

For the most part, when you are estimating what 150 million Americans did based on a sample of 15,000...you have to make certain assumptions. If those assumptions are wrong, it will influence the accuracy.

[mr_lebowski]

3. Depends on how one defines 'being wrong' ...

The inaccuracy you refer to is accounted for by confidence intervals. The statistics essentially say "I'm 95% confident that the true (i.e. total population) value is X".

And if you CORRECTLY sampled completely randomly ... and applied the correct formula (z-score vs. t-score, etc) ... that 95% confidence interval will basically always be correct ... because you're only saying you're 95% sure

Of course your sample size can't be too tiny. But beyond a certain threshold (which is actually pretty low if the population has a normal distribution of values), having more samples just (generally) decreases the width of your confidence interval

[musicblind]

6. When it comes to traditional polls, it's getting harder and harder to get a truly random sample.

The sample self-selects in one way or another. Certain people answer phones, and certain people don't. A lot of online polling is opt-in first, and then those sampled are randomly chosen from the opt-in pool.

But, especially when it comes to political polling, people lie. Unlike most posters, I don't think people systemically lie on purpose. I think people believe the lie in the moment, but it still ends up being untrue.

An example:

Polling used to show 1.8% of the population was gay. As a gay man who worked extensively within the gay community, I knew that was B.S. — Then, polls showed 3% — Then, 4.5% — Now there are polls showing 8% to 15% depending on the age group.

People didn't become more gay; people became more willing to admit it—both to pollsters and to themselves.

[mr_lebowski]

7. Agree on all counts ...

And the reason that exit polls were considered the best available poll is that you're polling people who actually voted, not just people who either said they were (or were otherwise determined to be) 'likely'. You remove the variable of 'enthusiasm' as well as possible biases like 'people who say they're voting GOP are inherently more likely to follow thru with casting a ballot'.

A lot of issues arise when trying to apply statistical principles to things that are not, in some sense, a hard/physical measurement ... rings of trees, heights of 10 year old boys, etc. When you deal in hard data, and you sample randomly in a significant enough way, you'll never be 'wrong', it's not mathematically possible.

But with polling ... there's difficulties with true randomness, as well as with enthusiasm, lies (a particular problem if only one 'side' is apt to lie), people changing their minds, etc. These can be adjusted for, according to certain known principles, but it'll never be quite the same as measuring hard values.

And readers/people interpreting polls need to understand that. It becomes a lot more common to see a population where the value is out on the outer edge of the sample's confidence interval, esp. if you only use 95% instead of 99%.

[applegrove]

4. Malcolm Nance said that he doesn't see how exit polls, without early voter

numbers on race, can be accurate this time around because so many voted early or absentee. How do you exit poll that? He was on Bill Maher.

[qdouble]

5. The exit polls are clearly off.

How did Biden gain a higher vote percentage in urban areas while simultaneously getting less Black and Hispanic vote? It’s nonsense.

[mr_lebowski]

8. When you have multiple methods of voting you inherently undermine exit polls

But more so in this election than any other before it, because of the virus and the different messaging of the two parties (well, Trump).

The way they're undermined is: it's no longer possible to create a truly random sample of the population using only exit polls.

Totally agree with Nance.

EXCEPT I'd bet that in states that still require ONLY in-person voting ... and have strict rules where absentee ballots are quite rare ... exit polls still hold up quite well.