Stats nerds: two competing interpretations of polling data

[WillyBrandt]

I've got an honest question. I think both Sam Wang of Princeton Electoral Consortium and Nate Silver of 538 are righteous dudes who try to model elections as best they can and reports what their systems pop out.

And yet there is such a cleavage between them. Prof. Wang says Hillary is nearly a lock, and Silver says the odds are above 1-in-3 that Trump takes the White House. And each has an explanation:

1. Wang says that the race is remarkably stable. When you take the diff between HRC and DJT over time, it is in a narrow range (2-7%), with narrow volatility, over a protracted time. Overlay an electoral that is polarized and thus less likely to switch, you've got a recipe for a kind of stasis -- one that all the melodrama of cable news basically papers over. Makes sense!

2. Silver interprets the world very differently. He doesn't doubt that HRC is ahead, but sees two things that introduce a big bolus of uncertainty that gives DJT an big lump of probability. First, there are a lot of undecided: whither shall they fall? And second, he believes that states are correlates, so if state A's outcome deviates from polls, so a similar state B; this means mis-polling has a cascading, systemic effect that his probability estimate must capture. Kind of makes sense too.

How do we reconcile these two interpretations of the selfsame data? I'm torn, but lean towards Wang -- frankly, it's the simpler, more parsimonious interpretation of the data and that has an appeal. But I do like Silver's injunction against being confused by statistics or misled by a set of spurious normal distributions: sometimes the correlations go to 1 at extreme events.

Thoughts?

[napi21]

2. I just read a DU post that talked about how unreliable polling info is.

Many reasons. Cell phones exchanges don't necessarily mean that it is the area you are polling, most people simply won't answer their cell. That being the case, I'm more inclined to believe Wang. I saw him in an interview the other day and I'm pretty sure his pollsters ask specific questions to identify the persons location.

[DeadLetterOffice]

12. This is my issue, right here.

I do not trust the methodologies of the polls themselves, so why in the name of all that is beautiful would I trust the polls in aggregate, or deductions based on those polls?

Silver isn't wrong - weird shit certainly does happen - but in this case I think we all should collectively stop looking at polls, GOTV if we're able, and stop obsessing about data that's iffy to begin with.

[Barack_America]

6. Closer to an election, it seems that more local polls with dubious methodology...

...start rolling in.

That was certainly the case before the MI primary, and we all know how that turned out for pollsters.

I'm going with team Wang, because I think Silver's algorithm is being unduly influenced by a lot of crap.

[geek tragedy]

8. Silver doesn't trust the polls, so what he's really saying is he doesn't know

So his entire stance is that his model provides little insight.

[Lucinda]

9. I do not think that there is a huge horde of undecided voters waiting to

change the election.

I DO think there may be a good number of quiet Hillary supporters out there that aren't wanting to have to justify their votes....and there may be some for Trump, quiet for the same reason, but definitely not in great numbers.

The part of the electorate that is flexing its muscles, appears to be all those folks standing in hours long lines - and the majority seem to be Hillary voters. Those numbers seem to be quantifiable so I don't see much uncertainty there...

Maybe Nate just was really shaken by mistaking Trump's primary strength...

[Awsi Dooger]

11. I have some quarrels with both of them, but Nate is sharper

I think Wang is wrong to ignore national polls. I can't count how many political wagers I won by simply using the national margin in comparison to the statewide partisan index, while applying a PAN or Partisan Adjustment number toward how the polling typically errs in that state. Sounds complicated but is not.

Nate also adjusts the state polls but he does so based on the polling firm itself and which side/how much it typically errs. I'm surprised he never looks at states that simply are polled poorly period. He throws polls into his model that are frankly garbage, and doesn't seem to notice or care.

Let's face it, this cycle will be talked about for centuries. We've never had a nominee like Trump. Tapes of those three debates will be viewed in amazement by generations who are nowhere close to being born right now. Nate Silver understands the unique nature and therefore potential for surprise. It is a strength, not a weakness. With a bizarre candidate, unusually high number of undecideds, and so many states and demographics prepared to vote differently than what we've come to expect, this is hardly a plug and play election. Nate doesn't brag that he uses polling data back to 1972 for his model. He understands that data back to 1972 is not much of anything. We're still in the absolute infancy of this. Similarly weird cycles aren't included. You can't be sure of anything when the data sample is 10 or 11. Wow.

Anyone who uses 98% or 99% likelihood in essentially a future book event doesn't know what he is doing. But human lifespan realities may shield him/her from proper scrutiny in applying that 99%. Nate's approach to presidential election odds would dominate if all of us lived 500 years or more. We'd remember all those 99ers who lost, and the desperate backtracking rationale attached. Instead, Nate Silver will apparently be viewed as an idiot if a moderate favorite wins by moderate margin. That is almost hilarious.

[WillyBrandt]

13. I think Dr. Wang is smarter

Frankly, he seems to understand math and statistics at a deeper level that Nate, who is sharp himself. It's as though Nate has enough logical ability to reason through the binomial tree, but Dr. Wang has enough true understanding to expunge needless complexity. Who knows.

I do wonder if Nate is making a mistake in trying to extract value from dubious polls. I see his reasoning -- that a bad poll, if you compare it to its preceding version in an apples to apples way gives you inferential evidence about trend-lines. But, again, I wonder whether there isn't wisdom in simplicity: look at high quality polls only and avoid the risk of your correction introducing yet further data and spurious variation.

Lots to muse through...

[Also in a recent interview and post Dr. Wang said 90-99% in his model doesn't mean too much, so he's self-ware, and seems to get his model doesn't adequately account for systemic polling errors.]

[DeminPennswoods]

16. Wang said once over 90, the percent doesn't matter

Once the probability gets over 90, there's really no difference between 90, 95, etc.

[Dem2]

14. Nate's over reacts to changes and then has too much inertia to come back to the norm

That's all you really need to know. Look at the wild swings in his FL plot verses look at Florida on HuffPost that uses just polling averages; it's an eye-opener. One model has her losing the state where the other model has her winning the state relatively easily.