Here is why 538's odds on Hillary are decreasing while The Upshot is increasing.

[Loki Liesmith]

Last edited Tue Sep 20, 2016, 08:13 PM - Edit history (1)

I knew this, but had put it out of mind. 538 aims to be a predictive model and in doing so it projects trends in horse race numbers forward in time. Today's polls by Monmouth and St. Leo in Florida both had larger leads for HRC way back in August. So todays lead for her is indicatrive of downward trend in her numbers. So her numbers, coupled with weirdo Reuters semi-polls, actually got WORSE.

Frankly, I think this methodology makes no sense. If you do believe in statistical momentum (and I'm not sure I do), incorporate it into your model using the agggregate horse race numbers, not individual polls. If you fit the trendlines to individual polls, you effectively penalize that poll for having drawn an outlier in the past. I can't see a rationale for that at all.

Fitting data in this manner will cause your model to jump around haphazardly, based on the coupling coefficients to each polls trendline, because the polls come at stochastic time. Using an aggregate trend over all polls makes more sense if you value consistency in a model.

Nate Cohn, at NYTimes Upshow seems to have noticed this as well, and I don't think he agrees with Silver based on his twitter timeline.

[OhZone]

1. Predictwise has her back up. :) nt

[LAS14]

2. As far as I can see 538 hasn't incorporated St. Leo yet. At least....

.... for Florida.

[Loki Liesmith]

4. Nate said he had on twitter

Maybe not published yet

[Godhumor]

6. He did, +6 C adjusted to a +1 C. So it was a "bad" poll for Clinton

So, you know...

[Loki Liesmith]

7. Wow a 4 point deduction for trend

stupid.

[Godhumor]

8. I was composing a rather lengthy post on 538 when I saw your post

Last edited Tue Sep 20, 2016, 09:05 PM - Edit history (1)

Saved me a little time. Though, I was going to go into a bit more detail on how, on the past couple days, 538 had uploaded polls from February (!) for PA that adjusted to a Trump +9. Their weight is relatively small being so old, but it was enough to shift overall odds by a point yesterday.

Really, Nate purposely designed a noisy model for the ability to generate clicks and headlines (And make ESPN happy, of course). The whole idea of smoothing noise seems to be missing from his models this year, but he's rebranded it as "More responsive to fresh data".

I much prefer Nate Cohn's work this cycle. Not only is he actually running forecasts, but he's talking stabs at polling "right" and demonstrating how data is open to interpretation. 538 is not leading the way on the predictive side, anymore.

Nice to have another analyst on the board, by the way.

[LAS14]

10. Where do we find Nate Cohn?

[Godhumor]

11. The Upshot at NYT n/t

[Loki Liesmith]

12. Please post your thoughts too

I didn't notice that February poll. Silver's weights just never zero out. I have a problem with that too.
He must need a spark cluster to run this thing it's got so much data going through it. Parsimony?

And thanks for the shout out. There's a couple of us data wonks here. I have my own simple election model that was more or less tracking the Upshot. But for some reason it won't download any Florida polls from the HuffPost pollster api. I have no idea why it's just Florida but its annoyed me enough I stopped using it until I fix it.

[Godhumor]

13. St Leo adjusted to a +1 not a +2

So a 5 point correction.

Silver is experimenting with a model this year, which I don't mind. He's just not being upfront with the fact that it is an experiment and that it has some things that could be done differently. He's also repeatedly bristled at Sam Wang poking his methodology on Twitter.

[Loki Liesmith]

16. He hates Wang.

To be fair, Wang needles him a bit too much sometimes.

I'm a former academic in the same field as Wang, and I'm used to the kind of assholery academics use on each other. I don't think Silver is at all.

[Loki Liesmith]

14. speaking of 538 clickbait?

Is there any indication that 538 makes any money for ESPN? I can't see that the network's target demographic would really give a tinkers damn about it.

[Bradical79]

17. Sports analytics is a big thing for them

That's the main reason they have 538.

[jimw81]

25. Rumor has it 538 has budget problems

[unc70]

22. I had noticed the update of Feb data

I was perplexed when I noticed those poll updates with February dates. I had been looking at the national and certain state polls and probabilities trying to understand the current 538 models.

I am fairly much a quant wonk in these areas. Lots of weird stuff in the polling world these days. Was also true four years ago. I posted a bunch about major flaws with how one company then controlled the phone numbers that provided the samples for 90% of polling.

[LAS14]

9. Where should I be seeing St. Leo?

[Godhumor]

15. Under updates. It is Sept 10-16. Just past some Reuters stuff

You might have to clear cookies to get it to load, if you don't see it.

[LAS14]

18. Yes, I see it in Updates, but...

....still don't see it in this list. Thanks for the info.

[Godhumor]

19. Click "show more polls". It is there n/t

[LAS14]

20. Thanks!!!

[SCliberal91294]

3. Trumps polls

From last week still have a big influence. As new polls come out, those favorable polls for him will be phased out.

[Farmgirl1961]

5. Thanks for that information

It helps put things in perspective but I won't feel better til I see his predictive model ticking steadily upward for Hillary.

[LAS14]

21. Do we have a comparison between 538 and Upshot...

... for past presidential elections?

[Loki Liesmith]

23. fraid not

Upshot is an entirely new model. But then again, so is 538. The model has been retooled quite a bit.

The only model that is largely unchanged from 2012 is Sam Wang's at the Princeton Election Consortium. And he may have made changes under the hood that I haven't followed...

[Loki Liesmith]

24. Here is some background on Nate Cohn

http://www.politico.com/media/story/2015/01/meet-nate-cohn-new-york-times-new-young-gun-on-data-003282

[pangaia]

26. I have no idea what you are talking about.

[Farmgirl1961]

27. Thanks all for this thread

It is helping to calm me down a bit. But when I look at the tide of swing states that are either in play or are now trending Trump that gives me the heebie jeebies. For all of you policy wonks out there, thoughts on when we might see this settling down for Trump...in other words more favorable trends for Hillary?

[Loki Liesmith]

28. bumping

for math

[BzaDem]

29. Why is it bad to look at the trend?

Let's say a pollster had a large "house effect" (meaning that for whatever methodological reason, their result is more favorable to one candidate than the polling average).

Wouldn't you want to look at the trend? For example, let's say there was a Trump+10 pollster, that kept finding Trump+10. If we just incorporated the poll into the average without looking at the trend, wouldn't the average be mostly a function of whenever this pollster released their polls?

[Loki Liesmith]

30. over-correction

Nate already subtracts off the house effect. To do it again via trend is correcting for the same effect twice.