Debunking CNN's (weighted) poll results (This is geting dirty, Folks!)

[displacedtexan]

I've just posted my analysis of CNN's "weighted poll" results on my blog...

This CNN Inside Politics poll report is bogus!

Here's a snip from my analysis:

Today, CNN is reporting the results of their most recent presidential poll:

Among likely voters (trans. White suburban population):

Kerry 50% Bush 46% (MOE +/- 4%)

Looks like a dead heat.
Bad news for Bush, though. This is his strongest demographic!

But wait.

The survey of likely voters weighted the sample to assume a turnout of 50 percent, consistent with recent presidential elections.

In the current poll, a wider sample of 891 registered voters showed a slightly larger apparent lead for Kerry, but one that was still within the margin of error.

Fifty-one percent said they would vote for Kerry-Edwards, and 44 percent supported Bush-Cheney. LINK

Still within the MOE? 51% - 44%?

What a crock!

From The National Council On Public Polls:

Bad Weighting:
The most common bad weighting in political polls is weighting just the likely voters for the number of Democrats, Republicans and others. Party identification is correlated with voting, but what is missing are meaningful numbers of party members for the whole population. Without that, the weighting is a guessing game rather than good theory. Some pollsters use the numbers from an exit poll from a past election, but the numbers of people who consider themselves members of a party changes from month to month and year to year.

Much more (including links to NCPP, CNN Feedback, and the original CNN story)...

http://tinyurl.com/68qtr

[Nicholas D Wolfwood]

1. Whenever you see "Likely voters"

Stop paying attention. It immediately adds subjectivity to the poll, and that's when it stops being scientific.

[iconoclastic cat]

2. Wait, I heard the opposite.

I heard that the "likely" voter index was the less subjective type. Please explain.

[calico1]

3. I'd like to know too.

Its confusing. I thought "likely voter" means just that--that you are most likely to vote, so it presumes you are already registered, whereas "registered" means you are registered but might not vote. A lot of people are registered to vote, doesn't mean they will.

[Nicholas D Wolfwood]

4. But what does it mean to be "likely"?

Sometimes it means you voted in 2002. Sometimes it means you voted in the last 2 or 3 elections. Not only is it subjective in its criteria for inclusion, but it arbitrarily throws out people that could potentially vote this time around that did not in the past for whatever reason. Just because someone didn't vote before, that does not mean that they won't again, and not taking that into consideration radically skews the polls, especially since it is a known political law that people with an axe to grind are far more likely to go to the ballot box than those who do not. The people who are angry today are not the same people who were angry in the past elections. So ignoring their potential vote is flat out folly.

This is NOT the case for registered voters versus non-registered. A non-registered voter will not vote in the upcoming election because they are not eligible.

[iconoclastic cat]

9. gotcha---thanks!

[calico1]

17. Oh, Ok. I think I get it now

I was thinking that when they say "likely voters" they are actually asking people if they intend to vote! LOL. I think that's where my confusion was. What they are really counting as "likely" is someone who voted the last few elections and therefore they are assuming they will vote in this election, right? I understand now. This is definitely not a good measure. Thanks for the explanation.

[Nicholas D Wolfwood]

5. See post #4

[displacedtexan]

7. Likely voters = middle-class, suburban whites

Registered voters have committed to voting in the upcoming election.

Likely voters aren't yet commited... they just "support."

BTW, My husband is a demographer (doctorate-holding) in DC with The Mlton S. Eisenhower (Ike's liberal brother) Foundation.

[iconoclastic cat]

11. Dp'dTex: Any good "stats for idiots" resources?

I'm really curious about this sort of thing.

[displacedtexan]

13. I'll ask my husband.

Warning, though... he LOVES STATISTICS.

That link on my blog (Public Polling) will get you started.

[iconoclastic cat]

16. Cool! just message me if you have a good suggestion.

I dig stats, too.

[displacedtexan]

14. Likely voters aren't reliable respondents.

If you polled all likely dem voters or all likely repuke voters, you'd get better results. Same with all young or all old.

[hlthe2b]

8. Actually many leading pollsters (including one who guested

on TTM for Josh Micah Marshall a few weeks ago argue that use of likely voters is too volatile until just prior to the election and that registered voter trends from identical polls are most meaningful at this (still) early stage. Certainly a poll of activist Dems or Repugs would be meaningful when limited to likely voters, (since we all basically are), but the general public is too far out from the election to define a clear intent on plans to vote. Here some comments on the matter from Ruy Teixeira
(June 24, 2004 -- 06:17 PM EDT

There's been considerable confusion about which trial heat results to pay the most attention to at this point in the race. Here's my take, which should help clarify why I choose to focus on certain results over others.

One issue is likely voters (LVs) vs. registered voters (RVs). At this point, most polls are surveying only RVs and I believe that's appropriate and, in fact, preferable. It is way too early to put much faith in likely voter screens/models as representing very accurately the voters who will actually show up on election day. There is reasonable evidence that careful likely voter methodologies work well close to the election and do fairly accurately capture that pool of voters. But there is no such evidence for LV samples drawn this far out.

Indeed, my understanding is that Gallup does LVs this early not so much because they believe they are capturing election day voters this early, but more so that they can avoid having to explain sudden shifts in the horse race question as LV data replaces RV data in the fall (the traditional time to switch from RVs to LVs). There have apparently been some problems with this in the past, so reporting both from the very beginning of the campaign eliminates any potential embarrassments along these lines. But that doesn't mean the LV data is any better at this point in time--it merely means they're providing it.

In fact, since the sample size for LVs is smaller and since the composition of the LV sample will shift depending on how political developments are affecting interest and intensity levels among different groups of voters, additional volatility is built into the LV samples that is not there with the RV samples.

And then there are the comparability problems. LV samples are difficult even to compare to one another, since methodologies differ, and clearly can't be compared very well to RV samples, which are the bulk of polls at this time. That's another strike against paying much attention to LV results this early.
more:http://www.talkingpointsmemo.com/archives/week_2004_06_20.php

[Beetwasher]

6. Here's What Gallup Says About "Likely Voters"

http://www.gallup.com/content/login.aspx?ci=12052

Now, I could be wrong, but I don't seem to remember them splitting polls up into these categories until relatively recently. It seems to me that in past elections, the poll numbers would be based on the total sample.

IMO, the "likely voter" category is merely a way to manipulate the data. It also seems to me that the pollsters and reports on the polls emphasized the total numbers when they favored Bush and then when he started sliding, they started reporting the "likely voters" category, which always seem to be better for Bush. I could be wrong, but that's certainly the impression I have.

[sandnsea]

10. Whatever favors Bush

That's what the teevee reports. It's really that simple and I've noticed it for several months now.

[displacedtexan]

12. You're absolutely correct!

Since most people only read the headline... and a few read the first 3 paragraphs of news stories, CNN is banking on getting away with weighting and polling "likely" voters.

What a crock!

[Beetwasher]

15. They Keep Narrowing The Sample Size To Manipulate the Outcome

It works like this:

You start w/ a general, supposedly random sample of let's say 1,000 adults of voting age and you poll them and crunch the data and get results.

Then you pull out all the people who are not yet registered voters your left w/ a smaller sample of let's say 700 people. You crunch that data and get slightly different results.

Then you pull out all the people who, by some (questionable?) formula you define as not likely to vote and you're left w/ even a smaller sample of let's say 500 people. You crunch that data and get slightly different results again.

This is data manipulation. Keep in mind, the smaller the sample, the less accurate your results, no matter how you define the samples. The way you define "likely voter" can be VERY subjective and obviously, the way Gallup does it, it narrows the sample so that there are definitely more Bush supporters in that sample.