Obama's "Suburban Women" Problem? Experts Doubt It. MOE=9+ / Gap=6 pts(44-38). Polling Stat=USELESS

[Pirate Smile]

Is NBC Right About Obama's "Suburban Women" Problem? Experts Doubt It

UPDATED BELOW: MSNBC tells The Huffington Post its "suburban women" results fall "within the margin of error."

***
Patience is not typically regarded as a virtue in political reporting. Next to accuracy, being first is king. But sometimes the two goals are in tension, as shown by this week's rush to make sense of Sen. Clinton's exit from the presidential field. Over the last few days, pollsters have been racing to gauge the first reactions among women voters to the all-male general election match-up between John McCain and Barack Obama.

Gallup hit the news cycle first on Wednesday, announcing their discovery of an eight-point swing among women toward the Illinois Senator since Clinton left the stage. Then NBC and the Wall Street Journal came out with joint polling results that were broadly similar, but with a caveat that represented a potentially troublesome hole in Obama's female support.

The NBC-WSJ poll showed that while Obama had increased his lead among women overall (52-33), "suburban women" still favored McCain by six points, 44-38, while a hypothetical Clinton candidacy would beat the Arizona Republican.

Since that poll was released Wednesday, MSNBC has been reporting the "suburban women" finding often, sometimes hour-by-hour -- perhaps because it reinforces residual doubts about Obama's viability in the 'burbs. The question led Chris Matthews' "Hardball" program at 5pm on Wednesday with a graphic that read "Woman Trouble?"

But how solid was the NBC-WSJ poll's conclusion about those voters? With only 1,000 total respondents in the poll, and no guarantee that the sub-group of "suburban women" was balanced nationally -- meaning that these suburban women polled were drawn from a balanced cross section from America's vast suburbia -- a group of polling experts from across the ideological spectrum told The Huffington Post they viewed the findings with some suspicion.

"I am skeptical about results for smaller subgroups like 'suburban white women,'" said Emory University Professor of Political Science Alan Abramowitz. "There is more random 'noise' with smaller subgroups. How many of these 'suburban white women' were there in the NBC poll out of the 1,000 total registered voters? Figure about 750-800 whites, close to 400 white women, so maybe 150-200 suburban white women. A shift of a small number of voters would change the outcome."

Republican pollster Tony Fabrizio, who you might expect to celebrate the poll, also cast doubt on its findings regarding a preference among suburban women for McCain -- if for no other reason than that any one poll's margin of error increases as the reduced sample size of a "subgroup" becomes smaller and smaller. "If you have a thousand samples, maybe your margin of error for your overall sample is 3.1 percent (the margin cited in the NBC-WSJ poll)." Assuming approximately half of the poll's respondents were women, he said, "your margin of error goes up to 4.5 percent. Then take females and segment them among rural, urban and suburban (subgroups). ... You've doubled your margin of error in that group." (See update below: MSNBC revealed the margin of error tripled for its "suburban women" subgroup.) Fabrizio also said that disrupting the national distribution of a sample by looking at subgroups can throw any analysis out of whack by the same proportion, adding, "there are other vagaries that can go on to influence the sub-sample, too."

Democratic pollster Anna Greenberg agreed with Fabrizio, saying "I'm not sure I believe" the NBC-WSJ numbers on suburban women. She also said it was "bizarre" to single out the suburban women numbers as a statistically significant finding, given the more robust evidence that Obama was competitive with McCain among white women overall. "They missed the big story among women, in my view," she said. "It's not like white women are a 'gimme' for Democrats. Obama is doing significantly better among college-educated white women (than Sen. John Kerry in 2004)."

Republican polling magnate Frank Luntz also predicted the real breakdown among women voters would prove to be age, not location. "Obama will benefit from the usual Democratic advantage among women -- with one caveat," he said. "He is not as likely to win over older women compared to the traditional Democratic presidential candidate. His age, his inexperience and, frankly, his apparent association with people outside the mainstream will definitely frighten older women who would normally support the Democratic nominee. Even the crowds at his rallies raise questions. The aspects of his life and his language that is so appealing to younger women simply does not generate a similar reaction among older women. However, he will win an unusually large share of the younger female vote -- a very potent voting bloc in this election."

In the published 33-page breakdown of the NBC-WSJ poll's results linked to on Pollster.com, the following phrase appears on the front page: "NOTE: The results contained in this document reflect results among the national crossection of voters ONLY." Given that the "suburban women" findings are not included in that document, it appears the Hart and Newhouse polling firms, which conducted the poll for the two media organizations, recognized that its subgroup findings on "suburban women" were not representative enough to include in the breakdown. So far, that hasn't stopped MSNBC from turning those numbers into big news. Emails to NBC's political unit asking for a numeric breakdown of "suburban women" in their poll were not immediately returned.

UPDATE:

MSNBC has now provided The Huffington Post with more information on its "suburban women" finding showing a 44-38 McCain lead over Obama. "This is within the margin of error of 9.34 percent based on a sample size of 110 within the larger poll," an MSNBC source wrote over email. (That's three times the margin of error for the entire poll.) This means McCain's 44 percent figure of support among suburban women could actually be as low as 35 percent, while Obama's 38 percent figure could rise as high as 47 percent -- assuming a 95 percent confidence interval (for the stat wonks in the house). Alternatively, McCain could be leading Obama 53-29. While those distant outcomes are less likely true than NBC-WSJ's 44-38 finding, that broad variance raises questions about the statistical usefulness of this one particular crosstab, as opposed to the rest of the NBC-WSJ poll on the whole.*


SECOND UPDATE:

NBC Political Director Chuck Todd defends the use of the "suburban women" crosstab in an email response:

Here's what I can tell you on our crosstabs. We never use one unless we have more than 100 incidents; in this case, the incident rate is approx. 140 interviews. It's always easy to shoot a pollster but the track record of the NBC-WSJ poll is undeniable; campaign operatives on both sides of the aisle believe it's the gold standard for media polls.

http://www.huffingtonpost.com/2008/06/12/is-nbc-right-about-obamas_n_106806.html

[Patsy Stone]

1. Unless they're scared to death about terrorists attacking the mall

or the schools, or they were born and will die a Republican, McCain has nothing to offer suburban women; and in a few weeks, after the primaries are a memory, I think a lot of them will be firmly in the Democrats' corner. I would imagine most are more against their kids going to fight in Iraq than anything else. Next, I'd say it's gas prices, the environment, and the cost of food.

In fact, I could see more women staying home if they were ambivolent than turning out specifically to vote for McCain.

[Pirate Smile]

2. The margin of error on that stat is larger then the gap. If the MOE is over 9 pts and the

poll shows a 6 point gap between McCain and Obama then that stat is absolutely meaningless yet they are pimping it hourly.

Annoying.

[Patsy Stone]

7. They have a lot of time to fill

and nothing with which to fill it. If only they'd use their power for good instead of evil.

[K Gardner]

3. Thank YOU for posting this ! It is really frightening what they will do to sell a story.

[Blue-Jay]

4. Selective poll-quoting makes for more interesting "news".

Everyone knows that so-called news organizations are more interested in promoting ratings than they are in reporting actual news.

I wonder how the polls break down in terms of "little people". Who are Small Americans backing for POTUS in November?

[Maureen1322]

5. White, suburban woman here, and I have no problem.

Go Obama!!

[lukasahero]

8. No sh*t

Aren't you tired of hearing all about who you are and what you think on the news these days? I can't begin to tell you how tired I am of being a f*cking demographic but it's so much worse when they get it freakin' wrong.

Signed, a suburban white woman who voted for Obama in her primary and doesn't know a single Republican suburban white woman...

[ayeshahaqqiqa]

6. Ah HAH!

This is EXACTLY what I expected when I read about this.

[creeksneakers2]

9. My response on NBC blog Daily Nightly

Why is it that NBC Nightly News reported that white male voters are a problem for Obama but didn't report the rest of NBC's poll that shows McCain having problems with (A)women 52/33; (B)Hispanics 62/28; (C)African Americans 83/7; (D)Catholics 47/40; (E)independents 41/36; (F)blue collar workers 47/42 G)white women 46/39?

That means McCain is having trouble with everybody but white, protestant, white collar, Republican males. Shouldn't your newscast have been about McCain's demographic troubles?

Your report also singled out white suburban females as a group Obama is having trouble with. He's down 44/38. That's only six points. Your report characterized the Obama's lead in the overall poll of all voters, six points, as a "very tight race." You characterized the six point lead among the suburban white women group that only makes up 10% of the electorate as a "problem" for Obama. What's more, the subgroup still has 18% undecided for Obama to work with.

There are two obvious conclusions. From those numbers, your report shows NBC is still openly supporting McCain and failing its responsibility to provide fair and balanced coverage. And, NBC feels the votes of establishment white folks matter more than anybody else's, particularly when those voters are male.

[Pirate Smile]

10. Nice.

:toast:

[Pirate Smile]

11. kick

[Bill McBlueState]

12. Excellent analysis

While this is specific to the recent reports about "suburban white women," it generalizes really well to show why these goofy demographic breakdowns aren't as useful as Norah O'Donnell & co. want us to think they are.

As for Chuck Todd's response, it's great that they have some lower bound, but 100 incidents carries a margin of error of 10%. If they're reporting on data with a 10% MOE, they need to make that clear to their viewers.