How good are the public polls? (Feingold is not a fan)
By Craig Gilbert of the Journal Sentinel
Oct. 18, 2010 1:49 p.m. |(30) Comments
Before introducing first lady Michelle Obama at his Milwaukee fundraiser last week, Russ Feingold told supporters that he had closed the gap with opponent Ron Johnson in his own internal polling.
“I’m no longer behind!” he roared.
Feingold obviously wants Democrats to be energized, not discouraged, about his race. But it’s a hard sell when you’ve trailed in every nonpartisan public poll since late July. Feingold pollster Paul Maslin argues that the public polls are flawed, that they don’t accurately reflect who is likely to vote next month, and that the race is now neck-and-neck.
Could he be right?
Only if all the public polling is wrong. So you have to be skeptical.
But let’s leave aside the horse race question for a moment, because Maslin’s argument involves two broader questions that are getting a lot of discussion these days.
One is what the makeup of the electorate will be Nov. 2 -- whether or not it will be markedly more conservative and Republican than in the recent past.
Two is the quality of public polling in statewide elections like this one.
We’re swimming in polls – partisan polls, media polls, academic polls, one-day robo-polls, big polls, little polls.
I would divide expert opinion on this subject into two camps.
One camp believes that state-level polls are frequently flawed, done on the cheap, lacking in transparency (about how the survey is conducted) and don’t deserve much credence. The greatest skepticism is often directed at “robo-polls,” automated surveys such as Rasmussen that don’t use live interviewers, don’t call cell phones (which by law must be dialed manually) and are often done over the course of a single night, which means they only reach a small percentage of the people called and have to be statistically adjusted to compensate for the types of people they systematically miss. Well-known election analyst Charlie Cook recently wrote that “90 percent of the public polling in statewide and district races is mediocre at best, and much of it is very close to worthless.” These are the polls that drive public perceptions of the race, he complained, calling most local media and academic polling “dime store junk” and saying he found far more value in the private polls done at much greater expense by campaigns, parties and interest groups.
The other camp believes the situation isn’t nearly so dire -- that while polls vary a lot in quality and method, when you consider them in their totality, they give you a pretty reliable picture of the race and are highly predictive of election outcomes. This is the case for the polling averages you see on RealClear Politics and Pollster.
Tom Holbrook, political science chair at UW-Milwaukee, makes the argument for how well polls have performed historically on his Politics by the Numbers blog.
Experts in both camps generally agree on some key points for consumers of polls: that you should be wary of surveys with small samples (400 likely voters instead of 800), of one-day polls, and of polls that don’t tell you how they were conducted or don’t give you demographic breakdowns of the people they surveyed.
Experts also agree that you should not put too much stock in any one poll (I’d say this is easily Rule Number One for journalists and voters). Don’t assume that one new poll constitutes a trend (even good polls can be outliers) or is even comparable to previous polls. Be cautious about all polling results unless lots of different polls are saying the same thing.
So how does the polling in the Feingold-Johnson Senate race look through this prism?
About half the non-partisan polls this fall in the contest have been robo-polls. Many have been one-day polls. Hardly any have included cell phones. Last month's Marist poll is the one exception I'm aware of. Cell-phone-only users are now about 25% of the population and they obviously skew young. A recent Pew study suggested a growing bias against Democrats in polls that leave cell-phones out.
Does that mean that we should be ignoring these polls?
Not if you believe in the power of polling averages, which currently put Johnson ahead by high single digits.
“There is no evidence from the last few election cycles that polls are anything but accurate. Sure, they miss in a few cases but usually just when the polls themselves say it is a close race,” says Holbrook. “The simple fact is that in the last two election cycles virtually every Senate and gubernatorial candidate who maintained a lead in the polls throughout the fall campaign went on to win.”
Polling expert Charles Franklin of UW-Madison says that even assuming that Rasmussen skews Republican and that the failure across polls to call cell phones understates support for Democrats, that doesn’t explain away most of Johnson’s lead.
“I’m certainly open to saying there are some concerns about the polling,” says Franklin. “The cell phone issue is one of the biggest. There’s the uncertainty of the composition of the electorate which could be moving more Democratic if the enthusiasm gap is waning a bit. But I’d be hard-pressed to think of any legitimate reason to move that average more than a couple of points.”
Over at the New York Times, statistician Nate Silver is confident enough in how well polling trends have performed in predicting past election outcomes that he puts the probability of a Johnson victory at 94% (!), even though the official Times rating of the race remains a toss-up. Here are the two rather discordant assessments side by side on the Times web site:
for full article:
http://www.jsonline.com/blogs/news/105194354.html
Printer-friendly format
Email this thread to a friend
Bookmark this thread
(1000+ posts)
Send PM |
Profile |
Ignore
