Be Careful of Recent Pennsylvania Polls :The Affect of Phones

[McCamy Taylor]

I am not a statistician or a polling expert, this is just something that occurred to me tonight that I think we ought to discuss, since we talk a lot about polls around here, and spin them around like frisbees. And sometimes get burned doing so. Like in New Hampshire, where all hell broke loose, and a lot of nasty bitter things were said, because the pollsters got it so wrong, and they built up some expectations they should not have.

We are all used to a one size fits all type of poll for everything. We pretty much take for granted that all US households have an equal chance of participating in a telephone survey, but this is not true. In the past, you had to have a land line to be surveyed, and as far as I know this is only beginning to change in select cases that specifically target a youth population.

The public health survey text, Designing and Conducting Health Surveys Lu Ann Aday from the good old days of 1996 discusses the drawbacks of telephone surveys. People were more likely not to have a phone if they lived

1. in the south
2. outside a major metropolitan area
3. single person household
4. 7 or more people in the household
5. minority
6. never married, divorced or separated
7. unemployed
8. less than a high school education

In addition the text says that these same people are more likely to have problems that restrict their activities or cause them to be bedridden, perhaps making them unwilling or unable to participate in other kinds of surveys.

Here are the latest statistics:

http://www.harrisinteractive.com/harris_poll/index.asp?PID=890

Use of cell phones is increasing and traditional landline telephone coverage is decreasing. In fact, one in five adults do not have a landline and only 79 percent currently do. One in seven adults now uses only cell phones. Furthermore, while the use of cell phones among younger segments of the population has been widely reported, the technology is becoming increasingly popular among older populations as well. Remarkably, about half of U.S. adults who only use a cell phone are 30 or over. One-third of 18 to 29 year olds only use a cell phone or the Internet for making phone calls.

Cell phone only users:

* Less likely to be age 40 or older (29% versus 60% of the general population)
* More likely to have at least some college education (60% versus 53% of the general population)
* More likely to be male (57% versus 48% of the general population)
* More likely to have household income less than $15,000 (16% versus 9% of the general population).
* Less likely to have household income of $75,000 or more (28% versus 37% of the general population)

snip

Those who use a cell phone as their only telephone service account for just 14 percent of the total population of US adults, and this proportion is on the rise.

There is a demographic chart.

Here is a sample poll:

New York Sen. Hillary Clinton has stalled Illinois Sen. Barack Obama's drive in the Pennsylvania Democratic primary and holds a 50 - 44 percent lead among likely primary voters, according to a Quinnipiac University poll released today, unchanged from April 8 results.

There was no noticeable change in the matchup in polling April 12 - 13, following widespread media reports on Sen. Obama's 'bitter' comments.

In this latest survey of 2,103 likely Democratic primary voters by the independent Quinnipiac (KWIN uh-pe-ack) University, 26 percent of Clinton supporters would switch to Arizona Sen. John McCain, the Republican, in November if Obama were the Democratic nominee. Nineteen percent of Obama backers would switch to McCain if Clinton were the Democratic nominee. A look at other groups shows:

* White voters for Clinton 57 - 37 percent, compared to 56 - 38 percent last week;
* Black voters back Obama 86 - 8 percent, compared to 75 - 17 percent;
* Women back Clinton 54 - 40 percent, unchanged from 54 - 41 percent last week;
* Men are for Obama 51 - 43 percent, compared to a 48 - 44 percent tie last week;
* Reagan Democrats back Clinton 55 - 40 percent;
* Voters under 45 go with Obama 55 - 39, while older voters back Clinton 55 - 40 percent.

"Sen. Hillary Clinton is fighting off Sen. Barack Obama's drive to make it a close race in the Pennsylvania Democratic primary, holding the six-point edge she had a week ago. She seems to have halted the erosion of whites and white women in particular from her campaign," said Clay F. Richards, assistant director of the Quinnipiac University Polling Institute.

http://www.quinnipiac.edu/x1327.xml?ReleaseID=1168

Here is the methodology:

http://www.quinnipiac.edu/x2010.xml

Interviewing for the Quinnipiac University Poll is conducted from the facilities of the Quinnipiac University Polling Institute. Professionally trained students and non-students conduct the interviews using a CATI (Computer Assisted Telephone Interviewing system). For a typical public opinion survey a randomly-selected sample of about 1,000 adults aged 18 and over are interviewed over a 5-6 day period.

For a sample of 1,000 adults, the sampling error is 3.1 percentage points at the 95 percent level of confidence. This means that 95 percent of the time, the results obtained should be no more than 3.1 percentage points above or below the figure that would be obtained by interviewing the entire population.

Note that it does not say if cell phones are sampled. We know that households without phones are not sampled.

A study from 2006 discusses the differences between cell phone only users and land line only users.

http://people-press.org/reports/display.php3?ReportID=276

People reached in the cell sample have a considerably different demographic profile from those reached in the landline sample, especially with respect to sex, race, age, education, and home ownership. On many variables, the landline sample was closer to the population parameter than the cell sample, though on some measures the cell sample picks up certain kinds of respondents that the landline samples under-represent.

A majority of those interviewed in the cell sample (55%) were men. Most landline surveys interview too few men, and require quotas or other techniques to obtain the proper proportion of men vs. women. As noted earlier, most landline surveys have too few young people in their samples (7% under age 25, vs. 13% in the population), but the cell phone sample had too many (21%). Conversely, the landline sample has too many older respondents (23% are 65 and older, vs. 16% in the population), while the cell phone sample had too few (just 8%).

The cell sample also proved to be effective at reaching African Americans, as 13% of the sample identified themselves as black. Landline samples often fall short of the population parameter (11%), though the landline sample in this project was very close (10%).

Although the survey was conducted only in English, fully 11% of the cell phone sample was Hispanic compared to just 6% of the landline frame sample. Hispanics constitute approximately 12% of the U.S. population.

Both samples include too many people with college experience, compared with the U.S. population. U.S. government figures show that 26% of the public has at least a four-year college degree, compared with 36% in the landline sample and 35% in the cell sample.

The people reached through these two samples differ in other ways as well. Over seven-in-ten (71%) of those interviewed from the landline sample report being a homeowner compared with closer to half (57%) of those reached on a cell phone. (The U.S. government estimates that 69% of the public are homeowners.)

In addition, fewer of the landline sample respondents were parents of children under 18 – a finding that likely reflects the presence of more young adults in the cell phone sample. At the same time, however, the samples were fairly similar in the percentage of respondents who were married (57% in the landline sample vs. 52% in cell sample – compared with 59% from U.S. government data), though the mix of unmarried people is very different in the two samples. One-third (33%) of the cell sample reported having never been married, compared with just 18% in the landline sample; according to the government, 25% of the adult population has never been married.

Extrapolating from these statistics, those with no phone service apparently are less likely to have a college degree.

This article concludes that excluding cell phone users and those without phones from surveys will not affect results.

However, in 2008 the issue has come up again.

http://www.boston.com/news/nation/articles/2008/01/07/cellphone_only_voters_may_be_problematic_for_pollsters/

But Andrew Smith, director of the University of New Hampshire Survey Center, said there are fewer people in New Hampshire without landlines than in many other states. He does not think exclusion of cellphone-only voters will have a statistically significant impact on polls this time around.

Nonetheless, Cullen said that possible under-polling could affect candidates such as Representative Ron Paul of Texas. Many of Paul's voters are young and might be more likely to have only a cellphone. The difference might be small, perhaps one percentage point in the polls. But it could be significant in a tight race. The question is whether Paul would draw equally from his opponents or take a disproportionate number from one candidate.

"A pollster would tell you that it doesn't matter if you exclude them provided they are voting in the same way as other voters are," Cullen said. "But I personally am not persuaded by that. It seems clear to me that younger voters have different preferences from other voters and that younger voters are disproportionately more likely to have given up their landline."

Under normal circumstances, the demographics of those who do not keep a land line or use a phone at all and the demographics of those who use only a cell phone would not make them react to events of the day in a way that differed much from those who kept land lines. However, if you consider the fact that those with land lines are more likely to own homes, have higher incomes, live in urban areas, be married and to be employed and to be older, then some events could have a differential effect on voting and might be unlikely to be sampled by the cheap and easy traditional polling methods. One example would be remarks that could be construed as innocent by most of the population but as troubling by people who are poor, unemployed, living in rural areas and who do not own phones.

What concerns me is that the pollsters may be painting another false picture as in New Hampshire, because they are unable to reach the neediest people in the state of Pennsylvania for whom economic matters are the most important. Hillary Clinton has spent a great deal of time in economically blighted industrial regions like those in Pennsylvania. If the Obama camp gets a false sense of security from polls that do not interview the appropriate voters, they will not know that a problem needs to be addressed. The campaign would be wise to conduct internal door to door polling of appropriate areas to get a feel for the mood of the state.

Another problem is the unusually high turn out this primary season in which more people than usual have been coming out to vote. This increased turn out adds a new degree of uncertainty, making polls even more unreliable. The massive sums of money being spent in Pennsylvania almost guarantees huge turn outs.

So, I would be careful about spinning anything around Pennsylvania. The wisest course for both Hillary and Obama might be to say that with the economy as bad as it is, the voters are likely to be unpredictable.

[datopbanana]

1. Ever think for a second maybe bitter-gate is really just a faux outrage

no? ya must be bad polling. :eyes: