What Do The Polls Really Tell US? "Polls of Polls"

[aaaaaa5a]

Real Clear Politics has what it calls the “RCP” average for predict election results. It takes the last 5 polls and averages them together to get a consensus or predicted outcome.

This system is used frequently on Cable News and is often called the “poll of polls.”

Below is a look at the RCP “polls of polls” margin of victory prediction and the actual results.

The results are from Primary states. Caucus states are not included so that we can have (as much as possible) an “apples to apples” comparison to the polling data for Pennsylvania.

There are no available “RCP Poll of Poll averages” for Primaries held in several states including Delaware, Mississippi, Vermont and Oklahoma. This was probably due to the lack of available polling completed for those states.

This left Real Clear Politics “polls of polls” available for 19 primary contests.

I separated the RCP results into the following categories.

RCP was extremely accurate (within 3 points) 42% of the time.
8 contests

New Jersey
RCP… Clinton +9.8
Results … Clinton +7.7

New York
RCP… Clinton +17.2
Results… Clinton +17.5

Illinois
RCP… Obama +33
Results… Obama +31.5

Arizona
RCP… Clinton+ 6
Results… Clinton +8.8

Tennessee
RCP… Clinton +13.3
Results… Clinton +13

Ohio
RCP… Clinton +7.1
Results… Clinton + 10.1

Maryland
RCP… Obama 22.3
Results… Obama 23.5

Texas
RCP… Clinton + 1.7
Results… Clinton +3.5

RCP predicted the correct winner but the margin of victory (outside of 3 points) was wrong 32% of the time.
6 contests

South Carolina
RCP… Obama + 11.6
Results… Obama + 28.9

Virginia
RCP… Obama +17.7
Results… Obama +28.2

Wisconsin
RCP… Obama + 4.3
Results… Obama + 17.4

Rhode Island
RCP… Clinton + 9.7
Results… Clinton + 18

Massachusetts
RCP… Clinton + 7.0
Results… Clinton + 15.4

Georgia
RCP… Obama + 18
Results… Obama + 35.3

RCP predicted a margin of victory for the wrong winner 26% of the time
5 contests

New Hampshire
RCP… Obama +8.3
Results… Clinton +2.6

Connecticut
RCP… Clinton + 4.0
Results… Obama + 3.1

California
RCP… Obama + 1.2
Results… Clinton + 9.6

Alabama
RCP… Clinton + 1.2
Results… Obama + 14.1

Missouri
RCP… Clinton +5.7
Results… Obama + 1.2

Quick take home points.

It’s interesting that the RCP average of the best pollsters in the country (polls of polls) actually yielded the wrong winner (outright) a whopping 26% of the time or better than 1 out of every 4 contests!

I also found it interesting that of the 5 times the RCP predicted the wrong winner, 3 of the 5 mistakes (60%) involved Obama’s support being under-represented in polling.

The “Bradley effect” in today’s polling is very overrated. The media made a big deal about this following Clinton’s win in New Hampshire. But what happened in Alabama where Clinton was favored to win but Obama won by 14 points? Or in Missouri and Connecticut which do not have large African American Populations?

In the 6 contests were the RCP (poll of polls) has the winning candidates margin off by 3 points or more, (32% of the time) again it was Obama’s support that was underestimated in 4 of the 6 (67%) results. In fact only in Massachusetts and Rhode Island has Clinton exceed her RCP predicted margin of victory by 3 or more points.

The RCP polling average (Polls of polls) had the correct winner and the correct margin of victory (within 3 points) in 8 contests or 42% of the time. Amazingly in those contests, Hillary Clinton was the winner in 6 of 8 or 75% of the time.

Currently the RCP polling average for Pennsylvania is Clinton + 5.7.

Public Policy Polling and Survey USA should have new polls out Monday. Also Zogby is tracking the state daily and I believe Rasmussen will have another statewide poll out before voting on Tuesday.

You can find all of this information at realclearpolitics.com

[jbm]

1. great research

thanks for doing the work-and welcome to DU!

[bigwillq]

2. Hello.

Welcome to DU! :hi:

[cliffordu]

3. Great research -

I have been wondering about the accuracy of these deadbeats, too.

:patriot:

[rocktivity]

4. Polls are snapshots, NOT prophecies

Which is why they should always be taken with a pillar of salt--and why I've been posting this every day:

GET THEE BEHIND ME, POLLS!

UNCLEAN! UNCLEAN!!!

:headbang:
rocknation

[SoCalDem]

5. and landlines are pollsters tools.. I'm guessing that MOST

of the newer voters have never even HAD a landline..Pollsters do NOT poll cellphones...

:)

Time will tell..and of course Fast eddie Rendell & his henchmen are "in charge", so who knows how the "count" will work out..
I'm just hoping that enough of the New and unpollable ones show up, that any mischief is overtaken :)

[aaaaaa5a]

6. The Cell Phone Issue

I've heard a lot about pollsters missing votes because of the use of cell phones. But I think that issue is a little overrated.

Don't people in Texas and Ohio have young people with cell phones too? Yet the "Polls of Polls" were fairly accurate for those primary states.

Obama could win on Tuesday. But I doubt it will be because of an underground "cell phone vote."

[SoCalDem]

7. Obama won texas and was not favored in Ohio, but still came close

The Rushites For Hillary were devoted enough to show up the the AM, but not enough to show up at night when it counted :rofl:

they "gave" her a bump in the popular vote, but NOT in the delegates vote.. Popular vote is not used in Primaries (unless you are HRC and desperate to overthrow the wishes and votes of the nation )

[johnnydrama]

10. one difference

It's not that people everywhere have cell phones too, it's people that have cell phones as their only phone in their house.

Like me. I've never been polled or received a robocall at all.

Now, i have no idea if some states or cities are prone to have more people like me, or less people like me. It would be an interesting study. My guess is people in large cities would more often have a cell phone as their only phone.

Just a guess though.

[Coexist]

8. thanks - and welcome to DU!

[TheDoorbellRang]

9. I'm so glad you put this together

I had haphazardly looked at one or two of these and noticed they were off quite often, but laying it out like this really points out how unreliable the polls have been on this.

Have you seen a copy of the Obama campaign's spreadsheet? They've got a lot higher accuracy rate than the Poll of poll averages. They've predicted a 47-52% margin with Clinton winning PA on Tuesday, so it'll be interesting to see how close he gets.

Welcome to DU. Good to have another math dude on board.

[DerekJ]

11. Hey! Thanks for the sheet. I heard about it, but never seen it before.

[dchill]

12. Nice work.

Welcome to DU.

[foo_bar]

13. the RCP averages should probably assess meta-confidence

For instance there weren't many Wisconsin polls in the final days, so the average reflects three polls from 02/08-02/14; in Ohio and Texas they used seven from 02/27 - 03/03, Alabama and Virginia only four. Iowa and NH didn't do well in spite of an abundance of polls, but caucuses with a two-step system are basically unpollable: while some of it can be gleaned from voters' stated backup choice, the decision for Kucinich supporters to ultimately caucus with Dean or Kerry or walk out entirely wasn't made until Kucinich was eliminated, and a voter's intended fallback candidate might become #3 if friends and family defect en masse to Candidate Y (i.e., the bandwagon effect). So basically any metapoll with a small (meta)sample or one that uses straight up telephone polls for a caucus should get a metaconfidence of bupkes, and the big picture might look clearer. Still doesn't explain New Hampshire's orneryness or difficulty to poll (or rampant fraud assuming the Kucinich recount was part of it), but large states have more pre-election polling and large states also trend Hillary, so that might explain teh 42% going 6 for 8 Clinton.

[quantass]

14. Here is an interesting Curiosity about SUSA

I decided to look at the polls for each of the primaries since january and looked at all the ones that "Survey USA" did a poll they were within 4pts 80% of the time....however, i noticed that they only polled about 10 states of the 40...they were completely wrong on Missouri (there worst poll prediction -- Obama won that State by 1,despite SUSA predicting a Clinton win by 11 days before)....

[aaaaaa5a]

15. Information About Survey USA

I just posted information about the accuracy of Survey USA during this election season. It sounds nerdy, but I actually had a file on Survey USA's results saved on my computer. You can take a look at it. It's very interesting. Overall they have been very good. But they did get the winner wrong twice. (Texas, Missouri)

[aaaaaa5a]

16. I just wanted to update the Real Clear Politics "polls of polls"

Poll Date Sample Clinton Obama Spread

RCP Average 04/17 - 04/20 - 49.0 43.6 Clinton +5.4

Rasmussen 04/20 - 04/20........................... 722 LV 49 44 Clinton +5.0
Zogby Tracking 04/19 - 04/20...................... 602 LV 48 42 Clinton +6.0
Suffolk 04/19 - 04/20............................. 600 LV 52 42 Clinton +10.0
PPP (D) 04/19 - 04/20............................. 2338 LV 46 49 Obama +3.0
Strategic Vision (R) 04/18 - 04/20................ LV 48 41 Clinton +7.0
Quinnipiac 04/18 - 04/20......................... 1027 LV 51 44 Clinton +7.0
SurveyUSA 04/18 - 04/20.......................... LV 50 44 Clinton +6.0
Mason-Dixon 04/17 - 04/18........................ 625 LV 48 43 Clinton +5.0


The RCP average is a Clinton victory by 5.4 points.

In every primary this season when Clinton was predicted the winner by the RCP "polls of polls" she exceeded her margin of victory by 3 points only twice. (Massachusetts and Rhode Island)

This means if this trend holds true, her margin of victory will be under ten points.

Zogby will should have their Monday polling numbers out this evening. But it's not likely to move Clinton's RCP victory line too much.