Proposal for objective measurements on voting and exit polls

[JMDEM]

Correlation between actual voting and exit polls in paper-based voting versus electronic based voting.

Direction of skew in results that don't closely match across elections. Skew should be random if it exists.

If results show that exit polls are close in paper-based voting but much farther off in electronic-based voting, this should offer proof that something is wrong, even if we haven't figured out what.

If skew is not random, then it shows possible manipulation by one party but not the other.

Anyone have any numbers?

[OnTheOtherHand]

1. there are lots of numbers out there

The Edison-Mitofsky evaluation report in early 2005 reported that the discrepancies in precincts that used touchscreens, optical scanner, and punch cards all were similar; the discrepancies in lever-machine precincts were higher, and the discrepancies in the 40 hand-count precincts were lower. (See page 40.)

Some folks have interpreted that as evidence that only the hand counts were accurate -- but as the report pointed out, hand counts were used almost exclusively in rural areas, and the discrepancies were lower in rural areas regardless of the equipment they used. Controlling for size of place, the hand count discrepancies are statistically indistinguishable. (The lever machine discrepancies probably have to do with place, not equipment. In New York, John Kerry did astonishingly well in the exit poll, but his vote count was in line with -- slightly better than -- pre-election polls.)

The analysis turns out the same way using state-level data (see here). Moreover, if one looks in states like New Hampshire or North Carolina that used a mix of hand counts and other methods (or Ohio, which used several methods although not hand counts), there's no evidence of a technology effect. I've written up some of those results, but never bothered to post them because no one's attention span seemed to be that long. Here is an interesting analysis.

All that is with respect to the 2004 general; I've seen no reason to think that the results would be qualitatively different in 2006 or the 2008 primaries.

[JMDEM]

2. Edison - Mitofsky????

Now they wouldn't have any reason to fudge the data, would they?

[Wilms]

3. They would in a way.

Exit polls are made of fudge.

And some actually think they confirm election outcomes. I know, I know. Sounds amazing that people who are hip to the possible manipulations of votes would place so much blind faith in exit polls to the point where they actually claim those polls prove something.

Laughable, I know. But we all have rent to pay.

[JMDEM]

4. Then why does the US always point to exit polls in other countries

to scream "foul" when an election doesn't turn out as predicted?

Please back up your assertion that exit polls are crap with numbers. And drop the fucking condescending attitude.

[Wilms]

5. "condescending attitude"? Read my response.

:hi:

[OnTheOtherHand]

6. well...

First of all, if you think E/M fudges exit poll data in general, then I guess this whole discussion is kind of pointless.

Second, if you think that E/M fudged the 2004 data retroactively to cover up Republican election fraud, well, I'm not sure what reason they would have to do that. Warren Mitofsky never struck me as a Bush supporter. I suppose we could always speculate that someone threatened him -- although actually, I have a hard time imagining Warren Mitofsky meekly doing what he was told.

But, third, it's moot, because the state-level analysis I linked to works with screen shots. And the analyses of election data don't have to use exit polls at all.

Now, maybe it's time for you to do some work. Can you support your assertion that "the US always point to exit polls in other countries to scream 'foul' when an election doesn't turn out as predicted?"

[Sancho]

7. Do you know of any similar regression analyses for undervotes?

In Florida, besides the upfront caging that occurs in registrations, the biggest discrepancy is the differential undervotes from precincts and counties. There is almost no logical explanation except for error in the voting method or machine. As you likely have noticed, post hoc "testing" confirms that the machines work correctly...so the only conclusion is that voters in one precinct are idiots compared to those voters across the creek!?!

Since I've witnessed a touchscreen machine in 2004 continually "change" a vote from the original intent to the review page, I know that they are capable of such errors that even reasonable voters would not always catch. Even after rebooting the machine and replacing the voter's recording "card", that machine continued to change the votes.

I suspect that those "errors" are showing up as undervotes in the tabulators, but I have no idea how to find evidence of it without a comprehensive "revote" as an exit poll, a controlled random sampling design during the election to check each type of voting machine, or video of every voter's hand pushing the buttons but not revealing the face of the voter.

[OnTheOtherHand]

8. not at the precinct level, no

although you might check the VoteWatch(?) work in New Mexico -- I think it was on too small a scale to pick that up. Even precinct-level would be iffy. Back in 2000 about 5 1/2% of Florida precincts had residual vote rates over 10%, which is shocking -- but if there were 50 precincts exit-polled that year, maybe 2 or 3 of them would be affected.

Exit polls typically indicate intentional undervote rates (for top-of-ticket races) under 1%, so they could be used to rule out crackpot hypotheses such as FL-13 being due to intentional undervoting (not that that was top-of-ticket, but still...). Of course, there were many other lines of evidence that ruled that out. And of course there were both ecological and ballot-image-based estimates of how likely Jennings voters were disproportionately affected.

In places where off-the-wall undervotes are endemic, I think voter-experience exit polling could be useful. Actually I suspect lots of useful things could be done with voter-experience exit polling in all sorts of places.