Rebecca Mercuri: On Lotto ball drawing for mandatory VVPB audits

[FogerRox]

As it turns out, it is FAIRLY STRAIGHTFORWARD to produce a 1% random
audit of all precincts in the state by drawing ONLY ONE BALL from a
state lottery machine, and with only simple modifications to the
regular
end-of-election day procedures. (It is possible to apply this scheme to
any percentage of audit, but 1% was used for example. Also, I have used
NJ's procedures as an example, but other states should be able to
modify
theirs similarly easily.)

Here's how the Mercuri Recount Method works:

1) Prior to the election, assign each precinct a recount number from 00
to 99. (Yes, there will be precincts that have the same number of other
precincts, but that is OK.) The assignment of numbers must be done in
such way that there is an optimally equitable distribution of numbers
to
precinct locations and Counties throughout the state. Since the
precinct
numbering list is only created once, and published well in advance of
its first use, the math wonks in the state can ensure that the
assignments were made appropriately. The same recount numbers can be
used for all elections every year (like snow numbers for school
districts).

2) Also prior to the election, create a local recount board for every
100 (relatively adjacent) precincts, similar to the manner in which the
poll worker groups are created. Provide training for all recount
workers
so they know what they are supposed to be doing. Give each recount
board
member the number list and addresses of the 100 precincts that they
might be deployed to on election night, along with names and contact
phone numbers for each member in their local recount board (and some
emergency phone numbers to use in case people get lost or don't show up
as assigned).

3) Put 100 balls in the lottery machine, labeled 00 to 99. At 8PM on
election night, start up the machine and have it eject one of the
balls.
This should be a televised event (with witnesses, just like the
lottery). NOTE that it is important to use the ball technique, because
computers do not generate truly random numbers and their selection
process can be compromised (or revealed in advance of the end of the
election). Have this number announced on the radio (and perhaps also on
the Web and via a call-in phone line). The media can also optionally
use
the state precinct map to announce which precincts in the state are
going to be recounted, and they should send reporters out to cover the
event.

4) Recount workers and poll workers tune in to the broadcast at 8PM (or
phone in, etc.) to find out what precinct recount number was selected.
Recount workers then head out to the designated polls. Poll workers for
the selected precincts will need to stay around until the recount
workers arrive, in order to hand off the ballot boxes, before they
leave
to turn in the rest of the election materials (including the computer
generated vote totals, but excluding the voter verified paper ballot
boxes left behind for the recount workers). (Most likely, the recount
workers will arrive in the designated precincts before the regular
precinct paperwork is done anyway.) Poll workers for all other
precincts
can leave to turn in their election materials (including their voter
verified paper ballot boxes) once they have finished their end of
election paperwork (and they can proceed to the election night
parties).

5) Recount workers can begin the process of counting the ballots as
soon
as their recount board has assembled. NOTE that the recount workers
should not be informed of the electronic vote totals in their counting
precinct until AFTER they END their counting, so as not to bias their
results. Recount workers return the voter verified paper ballots to the
County along with a report of their agreed upon results (with a
possible
minority report if consensus is not achieved) when they have concluded
their count. Estimates indicate that it should be possible for most
recount precincts to have their results finalized by 11PM, especially
if
the process is organized such that it is performed in an efficient
manner (for example, through the use of hash bins or barcoding --
explanations will be provided in a later memo). Even if this takes a
longer amount of time, the results from the voting machines, as turned
in by the poll workers, in all precincts can still be used as an
UNOFFICIAL result for the election, in the same way that precinct
results are currently used. Note that the recounts can proceed even if
the computerized voting machines are unable to produce their end of day
report, because the recounts rely only on the voter verified paper
ballots in the ballot boxes. The County election office will have to
remain open to receive the recounts, but generally they are still open
through 11PM anyway (to deal with broken voting machines and so on), so
this is unlikely to add any additional hardship.

6) Final confirmation of the overall election results still lies in the
hands of the County officials, who may be required to conduct
additional
precinct audits if the recounts disagree with the electronic totals, or
if a further election recount is authorized by the court.

7) If you want to incentivize voting, have each voter issued a ticket
numbered 00 to 99 when they sign the poll book (it can be a tag that is
ripped off and retained by the voter, when they are handed their voting
authorization slip). Voters who have the matching number get to take
$5.00 off of their state taxes (or they can donate the $5 to any of the
state charities) if they mail in their ticket with their tax return.
;-)

Alternatively, all recounts could occur in the County offices on the
day
after the election, but this could raise questions as to the
appropriate
impounding and transfer of the ballots, since all ballot boxes would
have been returned to the County on election night. Next-day recounting
also lessens the transparency and immediacy of the recount process, and
could introduce bias because the precinct results would be known, but
it
does make it a lot easier in terms of transportation logistics for the
poll workers and recount workers. If recounts occur the next day, the
statewide lottery ball selection should be conducted that day as well
(as soon before the beginning of the recount as reasonably possible),
so
as to reduce the chance of corruption of the contents of the designated
precincts' voter verified paper ballot boxes due to the advance notice.

Please let me know if you see any overt flaws in the above process
(perhaps excluding step 7). Feel free to distribute this email in its
ENTIRETY (including my name and email address) to interested parties.

Rebecca Mercuri.
[e-mail]

[shraby]

1. One flaw that I can see....

a microscopic pin hole in certain balls would make them bounce differently and not be chosen for audit.
It was discovered this happened with one state lottery.

[Junkdrawer]

2. We need two random audits...

The Central Tabulator Hack (and it's cousins) rely on the ability to intercept and change precinct numbers on election night. Later, new paper tapes are created that match the hacked results. We need to be able to go back to the voter verified source documents on a random basis and verify the true precinct totals AFTER all the precinct numbers are posted and finalized.

I do like the lotto ball method of picking random numbers - assuming a Nick Perry type hack isn't used. ;) See: http://en.wikipedia.org/wiki/Nick_Perry

[FogerRox]

3. email exchange witha fellow NJ Election Reformer

In an audit, the % sample has to be high enough and purely random "

This "purely random" is problematic. Election Reform groups across the country have pretty much come to the conclusion that in a purely mathematical sense, we dont want "purely random".
Think: Representative, selected in a purely random manner.
Purely random could end up auditing only in South Jersey, a one in a million shot, but in a purely random selection, the odds are it WOULD HAPPEN.

"The closeness of the race does not really change this. "

Yes it does. There is an existing body of work that indicates this is so. I will dig up some cites for you. In the Mean time-

Race 1: 998 to 1002. The margin is 4 votes.
Race 2: 500 to 1500. The Margin is 1000 votes.

A static %, say 2% will audit:
Race 1: 40 votes
Race 2: 40 votes

Lets assume I hacked both races.
Race 1: I switched 5 votes
Race 2: I switched 1001 votes.

AS a tool, a certain %, say 2%, is far more likely to find the problem in race 2. Yet if race 1 was a local race for Town Council- with maybe only 2500 votes cast, why not hand count the entire race? Do it Election night. Get it right the first time. This is what would instill confidence in the system. If the Losing candidate witnesses the hand count of all 2500 ballots-- the Candidate KNOWS THEY LOST. There can be no question of the results, at that point. I think 10 people could hand count 2500 paper ballots in an hour, easy.


"An audit is not a recount. If a race is really close then there needs to be a recount, and candidates have a right to demand this under current law "

I would contend the public & the candidates have the right to have confidence in the results, the first time. If a losing candidate doesnt have confidence in his or her loss, what is the purpose: is it to instill a lack of confidence in the voting system? If a race is really close it needs to tallied right the first time. This is what will instill confidence in the system. Google "recount", you will find that last November there were literally 100's of recounts across the country, because of a lack of confidece in the results. Isnt this a scenario we wish to avoid?


Truthfully, I don't know if 2% is enough to be reliable. But politically, if we want more than that, we'll have to give strong evidence that 2% is not sufficiently reliable. Any statisticians out there?

Most states seem to use a static 3% audit, I would consider that to be a nation wide benchmark. IIRC Holts bill does use 3%. S55 in Conneticut uses one machine in each polling place--IIRC- and can appraoch as high as 8% to 12% depending on the voting machine disbursement.

Let me stress this one more time. A static % audit is fine, except in closer races. Its akin to an astronomer needing a larger telescope to see more detail on the moon.

1) Automatic Audit picked by Lotto ball, statewide. Say !%
2) Automatic Audit picked by Lotto ball, each county. Say 1%
3) Any race, the closer the results, the more it gets audited, up to, say 10% or 15%.Though I would settle for about 6% or 7%.

Items 1 & 2 would total 2% statewide. This provides for a bit more representation than Nias Bill does, she provides for one machine per county, as some counties have 900 machines while a small county may have only 100 machines. This is very unrepresentative.

Under my version, Essex, with about 700 voting machines would get 1% or about 14 machines audited. While a small county like Warren with 100 machines would get 1% or one machine audited. In the above, it should be assumed that each voting machine serves about 100 voters. There is no reason that Essex should have the same audit rate as Warren, but Nias bill does say exactly that.

Summary-

NIAS BILL -

1) Mandates the same minimum audit rate of one voting machine in each County.
This is not representative of the differing population sizes in each county.
2) Appllies the same audit tool ( a static % audit) to mathematically differing situations,
this will create a lack of confidence in the results in close races. This may also create an eviroment ripe for contention and lawsuits.
3) Completely lacks a Tigger event, that mandates a full hand count of all ballots.

If the procedures are spelled out in well defined language-- that makes it less likely that anyone can game the system- RE: the NJ AG. Though Richs point about the NJ AG has to be seriously considered.

Thanks-
Roger Fox
-------------------

I like the Tabulator and local checks-- very good.

[nicknameless]

4. K&R for more to see.

[Bill Bored]

5. I hate to bring this up but

doesn't Joyzy's VVPAT law not go into effect until 2008?

[Wilms]

6. Point being?

[Bill Bored]

8. Point being that they won't have anything to audit for another 2 YEARS! nt

[Wilms]

10. I should have added, "apart from the obvious".

:spank:

[FogerRox]

7. NJ VVPB law has no mandatory audit --

[nicknameless]

9. I like the lotto ball method of making random selections,

provided that previously mentioned strategies for cheating can be prevented.

I like that it allows the public to observe the selection.
I would go for a higher percentage for auditing. (Read 10% recommended somewhere. Thought it was on Chuck Herrin's website, but couldn't locate it.) As you point out, that would be especially important if the race were close.

(I prefer 100% hand counts. Entire countries count all of their ballots by hand in a matter of a few hours, so I don't see the problem recounting a higher percentage.)

I think that ALL recounts should be conducted in public view. In the UK, counting is done with observers and cameras present.

I would ABSOLUTELY be opposed to the "next day" "in the County offices" approach. Prepared ballots to swing the vote totals could be a real problem. Fitrakis' reply about the Warren County lock down is a good example of why that should be avoided.

... 2 cents ;)

[nicknameless]

15. Clarification:

Chuck Herrin personally opposes any kind of machine counting. From his website:
http://www.chuckherrin.com/hackthevote.htm

As always, click around, enjoy, and write your elected officials and tell them you want Hand-Counted Paper Ballots. Now.

Thanks! :-)

Chuck

[Bill Bored]

11. Hypergeometric Distribution/Sliding Scale

Use the Hypergeometric Distribution to calculate the sliding scale percentage based on vote margin, how many corrupted machines it takes to overturn the race in question, number of machines in jurisdiction, etc.

Kathy Dopp has this one right:
http://uscountvotes.org/ucvAnalysis/US/paper-audits/Paper_Audits.pdf

Use the associated spreadsheet to see how much security 1% will buy you.

Calculate odds of finding a corrupt machine based on 1% audit and number of machines in the county.

How many corrupted machines would it take to swing the election by the margin + 1 vote? (You will need to know average number of votes per machine for this part.)

Audit until you have 100% chance of finding that many.

Something like that anyway.

Peace.

[Febble]

12. I agree that the random bit

is essential, and that Kathy's tables are a good start. However, her tables give the probability of finding a corrupt precinct for a given percentage audited, given the total number of corrupt precincts, which obviously you don't know, a priori.

So your point about calculating the amount of corruption needed to swing the election is also good.

But we still have an unknown, because we don't know whether to test for a small amount of corruption in a large number of precincts or a large amount of corruption in a small number of precincts. As I understand it, Kathy's tables give you the probability of detecting a corrupt precinct, not the probability of detecting corruption of a given effect size in terms of margin.

The other geeky point that may be worth pointing out is that the probability of detecting a corrupt precinct will only actually be 100% if you sample more precincts than there are corrupt precincts - she has, quite legitimately, rounded the probabilities up. But conceptually, it's still worth thinking about, because, clearly, if, as in the example at the top of her table, there are 500 clean precincts, and 500 corrupt precincts, and you sample only 20, you don't have a 100% chance of finding a corrupt one (it's actually 99.998% according to my version of Excel, if that hasn't been corrupted). You will finally hit 100% if there are 501 precincts in your sample.

And that brings me back to the first point - clearly you don't know in advance how many corrupt precincts you have, but you also don't know how thinly fraud on a scale to swing your election would be spread. The more concentrated fraud on a given scale (total number of votes switched), the smaller your chance of detecting it on a given percentage audited (but the larger the chance of it being detectable as a swing-shift correlation....)

So I'm not knocking it. It is certainly worth knowing, when designing your audit, what percentage of precincts must be clean at a given probability if your audit comes out clean. And the more clean precincts you have, the more egregious the fraud would have to be in the corrupt ones to swing your election.

But the really essential thing, to me, is that the sample is random (stratified random if you like, or weighted by size to give each vote an equal chance of being recounted, or weighted in some other way) - but the point is to make sure that NO-ONE knows in advance which precincts are going to be counted, and also that NO-ONE has had a chance to tamper with the ballots in between. No use having a random selection, and then giving the crooks the chance to fiddle with the selected ones.

Secure custody of the ballots goes hand-in-hand with random audit.

My two pence, or three.

[Febble]

13. Correction

Not my version of Excel corrupted, just my brain:

The answer is: 99.99992%

Good enough for jazz.

[Bill Bored]

19. Jazz is cool, but is it close enough for Government work?

It's true that this is the probability of finding one corrupt precinct. But at least, if the audit is truly random, you can find that probability.

Let's walk through an actual example using Kathy's PDF tables:

How many corrupted precincts would it take to reverse the outcome? This depends on the absolute vote margin and the average number of votes per precinct. (Should it be average, or some percentile-based statistic such as median?)

Let's say the absolute vote margin is 1,000 votes and the average number of votes per precinct is 100. It's not likely you're going to have a 100-0 precinct, or not many of them at any rate, so it should be safe to assume that it would take at least 10 precincts to reverse the outcome and probably a lot more than that.

Being conservative, assume you're looking for 10 out of the 1000 precincts.
With a margin this close, it will take a lot more than a 5% audit to be sure. My Excel tells me it would take about 60%!

The good news is that with a 1000-vote margin and 100 votes per precinct and 1000 precincts, you'd have 100,000 total votes and therefore the margin would be only 1%. You might want to do a full recount under those circumstances anyway, but a 60% recount seems to be good enough.

Might be interesting to float some more examples like this. Some jurisdictions have mandatory recounts if the margin is .25% or .5% anyway and that's easy enough to legislate if they don't. So I think we should look at some slightly wider margins like the one above.

Interested in your comments.

[Febble]

21. Well, I think that was my point

although in a far-off universe I believe Kathy has misunderstood it, probably because I was unclear, possibly because her default is to assume I'm messing up.

So I'll try to clarify one thing: clearly if you audit a precinct, you count all the ballots, so you should be able to detect even small scale fraud in that precinct, and in any others you audit. So if you are trying to find a large amount of total fraud distributed over a large number of precincts, you need to sample fewer precincts to find it, than if the fraud is concentrated in a few.

So I think you are right, you have to have a sensible hypothesis about the magnitude of fraud likely in any one precinct, and the likely prevalence overall. Then you can figure out the probability of detecting fraud on a scale to overturn a given margin given your sample size.

But your probability will depend on your assumptions regarding a) the magnitude of fraud likely in any given precinct and b) the likely prevalence of corrupted precincts.

(edited for spelling)

[OnTheOtherHand]

22. there isn't exactly a statistical answer to this problem...

Once you start conjecturing the possibility of 100% error in a precinct, there is no reason to stop there. Remember the Gahanna precinct with (initially) over 4000 votes recorded. I don't think that was an attempt at fraud, but it really doesn't matter -- there should be recourse for revisiting such results.

Oh, just a comment on "100% error" (my phrase, not yours). Right, switching a 75-25 precinct to a 25-75 precinct will be a net swing of 100 votes, and there won't be many opportunities to steal a higher proportion of votes than that, by switching. In theory, up to 200% error is possible through vote-switching alone: from 100-0 to 0-100, a net swing of 200 votes.

[yowzayowzayowza]

14. Any audit which fails to verify EVERY...

counting device is suspect. Would a bank only audit 1% of itz ATM machines? Every automated process MUST prove itz werth in every election, otherwise there will always be exploitable holes in the process.

Sorry, don't have confidence in this type of validation process.

[FogerRox]

16. correction of my mistake

Race 1: 998 to 1002. The margin is 4 votes.... 10 corrupted votes
Race 2: 500 to 1500. The Margin is 1000 votes. 10 corrupted votes
In the above-- probability of finding corrupt votes is the same, if everything else is the same, % of audit- method of audit.

SO it would seem that because a race is closer, there is no need to audit at a higher %. DO I have that right?

[OnTheOtherHand]

17. well, it may depend on what you are looking for

First of all, did you really mean that Race 1 and Race 2 have the same number of corrupted votes -- with the result that the corruption determines the outcome in 1 and not in 2? If so, sure, it won't be any easier or harder to find the 10 corrupted votes in either race. But it will be more important in Race 1, no?

An important rationale for a full recount in close races has been to catch accidental errors, as opposed to deliberate corruption. Are you considering that? (Not a rhetorical question -- I am coming late to the party, and I am sleepy.)

Back to corruption -- first of all, let me think about bigger races. Certainly if there are only 2000 ballots total, one might as well just count them. Basically, I think the closeness of the race may matter somewhat inasmuch as it might influence the fraud strategy (specifically, how many precincts are corrupted), although I'm not sure how much it matters in practice. It also seems to me that one should expect some legitimate differences between a machine scan of some ballots and a hand count of the same ballots. Therefore, you need a protocol that will accommodate legitimate diffs that don't bring the outcome into question, while catching any systematic bias (fraud or even some subtle innocent bias that human counters can detect -- an example of the latter might be rejecting as overvotes ballots that have a vote and a write-in for the same candidate).

[FogerRox]

28. OTOH

First of all, did you really mean that Race 1 and Race 2 have the same number of corrupted votes -

not at first, I thought 5 corrupted votes for race one- 1001 corrupted votes for race 2, then realized I was not speaking math, so I tried to correct the example
- with the result that the corruption determines the outcome in 1 and not in 2? If so, sure, it won't be any easier or harder to find the 10 corrupted votes in either race. But it will be more important in Race 1, no?

basically I confused myself. -putting foot in mouth too-

Now after digesting more info-- I'll guess that the best way to steal an election is to corrupt a few votes, in enough (widespread?) precincts.

But would there be precincts that "appear to be safer" to flip more votes? --Landshark would feel he could get away with more vote flipping in certain precincts.

SO I'm thinking we have to devise one method that would be able to spot 2-3 or 4 different style problems. Yeah, easy, no problem.

[Usrename]

29. just so you don't fall off the path,

The truth is that the margin in an election should never be used to determine audit percentages. To do this would mean that all the election tamperers would have to do in order to avoid proper audits is to merely create a large margin. i.e. the more vote fraud, the less the possibility of detection. Don't let them steer you away from seeing this. They may never cheat any harder than they have to, but if they have to cheat harder, they will.

[FogerRox]

30. LOL-- yeah I see that now-- LOL

[Febble]

35. Exactly

which is why I am somewhat suspicious of laying down what needs to be done in too inflexible way. It would simply inform any potential fraudster of what to avoid.

As BB says downthread, one issue is what procedure to follow once you find an anomaly.

Of course in the UK we do a second full handcount....

[OnTheOtherHand]

36. OK... (sorry, I lost this thread for a while)

I can't offer any huge generalization about "best," but stealing a few votes in lots of precincts is (as far as I can tell right now) most likely to escape statistical detection by looking at the returns. Stealing 15% of votes in 20% of precincts tends to stand out -- although, if one could actually control all the precinct returns over a period of time, things could get weirder and weirder.

Now, the problem is, it's one thing to be able to detect concentrated fraud, and it's another to be able to do anything about it. I haven't seen anything in the Ohio returns that I'm sure was "fraud" per se, but there are certainly things that Totally Make No Sense -- and there is no recourse. (I don't think the things I have in mind would have swung the outcome, but that isn't even the point; we are trying to design systems that are reliable, not systems that might work well enough most of the time.)

So, I'm inclined to like systems that allow both for a broad, purely random and rigorous recount regardless (3% sounds pretty good) and for some way to target a recount (the Gold Star regime provides one; many others are possible). That said, fraud would probably have to be really concentrated in order to evade even a 3% audit with high probability -- assuming, of course, that the 3% audit is well designed.

I disagree with Usrename, perhaps, in that I think it could be perfectly legitimate to order a more stringent recount when the race is closer. (It depends on the other decisions.) But I agree that a big official margin isn't grounds for a lax recount.

[Febble]

18. I'm not quite clear what you are asking here

or whether I could answer even if I knew.

But if you are concerned that the level of corruption may exceed the winning margin, there is a problem with deciding on the audit percentage at the precinct level rather than the vote level. As I said upthread - a lot depends on whether you suspect a lot of salami-slicing fraud or a little major fraud. If you think someone has been shaving votes all over the place (as I would do if I could) then you would need to audit a lot of precincts - enough to tell you whether errors in any one direction where significantly greater than errors in the other direction.

On the other hand, if you have reason to think that fraud was concentrated in a few places, then you'd be better off using some yardstick such as swing from previous election (as in NH) to handpick suspicious precincts.

I don't think there is any one answer - in fact I wonder whether rules of thumb are asking for trouble, like advertising where your burglar alarm pads are placed.

And it's one of the reasons I think that we need to think in detailed ways as to how an election might be stolen, and what the evidence is for what. These forms of fraud would need to be checked for in different ways:

Pre-programmed voting machine fraud

Tabulator hacks

Opportunistic ballot-stuffing

Messing with punchcard precinct ideas

Messing with ballot rotation

Shutting down a count because of a "terrorist alert" and having your evil way with the numbers in private

As I keep saying, ad nauseam, what you test for depends on your hypothesis. And having an a priori hypothesis hugely increases the statistical power of your test.

[Febble]

23. It has come to my notice

that in the outer reaches of the galaxy, someone may have interpreted my remark:

"If you think someone has been shaving votes all over the place (as I would do if I could)...." to mean that I intend to rig your elections.

May I take this opportunity to declare that such is not my intention.

I merely stated it as what in my opinion would be the best strategy for getting away with massive theft if you could manage it. But not, of course, if you had a good auditing system in place.

Because I'd also like to make it clear that I am entirely in favour of random audits, and that Kathy's paper is an excellent place to start figuring out how it should be done, given the nature of your suspicions.

Just in case something got lost in inter-galactic translation.

[OnTheOtherHand]

24. c'mon, really?

OK, you got me to look.

So, you are my "sidekick"? It is to laugh.

As for the other one, well, I think she should create a blog devoted exclusively to anti-Febble rants so we can all find them more easily. Just a thought. If she wants to create a separate one for me, hey, whatever.

OK, back to work....

Oh, by the way, I just simulated 400 elections with 15% fraud in 75% of 1250 precincts. The mean correlations were -0.17 for WPE, 0.15 for the original Febble function, 0.17 for tau, 0.22 for tau prime, and 0.15 for WLS tau prime. So, for nationwide analysis, the choice of measure just doesn't matter that much, for that extent of fraud. For state analysis -- say, 50 precincts -- it looks as if tau-prime without WLS is most likely to yield a "significant" non-zero correlation, but that is very rough. The main finding there is that, unsurprisingly, all the standard errors are so much higher regardless of measure that it is hard to distinguish fraud from bad luck. So, we really need an audit regime that actually works.

[mgr]

26. ! n/t

[Bill Bored]

33. Also depends on how you proceed once the first error is found.

I was talking to an elections official the other day and he said he'd have an automatic hand count for any race where the margin is < 0.5% OR if the results of the initial (3%) random audit don't match the machine counts perfectly. So, if his rules are adopted, all there is to worry about is the stuff in between. If the margin is 1%, and your initial audit is perfect, you have a 60% recount, and so on, based on the probabilities in Kathy's spreadsheet and the parameters we discussed upthread. I'm going back there where I used an actual example.

I also need to do a spreadsheet that looks at more possibilities.

I'm not sure if absolute margin or percentage margin makes a difference.

In calculating a percentage margin, we have to take into account the possibility of excessive undervotes that might actually turn out to be votes if an audit were conducted! This might mean that we can't just go by percentage margins to come up with a sliding scale. Does this make sense or am I getting ahead of myself?

[Time for change]

34. "If the margin is 1%, and your initial audit is perfect, you have a 60%

recount and so on".

Bill, I don't understand that. What do you mean by "and so on". I don't see the pattern that you're referring to by that phrase.

[Bill Bored]

39. The pattern is based on the hypergeometric distribution in the spreadsheet

and this takes into account the number of machines in the jurisdiction and the probability of finding a corrupted one with a given percentage of auditing.

Then based on the average number of votes per machine and the absolute vote margin, you determine how many to audit to ensure that the outcome is correct.

In other words, there's a sliding scale, which in this case uses a 60% audit to confirm that a 1% vote margin cannot be reversed.

[OnTheOtherHand]

42. you could put it another way

In your example upthread, the average number of votes per machine and the absolute vote margin pretty much cancel out, leaving: the vote margin equals the hypothetical percentage of corrupt precincts that you are concerned about. It's crude, but we can work with it. So you suggest that for a 1% vote margin, really concentrated fraud in 1% of precincts could switch the outcome. If we say 10 out of 1000 are corrupt, then a 3% count would only have about a 1-in-4 chance of catching the corrupt ones. Even a 10% count would have about 2 chances in 3. In order to get up to whatever you were counting as "certain" (looks like 99.99%, or one chance in 10,000 of not catching the fraud), you would have to count 600 precincts.

That doesn't seem like the right route to me. If there is actually anything approaching a 100% vote miscount or theft in a precinct, it should be pretty obvious. One shouldn't have to randomly sample ducks looking for the crows. (And one shouldn't have an audit regime that prevents you from challenging the crows unless they happen to turn up in the random sample.)

I'm not adamant about this -- if people are willing to pay for 60% or even full recounts in races with 1% margins, what the heck. And it may be easier than writing a regulatory implementation of "crows" (the Gold Star protocol offers one approach, allowing candidates to choose some precincts to recount would be another). But as a matter of statistical logic, banging the table for 60% just doesn't seem right to me.

[Febble]

49. In NH they looked for crows

but people complain it wasn't a random sample.

Which is why I bang on about custody of the ballots. Random has two purposes, and one of them is to make it unpredictable so no-one can jigger with the ballots before the recount.

But if you have anti-jigger protocols in place, then one reason for random drops out. If all the ballots are behind bars, then maybe it's OK to pick out the crows. But I'd like a closer look at those ducks too.

Lizzie

[OnTheOtherHand]

50. I'm already assuming a look at the ducks

and yes, that won't work unless the ballot custody is in order. But I think a truly random 3% (we can doodle over percentages) audit is the minimum. What I don't think is that we have to ratchet it up to 60% in order to deal with a race with a 1% margin.

I think we need a 3% recount even if anti-jigger protocols are in place. Otherwise, it is too easy to steal the election by, umm, shaving a bit off every duck's tail. No crows required.

(EDIT to paste over my egregious Latin error)

[OnTheOtherHand]

37. I'm flailing here -- what's the voting method?

If we're talking about a 3% hand count of manually completed op-scan ballots, then I'm afraid we're looking at many Ohios -- because people are not going to count the ballots EXACTLY like machines would (even the machines may not be entirely consistent), and election officials are going to be desperate to avoid "gratuitous" full recounts. (Or, if we want the standard to be voter intent, then people SHOULDN'T count the ballots exactly like machines would.)

I'm not saying this is the only thing to worry about -- and there are worse things than full manual recounts. But it's a real worry, I think. If we're counting automarked ballots, then clean and consistent counts seem much more likely.

What specifically are you worried about getting past a 3% random audit with a 1% margin of victory? It seems to me that if we think we can insist on 100% accuracy in the 3% audit, then very little could get past -- except for the small problem (heh) that then the audits are likely to get jobbed. So I'm wondering whether there is a need for some latitude there. E.g. if the audit uses a voter-intent standard, then I think it would be perfectly reasonable to conclude "OK, we've matched the machine count to within 0.1% and our voter-intent standard gives one candidate an additional 0.6%, but the margin in the race is 10 points, so -- given that there is no evidence of fraud in the count -- we are done," or "We matched the machine count exactly, but the voter-intent standard gave one candidate an additional 0.6%, and the margin is only 1.0% in favor of the other candidate, so this is iffy enough to at least keep on counting some more."

[Bill Bored]

40. Methods

Consider that the 2002 federal standard for both DREs and Optical Scanners is 1 error in 500,000 ballot positions. Since there are no precincts this large, it stands to reason that any scanner or DRE that is 2002 certified should count every ballot in a precinct correctly. If not, there needs to be some more auditing.

In other words, any precinct that's off by 1 vote or more is beyond the error rate spec of the voting system. So either the spec is bogus/useless, or there is a problem with the system. So you audit.

That said, I think if you want to give some latitude, you might do it e.g., by NOT counting overvotes where the write-in candidate's name is the same as the selected candidate on the ballot as a machine error in the case of Op Scans. Count the vote correctly in the end, but don't count the discrepancy as a machine error. It's actually a voter error, but because the intent is clear, the vote is counted. Because it's not a machine error, it doesn't trigger a full hand count.

There may be other examples which go to the machine's inability to determine voter intent rather than actual error in reading the ballot markings. But these could be spelled out exactly so that the totals can be reconciled and only machine error is taken into account when deciding whether additional auditing is needed.

If the system is supposed to be accurate to within 1 in 500,000 votes, it sets a pretty high standard that we should try to take seriously.

[OnTheOtherHand]

41. OK, machine error vs. voter intent is crucial here

I agree that op-scans should count properly marked ballots just about perfectly. (I wouldn't go so far as to conclude that if there is a one-off error in one precinct, then obviously the entire county should be recounted by hand.) But the ballots are not all going to be properly marked, and you will have recount staffs trying to infer "What Would the Machine Do?" I've done just enough work with mark-sense and handwriting recognition myself to have a healthy respect for how this might go wrong.

What I don't have is any actual knowledge of the likely rates of problematic "marginal" ballots, ones where even conscientious recounters could be legitimately uncertain what the machine result would be. You need a really clear protocol for dealing with the gray. Otherwise, you will either get lots of useless recounts, or you will get lots of subjective fudge that makes the doubters even madder. That's my fear, anyway. But if, say, Doug Jones tells me to chill out, I will.

[Febble]

43. Some thoughts....

Neil B Forzod, whoever he was, made a point (I think in a PM to me) that punchcard results at best will approximate the "true" count - he talked about chads flying about in clouds while the readers are working, I dunno if you guys have seen that.

So presumably, every time you run the thing, more chads get detached. His point was that that Florida 2000 was a statistical tie, because the margin between the candidates was within the margin of error of the counting technology. Without wanting to get into the rights and wrongs of Florida 2000 (which Gore morally won, whatever the legality was), it brought me up short, being trained to think of ballots as a population, and audits or exit polls as a sample. But unless machine counts (or even handcounts) are perfect, they are also a "sample" and have, literally "sampling error". Each time you count, you'll get a different answer, and the machine variance may or may not be greater than the handcount variance, and the mean of each may be different as well.

(And with our HCPBs in the UK we always get a different answer when there's a recount. If it's really really close, we just keep recounting until everyone is happy).

So maybe some aspect of the audit needs to be to establish the differences in mean and variance of total number of valid ballots "estimated" by machine versus hand. And though of course in principle there should be no "significant" differences between the proportion of votes for each candidate counted by the machine and by hand, even if the machines reject/accept more ballots, if voter education is a factor, it may be that if Dems are more likely to produce a ballot rejected by the machine but accepted by hand. If so, then Dems will be disenfranchised by the machines, as with the punchcards in Florida.

Maybe you guys have thought of all this - things strike me more slowly from over here, our voting system being so elementary.

[OnTheOtherHand]

46. I hadn't put it quite that way, but...

there is a small literature on estimating the variance in hand counts (which is indeed higher than 1 in 500,000!). I didn't even go into punch cards because they have that special problem of the ballot actually changing as you count it -- although not altogether unique, because it is easy to imagine op-scan ballots going from machine-readable to -unreadable as they are recounted, never mind (gulp) getting lost. That point aside, when NORC did its intensive recount of Florida ballots, it had a complicated protocol for examining intercoder reliability.

There Will Be Noise in every counting method. Any protocol that decrees that There Will Be No Noise is, I fear, liable to break in horrific ways -- which is in fact more or less what I think happened in Ohio last time around (although, based on other evidence, I don't think horrific harm was done). I think we are looking for a protocol that has some fault tolerance with respect to individual ballots while being as reliable as possible with respect to the election outcome. (And a protocol that demands an arbitrary commitment of excessive resources is not likely to be implemented -- which is why we are trying to get some variation upon a random audit to be workable. But it has to be rigorous, or else it is useless.)

There are people who know more about this stuff than I am ever likely to. Doug Jones is someone who already seems to know a lot about recounts and counting methods. I think it was some folks associated with the CalTech/MIT project who wrote one paper on machine versus hand recounts. The NORC stuff is all still on-line, at least it was a month or so ago. Those are just three leads that I happen to know about.

[eomer]

44. OTOH, how do spoilage rate differentials fit into this?

Is the goal of the audit only to detect machines not counting the way they are designed to?

Couldn't there be a simultaneous goal of detecting those cases where the intent standard gives a significantly different result than the machine standard (assuming both are accurately applied)?

In other words, two different trigger events:

• A conclusion that machines are malfunctioning
  
• A conclusion that there is a significant rate of voter intent differing from the count on properly functioning machines
  
  
  

[Febble]

45. Hey, just what I was thinking!

But you said it better.

[eomer]

47. Hey to you too.

:hi:

[OnTheOtherHand]

48. yes, I think that's right

I think Bill is arguing for a very stringent standard -- a sensitive trigger -- with respect to the inference of machine malfunction, and I more or less agree with that (although I suspect that the 1-in-500,000 standard is meaningless because it will be swamped by other forms of unavoidable error). If we find evidence of machine malfunction, then a very extensive recount is in order at least so that we learn as much as possible about what has gone wrong, even if the outcome of that particular election is not in question. On the most benign interpretation, we are in a period of experimentation with election systems, and we cannot learn from our experience if we don't look at it.

But there should also be a trigger based on voter intent. So the audit protocol should probably include reporting back on such cases; there would be no particular precinct-level standard on what 'error' rate would trigger a full recount, but there would be a county- and/or state-level standard, which could vary according to margin. Even if the audit didn't trigger a full recount, we would want to know the estimate of "voter intent miscount," so we could factor it into refining the audit protocol, improving ballot design, improving voter education, whatever. Again, to learn from our experience.

I'm not adamant about any specifics. It seems to me that the machine-miscount trigger should be more sensitive than the voter-intent trigger, but I could also argue for making them the same, at least initially.

[Bill Bored]

32. This can be done by testing every device but...

...auditing is another layer of protection.

There is really no way to test every device during an election, is there? If so, tell me how you would do it.

Without open source s/w, digitally signed and verified on every machine, it's always possible for there to be a ringer or ringers in the mix.

Testing may be sufficient to reveal malicious intent, but may not be sufficient to reveal all possible coding errors. If the clocks are set to the Election Day times when the tests are performed, and the tests really do test the whole system, and the chain of custody of the machines from that point on is air tight, then there's some degree of confidence.

But there could still be glitches, so you audit.

[Einsteinia]

20. Hmm, there's another scheme that's been worked

on for the past six months using the California Lottery balls. I'll post it later tonight.

[garybeck]

25. is this article reprinted from somewhere else? link please?

[FogerRox]

27. Garybeck-- no Rebecca emailed me --

State Senator Nia GIll in NJ has recently written a bill to mandate an VVPB audit. NJ had passed a VVPB law w/o an audit. SO Rebecca decided to write this lotto ball idea up. SO, in NJ this is very topical, right now.

[garybeck]

31. we are in the same boat here in VT but we don't

have a senator trying to write up something. we have the VVPB w/no audit. we should work together. I am going to pass Rebecca's email on to our VT group if it's OK with you.

[FogerRox]

38. yes thats why she included the contact info

" Feel free to distribute this email in its
ENTIRETY (including my name and email address) to interested parties."

Rebecca Mercuri.
[e-mail]

DO you have a copy of the BIll-- its on another thread-- I will send it to you if you want.

[Einsteinia]

51. It's is in the Section called

"precinct reconciliation"

http://www.califelectprotect.net/GSAP_16d.pdf

This has been in development over the past year. It looks great minds think alike--because we knew nothing of this, and I don't think Rebecca saw any of the multiple drafts of ours that we out. It's interesting how they overlap, but are quite different.