Staff & Consultant Reports Up on SoS Website CA for wares for Mar. 1

[diva77]

hearing...

Download at:


http://www.ss.ca.gov/elections/elections_vs.htm


Agenda:

PUBLIC HEARING ON PROPOSED CERTIFICATION OF VOTING SYSTEMS AGENDA
LOCATION Office of the Secretary of State 1500 11 th Street 1 st Floor – Auditorium Sacramento, California 95814
MEETING DATE AND TIME March 1, 2006 10:00 a.m. Pursuant to Elections Code section 19204, notice is hereby given that a public hearing will be held to give interested persons an opportunity to express their views regarding the proposed certification of the voting systems identified below.

1. ES&S - Unisyn/InkaVote PBC
a. Unisyn Election Management System
b. InkaVote PBC
2. HART INTERCIVIC - System 6
a. Ballot Now
b. BOSS
c. eCM Manager
d. eScan
e. eSlate/DAU
f. VBO
g. JBC
h. Rally
i. Servo
j. Tally

3. SEQUOIA VOTING SYSTEMS - AVC Edge Voting System Version 5.0
a. WinEDS
b. AVC Edge Model 1 & 2 with Audio Box & VeriVote
c. Card Activator
d. HAAT
e. Optech 400-C
f. Optech Insight
g. Optech Eagle
h. Memory Pack Reader (MPR)
i. Edge Cartridge Reader/Programmer
j. Sycard PCCestend 70A Cardbus Extender

4. POPULEX CORPORATION - Digital Paper Ballot Voting System
a. Populex Election Management System
b. Populex Slate

[diva77]

1. Any talking pts. would be appreciated!!

[Wilms]

4. One talking point is that these reports are WAY overdue.

The SoS is supposed to make this stuff available ahead of the meeting by, I think, 30 days.

Another is that there aren't any reports for the Populex equipment that's also list for action this week.

The HART System - Consultant's Report is listed, but linkless. Perhaps a typo.

[nicknameless]

6. You're right. Reports were due in January (30 days prior to the hearing).

McPherson "welcomes" public comment regarding systems -- about which he won't release necessary information.

God knows what he plans to dump on us next.

[nicknameless]

8. 5 days instead of 30.

I guess we should be grateful to get that many.

Mr. Hates-the-Public (Bruce McDannold) is now the Acting Director of the new Office of Voting System Technology Assessment.

[Wilms]

12. Did you catch that?

the new Office of Voting System Technology Assessment.

How did McDannold get that job? Was he there before Shelly left?

[nicknameless]

22. I don't know about McDannold's background,

but I'm sure that he came in when McPherson did.

He's very pro-Diebold and very anti-public opinion. I've read on DU that he's McPherson's only contact with the public. Sounds like a body guard with corrupt political leanings.

I've also read that he's working on a "Centralized Voter Registration system" for California.
Wonder if ChoicePoint is helping him scrub Dems from the voter rolls.

[EFerrari]

11. Does the Elections Code say 30 days?

And, isn't this now a pattern of Mr. MIA McPherson?

[Wilms]

13. I'm not sure that it's law or "code" or just a reg.

Also, everything has been running late in this process. That doesn't make ok, but it's good to keep in mind.

[nicknameless]

21. Not sure where the requirement is stipulated.

Similarly, he's required to give 30 days notice before public hearings. He neglected to do that last November too.

He's also "legally late" in turning over documents formally requested by Senator Bowen.
He's "legally late" in complying with the 19202 Request.

He's basically pulling a Cheney. He doesn't want to turn over documents or comply with the law, so he won't.

[Wilms]

2. Staff Review and Analysis - ES&S Unison

A few notes that stood out.

Page 4

1. Federal Testing

Wyle Laboratories has successfully completed federal qualification testing of the
InkaVote Precinct Ballot Counter, v. 1.10 to the 2002 Federal Voting System Standards.
We have received a copy of the draft report from that testing, dated December 20, 2005.
A final report must be received from Wyle upon report acceptance from NASED and
prior to State certification of this system.

Ciber, Inc. has successfully completed federal qualification testing of the Unisyn EMS, as
well as end-to-end testing of the entire system, to the 2002 Federal Voting System
Standards. We have received a copy of the draft report from that testing, dated February
21, 2006. A final report must be received from Cyber upon report acceptance from
NASED and prior to State certification of this system.
Federal qualification of the system by NASED is still pending and must be issued prior to
State certification of this system.

Page 6

“Pre-Volume Testing” Results

Because the vendor could not produce the required fifty production PBC units, a fullscale
volume test in accordance with the established Secretary of State protocols has not
yet been conduced. As noted above, at the request of Los Angeles County, the Secretary
of State conducted a “pre-volume” test on the five available PBC units on January 27th at
the ILTS offices in Carlsbad, CA in an attempt to get an initial appraisal of the accuracy
and reliability of the PBC.

.pdf
http://www.ss.ca.gov/elections/voting_systems/ess_staff_report.pdf

[Wilms]

3. Consultant Report (Steve Freeman) - ES&S/InkaVote Precinct Ballot Counter

Page 5

6. Software Loader is a utility for loading software onto the PBC. Its use simplifies,
but is not essential to, loading the software. Due to the fact that it has not been
submitted to the ITAs for review and testing, it has been excluded from this
application for certification.

7. Transfer Media are USB memory devices that carry results from the precinct ballot
counters to the Vote Converter and Tabulator applications. During the course of the
test, there were several incidents where results could not be recovered from the
Transfer Media. In order to remedy the problem, these Transfer Media need to be
“purged” with a special program that is not a part of the package submitted for
certification. During the volume testing, this could become a potential problem.

8. The secrecy sleeve for the ballot is too short and can expose votes. This is
especially true for visually disabled voters, who cannot see what part of the ballot is
exposed.

9. The ADA booth only supports those who require an audio ballot. There is no visual
display of the ballot in that booth. The system does not provide a sip and puff
interface.


Page 6

Installation of the Trusted Build PBC programs was done by a utility Software Loader
that was not part of the Federal or State certification package. This program is a
warehouse/manufacturer level utility and is not considered as needed as a deliverable
item. During installation of the trusted build, we could not easily validate that it was
loading the witnessed build programs and not adding/modifying others. Both it and the
Election Loader access the Software Verification program on the PBC unit to verify the
correct files are installed. The process uses electronic hash signatures (MD5 hashes) to
verify the current installed software and to identify what programs need to be replaced for
the update. We used a manual procedure to duplicate the process for installing updates
and checking versions installed. We discovered that some older files were still on the
system, which caused a problem for the Software Loader since it did not detect a critical
file was out of date or recognize programs which were no longer needed. The Software
Loader also may be used to perform a simple verification that the correct version files are
installed based on an encrypted CD copy. The errors in recognizing and deleting the
unnecessary programs and not recognizing another file that needed to be updated
indicates more work is needed on this utility.


.pdf
http://www.ss.ca.gov/elections/voting_systems/ess_consultant_report.pdf

[Wilms]

5. WINEDS/AVC EDGE/INSIGHT/400C/EAGLE

A lot of errors are reported for the volume tests. Consider reading it.

They've recommended NOT certifying the Optech Eagle.


Page 4

2. Upon initial testing of the system, one of the Edge Model I units was found to be
out of calibration. The vendor agreed that Use Procedures for the system should
require calibration at the opening of the polls.

4. During testing, it was noted that if a voter mistakenly enters one of the certified
candidates as a write-in vote, the WinEDS feature for resolving write-ins cannot
resolve that write-in to the candidate. The system’s Use Procedures should
specify how to handle this. Future versions of the system should address this
problem.

5. The audio ballot mode presumes the voter is blind and always blanks the Edge
video display. This is the interface that is used for mobility-impaired voters using
the sip-puff interface. This means that these voters, or sighted voters who prefer
the audio ballot instructions for language support, must depend exclusively on
audio cues and instructions to navigate and vote their ballot. Additionally, the
VeriVote does not give these voters a chance to confirm or reject their ballot on
the assumption that they are blind. Instead, it simply prints the ballot on the paper
audit trail and finalizes the ballot, scrolling it out of view. This should be
corrected in future versions of the system.

6. During testing, it was noted that many of the WinEDS screens featured a print
button that did not work. It should be noted that the report printing function on
the database appears to work correctly.

7. During the reconciliation of the test primary election, it was discovered that on the
Eagle one ballot was incorrectly read in one contest. That same ballot had been
correctly read on both Insights and the 400-C. Visual inspection of all ballots for
that contest could not determine that any of the marks had been made incorrectly.
The vendor suggested that this was probably due to an incorrect composition of
the ink in the pen that had been used by the vendor to mark the test ballots.
Because the Eagle uses a different, older read-head technology, it is subject to
these types of errors. This error rate was one ballot in 439 (0.23% of the ballots),
or one vote of 2,687 votes cast (0.0372%) in the test election on the Eagle.

8. The Sequoia WinEDS system currently certified for use in California has a
condition on that certification that WinEDS couldn’t be used in a California
statewide primary election. This condition was put into place because it was
discovered, during testing of that system, that WinEDS could not separately
report the vote result for non-partisan voters who chose to vote in partisan
contests, where allowed to do so by the party, without using a utility that had not
been federally tested and qualified as part of the system. During certification
testing of this system, Sequoia demonstrated that they could track and accurately
report this breakdown of vote results by defining these voters as separate political
parties within the system. (e.g. Republican-Non Partisan, Democrat-Non Partisan
and American Independent-Non Partisan) However, with this approach the
system cannot automatically aggregate the vote results together with the partisan
votes within the same contest. (e.g. Democrat and Democrat-Non-Partisan votes
for President combined.) This must be derived manually or with some process
external to the system. The vendor has reported that they are currently working
on a modification to the system that will resolve this issue.

http://www.ss.ca.gov/elections/voting_systems/sequoia_staff_report.pdf

[nicknameless]

7. K&R

[nicknameless]

9. File won't download: ESS System - Proposed Use Procedures

Get error message: "404 - Page Not Found"

[diva77]

10. I had the same problem with this one

[Wilms]

16. Same here. Don't know if it was there to begin with.

In fact, I really don't know if the now gone links were any good.

I'm also wondering if the removed links were the raw test data.

[Wilms]

14. Well they've already pulled down a number of the reports.

The following were there early this morning. They're gone now. Essentially, all the Volume Testing reports.

SEQUOIA System - Edge I Volume Test Error Reports
SEQUOIA System - Edge II Volume Test Error Reports
SEQUOIA System - Insight Volume Test Error Reports
SEQUOIA System - Insight Volume Test Error Reports


HART System - eSlate Volume Test Error Reports
HART System - eScan Volume Test Error Reports


ESS System - PBC Volume Test Error Reports

http://www.ss.ca.gov/elections/elections_vs.htm

[JimDandy]

15. Did anyone download these Error Reports?

I bookmarked this thinking I'd go back and read everything tonight!
JD

[Wilms]

17. I didn't.

:banghead:

See my reply #16, upthread.

http://www.democraticunderground.com/discuss/duboard.php?az=show_mesg&forum=203&topic_id=414478&mesg_id=414619

[Wilms]

18. It's back up now.

Though who knows what happened, meantime.

[JimDandy]

20. Thanks! n/t

[Wilms]

19. Hart eSlate DRE & eScan OpScan Perform Terribly - DREs Rec'd, OpScans Not

Hart eSlate DREs & eScan OpScans had a terrible time with the volume test, it seems. Yet the DREs were recommended.

Based on one federal spec alone, a Mean Time Between Failure (MTBF) rate of 163 hours, I don't think the the Hart eSlate qualifies. (As a side note, that spec is woefully inadequate as it is.)

The footers of this document were mislabeled so ignore them.


(Hart) Volume Testing Results

Page 13 of 38

eSlate Volume Test

The Secretary of State staff and technical consultants Steve Freeman and Paul Craft
conducted the volume test of the Hart eScan and the eSlate on February 21st and 22nd,
2006, at the Orange County Registrar of Voters’ warehouse.

snip

(For the test they set up) one hundred eSlates equipped with the VBO AVVPAT.

snip

A total of thirty-four errors were logged during the eSlate volume test. Twenty-four of
those errors were related to the actual performance of the eSlate equipment.

During testing, four eSlate units got stuck in some kind of an error cycle that could not be
exited. Generally this occurred when trying to cast the ballot. At the time of saving the
ballot, the VBO would print "ballot cancelled" repeatedly on the audit record and the
screen controls would freeze. After rebooting the machine, the same symptoms
reappeared on the next attempt to cast a ballot. The vendor initially hypothesized that a
defective power adaptor or connector for the power adaptor caused the problem. After
swapping out the adaptors, two machines were retested and exhibited the same problems.

All four machines were taken out of service and not tested further. It should be noted that
all four of these exhibited these symptoms within the first five ballots, indicating the
possibility that this is a manufacturing problem that can be detected with robust
acceptance testing. We are awaiting further explanation from the vendor as to the cause
of this problem.

There were seventeen incidents of problems with the VBO AVVPAT printer. In one
incident, the eSlate locked and displayed the error message “EVBO-101” The vendor
explained this error code represents a printer battery error. This VBO was opened and
the batteries were disconnected, after which voting resumed normally on the eSlate. In a
second incident, the eSlate locked with the printer error "EVBO-103." The vendor
explained this error code represents a printer communication error. This VBO was
swapped out and taken out of service in accordance with the Use Procedures for this
system.

In the remaining fifteen occasions of VBO errors, the eSlate locked up with the screen
displaying cryptic error message: "Printer error EVBO-102". Per the vendor, this error
code represents an out-of-paper condition in the VBO. In two cases (incidents #32 &
34), this was correct - the VBO was out of paper. In the thirteen remaining instances, this
was a false out-of-paper error. In each case, the vendor was able to correct this by
removing and disassembling the VBO, and then advancing the paper, after which voting
resumed normally on the eSlate without further errors. In all such cases, the error was
triggered within the first ten ballots. It should be noted that the VBO is designed to not
be opened in the polling place, and if any issues arise, to be swapped out and replaced
with a new unit.

Overall, there were seventeen instances out of 100 eSlates that would have required the
replacement of a VBO in a polling place. Should this system be certified, use procedures
should require an appropriate number of VBO replacements to be on hand in each polling
place.

The remaining ten incidents were all attributed to human error:

• In one case, the assigned activation code for a ballot timed out before the voter
could use the code.

• In five cases, the voter was issued an activation code for the wrong ballot style.

Page 14 of 38

• In four cases, the error was attributed to voter confusion on the test process or
operation of the eSlate.

Based on the volume test results for the eSlate, the Office of Voting Systems
recommends that if this system is certified, the Use Procedures should specify:

• robust acceptance testing procedures that will identify any defective units subject
to the error cycle/lockout problem that caused the four units in the volume test to
be taken out of service

• an adequate supply of replacement VBO units on hand at each polling place to
ensure that there is no downtime of eSlates due to VBO failures

Further, as soon as possible the vendor should a) resolve the problem that causes the
VBO to falsely indicate a low paper condition at the start of the role, and b) replace the
cryptic error codes on the eSlate with plain-text, understandable error messages that
communicate to the voter and the poll worker what the problem is, how to resolve the
problem and the status of that voter’s ballot. These must be accomplished as soon as
possible to enhance voter confidence in this system.

http://www.ss.ca.gov/elections/voting_systems/HART-Sys6_2005-11_StaffReport_FINAL.pdf