Research on Voting Systems and Ballot Design

Usability testing voting systems | Ballot design research | Work at NIST | University research centers

 

Disenfranchised by Design - Susan King Roth, Information Design Journal. This 1998 article is one of the most frequently cited papers on the usability of voting systems. It is also one of the few that pre-dates the 2000 US presidential election.

 

Usability testing voting systems

The NIST Voter Performance Protocol (VPP) was developed as part of the work on the US Voluntary Voting Systems Guidelines (VVSG). If adopted by the EAC in the next version of the guidelines, any voting system will have to meet conformance benchmarks for human performance using the system.

• Usability Performance Benchmarks White Paper (2007)

  • Validity Data
  • Repeatability Data
  • Benchmark Data
  • Test Ballot Specification
  • Instructions for Participants
  • Informed Consent Form
  • Post Test Satisfaction Questionnaire
  • Post Test Confidence and Background Questionnaire

Earlier white papers

• Preliminary Report on the Development of a User-Based Conformance Test for the Usability of Voting Equipment (DOC). User-Centered Design, Inc and NIST, 2006
• Usability testing of voting systems (PDF) A white paper on a summative testing approach, developed at a UPA2004 Workshop. It was included as an annex to the (abandoned) IEEE P1583 standard
• A Proposed Approach to Testing the IEEE Usability/Accessibility Standards (DOC) - John O'Hara's white paper on usability testing of voting systems. This work was incorporated into the 2002 VSS annex

Ballot design research

• Ballot and polling place design guidelines (AIGA Design for Democracy for the EAC)
• Banner Blindness in Ballot Design - Jakob Nielsen, Feb. 2007

 

Work at the National Institute of Standards and Technology (NIST)

Working for the Election Assistance Commission, NIST and the TGDC have done research into the design and usability of ballots and voting systems

• Review of relevant information from other voting studies (PDF, 94kb) Ginny Redish and Dana Chisnell, 2006
• Guidelines for Writing Clear Instructions for Voters & Poll Workers Ginny Redish, 2006
• Review of the gap between instructions for voting and best practice in providing instructions Ginny Redish, 2006
• NIST Human Factors Report (NIST SP 500-256), Sharon Laskowski, Marguerite Autry, John Cugini, Bill Killam, James Yen, 2004

 

Research Centers

Several university research centers are conducting ongoing research into voting systems. The projects listed here include work on usability and design.

 

ACCURATE (A Center for Correct, Usable, Reliable, Auditable, and Transparent Elections)

Research funded by the NSF includes work on the usability of voting systems, ballots and audit processes. (Project site)

• Stephen N. Goggin and Michael D. Byrne. An Examination of the Auditability of Voter Verified Paper Audit Trail (VVPAT) Ballots. In Proceedings of the 2nd USENIX/ACCURATE Electronic Voting Technology Workshop (EVT’07), 2007.
• Ka-Ping Yee. Extending Prerendered-Interface Voting Software to Support Accessibility and Other Ballot Features. In Proceedings of the 2nd USENIX/ACCURATE Electronic Voting Technology Workshop (EVT’07), 2007
• Everett, S. P. (2007). The Usability of Electronic Voting Machines and How Votes Can Be Changed Without Detection. Doctoral disseration, Rice University, Houston, TX
• Byrne, M. D., Greene, K. K., & Everett, S. P. (2007). Usability of voting systems: Baseline data for paper, punch cards, and lever machines. Human Factors in Computing Systems: Proceedings of CHI 2007 (pp. 171-180). New York: ACM
• Greene, K. K., Byrne, M. D., & Everett, S. P. (2006). A comparison of usability between voting methods. In Proceedings of the 2006 USENIX/ACCURATE Electronic Voting Technology Workshop
• Everett, S. P., Byrne, M. D., & Greene, K. K. (2006). Measuring the usability of paper ballots: Efficiency, effectiveness, and satisfaction. Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting, (pp. 2547-2551). Santa Monica, CA: Human Factors and Ergonomics Society

CAPC (The Center for American Politics and Citizenship) Research on Voting Technology and Ballot Design

A project at the University of Maryland conducts research into elections, including usability issues with the HCIL (Project site)

• R. Michael Alvarez, Ballot Design Options
• John Bremer, Ballot Design in an Electronic Environment - Lessons from the Online Market Research Industry
• Frederick G. Conrad, Usability and Voting Technology
• Paul S. Herrnson, Richard G. Niemi, Scott Richman, Characteristics of Optical Scan and DRE Voting Equipment: What Features Should Be Tested
• Susan King Roth, Human Factors Research on Voting Machines and Ballot Designs: An Exploratory Study
• Michael W. Traugott, Testing Alternative Hardware and Ballot Forms
• Electronic Voting System Usability Issues - a report by Ben Bederson, Paul S. Herrnson, and Richard G. Niemi summarising the issues needing research, including a good bibliography of other research.
• An Evaluation of Maryland's New Voting Machines - CAPC and HCIL exit poll research on voter comfort and trust in new electronic voting machines (2002)
• Usability Review of the Diebold DRE System for Four Counties in the State of Maryland -Ben Bederson, Paul S. Herrnson

Electronic Voting Systems - University of Maryland HCIL

Ben Bederson's work on the usability of voting systems, and work on "Zoomable User Interfaces" for electronic ballots (Project site)

 

 

Auburn University Human-Centered Computing Lab

The Lab has created Prime III, an electronic voting system that offers a secure, open-source, multimodal electronic voting system that delivers the necessary system security, integrity and user satisfaction safeguards in a user friendly interface that accommodates all people regardless of ability. Prime III's implements a Universal Design. (Project site)

• Prime III VoComp Team wins Best Usability Metric and Best Human Factors on July 18, 2007.
• News report on Prime III YouTube

 

The Caltech-MIT Voting Technology Project
As part of the Caltech-MIT Voting Project, Ted Selker has worked on the design of voting systems, and methodologies for usability evaluation. (Project site)

• A Methodology for Testing Voting Systems Ted Selker, Elizabeth Rosenzweig, Anna Pandolfo, UPA Journal of Usability Studies, 2006, Vol. 2, No. 1: 7-21
• "A Day of Poll Watching, Reno and Sparks, Nevada"
  Ted Selker UPA User Experience Magazine, 2005, Vol. 4, No. 1
• "Comparative Voting Performance of Reading Disabled Voters"
  Jonathan A. Goler, Edwin J. Selker, and Lorin F. Wilde
  Interacting with Computers
• Augmenting Voting Interfaces to Improve Accessibility and Performance, Jonathan A. Goler, Ted J. Selker (unpublished, 2006)
• Who does better with a big interface? Improving Voting Performance of Reading Disabled Voters, Ted Selker and Jonathan Goler (unpublished, 2005)
• Ballot Design Prototypes - example ballots and applets. Ted Selker with Jon Goler, Bruce DiBello, Shawn Sullivan, Elizabeth Herbert