Want to see a wicked problem? Just look at designing ballots. This is from a talk I gave in 2012 at UPA and ACM meetings around the country.

1. Democracy is a design problem
How changes in design change
the outcome of elections
Dana Chisnell
1

2. Democracy is a design problem

7. Design affects world peace.

8. Why voting is harder than it seems
8

9. Legislation
9

10. Voting
Systems
10

11. Costs
In Maryland, cost per voter
increased by 179% going
from paper-based systems
to touchscreens
11

12. Costs
Paper has costs, too
printing
postage
storage
12

13. History
13

14. Skills
14

15. There are 3,247 counties (or equivalent) in the
US
Secretaries of state issue ballot templates
Counties design, print, and distribute ballots
There are many more elections every year than
you realize
15

16. Voting is still hard
16

17. Poor ballot design
frustrates voters
undermines confidence
contributes to Election Day problems
17

18. All voters are affected
thousands of votes are lost or miscast
all voting technologies are affected
greater risk for some groups of voters
18

19. Instructions Questions
Contests
19

20. Classic problems
splitting contests into different columns
placing different contests on the same screen
placing response options on both sides of names
using arrows rather than bubbles on paper ballots
leaving in disqualified candidates
20

21. Better Ballots: Re-designed Ballot, Kewaunee County
Keep contests
in same
column

22. Better Ballots: Sarasota County, FL (2006)

23. Better Ballots: Sarasota County, FL (2006)

24. Better Ballots: Sarasota County, FL (2006)

25. Classic problems
inconsistency in format and style
not using shading, bold text
not writing short, simple instructions
placing instructions too far away
not telling voters how to correct paper ballots
25

26. Better Ballots: Los Angeles County
Format
consistently

27. Top 10 design principles
Use lowercase letters Use accurate instructional
illustrations
Avoid centered text
Use informational icons (only)
Pick one sans-serif font
Use contrast and color
Use big enough type functionally
Support process and Decide what’s most
navigation important
Use clear, simple language
27

28. Before and after
28

29. Cedar County:
May 2006 for ES&S
optical scanners

30. Cedar County:
November 2006 for
ES&S optical scanners

31. TEST BALLOT
GENERAL ELECTION
NOVEMBER 4, 2008
MARIN COUNTY, CALIFORNIA
INSTRUCTIONS TO VOTERS:
• Use only the marking pen provided, or a ball-point pen with black ink.
• To vote for a candidate, completely blacken the oval next to your choice, like this . Do not vote for more than the
number of candidates to be elected.
• When there are two (2) or more candidates to be elected for the same office, blacken the ovals next to each of your choices.
• To vote on a measure, blacken the oval next to the word YES or next to the word NO.
• Write-in candidates: To vote for a qualified write-in candidate, blacken the oval next to the blank line for the office and write
the name of the person on the line. Do not vote for both a write-in candidate and a candidate whose name is on the
ballot for the same office.
• All distingushing marks on the ballot are forbidden and void the ballot.
• If you make a mistake, tear your ballot, or want to change your vote, return it to the election official and obtain another.
FEDERAL DISTRICT PROPOSITION 3: CHILDREN’S
HOSPITAL BOND ACT. GRANT
PRESIDENT AND VICE PRESIDENT MARIN HEALTHCARE DISTRICT, PROGRAM. INITIATIVE STATUTE.
Vote for One Party (1) DIRECTOR Authorizes $980,000,000 in general
Vote for no more than Two (2) obligation bonds for construction,
BOB BARR Libertarian expansion, remodeling, renovation,
WAYNE A. ROOT furnishing and equipping of eligible
JOHN MCCAIN Republican ARCHIMEDES RAMIREZ children’s hospitals. Fiscal Impact: State
SARAH PALIN Neurosurgeon
cost of about $2 billion over 30 years to pay
CYNTHIA MCKINNEY Green PETER CHRISTIAN ROMANOWSKY
off both the principal ($980 million) and
ROSA CLEMENTE Minister/Entertainer/Artist
interest ($933 million) costs of the bonds.
ALAN KEYES American Independent HARRIS (HANK) SIMMONDS
WILEY S. DRAKE, SR. Retired Physician Payments of about $64 million per year.
RALPH NADER Peace And Freedom SHARON J. JACKSON
MATT GONZALEZ Incumbent
BARACK OBAMA Democratic FRANK PARNELL YES NO
JOE BIDEN Physician/Businessman
PROPOSITION 4: WAITING PERIOD AND
PARENTAL NOTIFICATION BEFORE
UNITED STATES REPRESENTATIVE, TERMINATION OF MINOR’S
6TH DISTRICT PREGNANCY. INITIATIVE
Vote for One (1) MEASURES SUBMITTED CONSTITUTIONAL AMENDMENT.
TO THE VOTERS Changes California Constitution, prohibiting
abortion for unemancipated minor until 48
JOEL R. SMOLEN Libertarian hours after physician notifies minor’s
Registered Investment Advisor STATE
parent, legal guardian, or, in limited cases,
MIKE HALLIWELL Republican PROPOSITION 1A: SAFE, RELIABLE HIGH- substitute adult relative. Provides an
College Professor
SPEED PASSENGER TRAIN. exception for medical emergency or
LYNN WOOLSEY Democratic To provide Californians a safe, convenient,
U.S. Representative parental waiver. Fiscal Impact: Potential
affordable, and reliable alternative to driving and unknown net state costs of several million
high gas prices; to provide good-paying jobs dollars annually for health and social
and improve California’s economy while services programs, court administration,
STATE reducing air pollution, global warming and state health agency administration

32. NOVEMBER 4, 2008
MARIN COUNTY, CALIFORNIA
INSTRUCTIONS TO VOTERS:
• Use only the marking pen provided or a ball point pen with black ink.
• To vote for a candidate, completely blacken the oval to the left of the name, like this . Where two or more candidates
for the same office are to be elected, blacken the oval to the left of each of your choices.
• To vote on a measure, completely blacken the oval next to YES or NO.
• To vote for a qualified write-in candidate, completely blacken the oval next to the blank line for that office and write the
name of the person on the line.
• If you make a mistake, damage your ballot, or you want to change a vote, take your ballot to an election worker and get
another. All distinguishing marks are forbidden and void the ballot.
FEDERAL DISTRICT PROPOSITION 2: STANDARDS FOR
CONFINING FARM ANIMALS.
PRESIDENT AND VICE PRESIDENT MARIN HEALTHCARE DISTRICT, INITIATIVE STATUTE.
Vote for One Party (1) DIRECTOR Requires that certain farm animals be
Vote for no more than Two (2) allowed, for the majority of every day, to
BOB BARR Libertarian fully extend their limbs or wings, lie down,
WAYNE A. ROOT stand up and turn around. Limited
JOHN MCCAIN Republican ARCHIMEDES RAMIREZ exceptions apply.
SARAH PALIN Neurosurgeon
Fiscal Impact: Potential unknown
CYNTHIA MCKINNEY Green PETER CHRISTIAN ROMANOWSKY
decrease in state and local tax revenues
ROSA CLEMENTE Minister/Entertainer/Artist
from farm businesses, possibly in the range
ALAN KEYES American Independent HARRIS (HANK) SIMMONDS
WILEY S. DRAKE, SR. Retired Physician of several million dollars annually. Potential
RALPH NADER Peace And Freedom SHARON J. JACKSON minor local and state enforcement and
MATT GONZALEZ Incumbent prosecution costs, partly offset by increased
BARACK OBAMA Democratic FRANK PARNELL fine revenue.
JOE BIDEN Physician/Businessman
YES NO
UNITED STATES REPRESENTATIVE, PROPOSITION 3: CHILDREN’S
6TH DISTRICT HOSPITAL BOND ACT. GRANT
Vote for One (1) MEASURES SUBMITTED PROGRAM. INITIATIVE STATUTE.
TO THE VOTERS Authorizes $980,000,000 in general
obligation bonds for construction,
JOEL R. SMOLEN Libertarian expansion, remodeling, renovation,
Registered Investment Advisor STATE furnishing and equipping of eligible
MIKE HALLIWELL Republican PROPOSITION 1A: SAFE, RELIABLE HIGH- children’s hospitals.
College Professor
SPEED PASSENGER TRAIN BOND ACT. Fiscal Impact: State cost of about $2

33. Progress
Lessons from crises

34. Sarasota
34

35. No testing =
lost votes
King Co. 2009 undervotes were disproportionate
to other counties
resulted in legislation about ballot design
35

36. Ensuring
franchise
Minnesota SoS tests vote-by-mail after 2008
senate recounts
Before

37. Ensuring
franchise
Minnesota SoS tests vote-by-mail after 2008
senate recounts
After

38. DEMONSTRATION BALLOT [Insert Chinese Translation]
SAMPLE BALLOT FOR THE GENERAL ELECTION - City of New York - County of Kings - November 2, 2010 [Insert Korean Translation]
A B C D E F G RDH H WRITE-IN
CANDIDATO POR
1. BEFORE VOTING
THIS BALLOT,
Democratic Republican Independence Conservative Working Families Green Rent is 2 Damn Libertarian ESCRITO
[Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] High [Chinese]
PLEASE READ THE
[Korean] [Korean] [Korean] [Korean] [Korean] [Korean] [Chinese] [Korean] INSTRUCTIONS ON
[Korean]
THE OTHER SIDE OF
1A 1B 1C 1D 1E 1F 1G 1H WRITE-IN THIS BALLOT.
Andrew M. Carl P. Andrew M. Carl P. Andrew M. Howie Jimmy Warren
1. ANTES DE EMITIR
Cuomo Paladino Cuomo Paladino Cuomo Hawkins McMillan Redlich
[Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] SU VOTO CON
[Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] ESTA PAPELETA,
[Korean] [Korean] [Korean] [Korean] [Korean] [Korean] [Korean] [Korean] SÍRVASE LEER LAS
[Korean] [Korean] [Korean] [Korean] [Korean] [Korean] [Korean] [Korean] INSTRUCCIONES EN
EL DORSO DE ESTA
PAPELETA.
Electors for Robert J. Gregory J. Robert J. Gregory J. Robert J. Gloria Alden
Governor and Duffy Edwards Duffy Edwards Duffy Mattera Link
[Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese]
Lieutenant
New York sued
[Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese]
Governor [Korean] [Korean] [Korean] [Korean] [Korean] [Korean] [Korean]
[Korean] [Korean] [Korean] [Korean] [Korean] [Korean] [Korean]
2. BALLOT PROPOSALS
[Spanish]
ARE LOCATED ON THE
1 OTHE SIDE OF THIS
[Chinese]
K J ✺ I BALLOT.
2. LAS PROPUESTAS DE
[Korean] Taxpayers Freedom Anti-Prohibition
PAPELETA APARECEN
[Chinese] [Chinese] [Chinese]
for usability
[Korean] [Korean] [Korean] EN EL DORO DE ESTA
Vote Once PAPELETA.
Vote una vez 1K 1J 1I
[Chinese]
[Korean] Carl P. Charles Kristin M.
Paladino Barron Davis
[Chinese] [Chinese] [Chinese]
[Chinese] [Chinese] [Chinese]
[Korean] [Korean] [Korean]
[Korean] [Korean] [Korean]
problems
MARKING
INSTRUCTIONS:
Gregory J. Eva M. Tanya INSTRUCCIONES
Edwards Doyle Gendelman DE MARCADO:
[Chinese] [Chinese] [Chinese]
[Chinese] [Chinese] [Chinese]
[Korean] [Korean] [Korean]
Correct Mark:
[Korean] [Korean] [Korean]
Marca Correcta:
A B C D E F G
Testing
Comptroller Democratic Republican Independence Conservative Working Families Green Libertarian
[Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese]
[Spanish] [Korean] [Korean] [Korean] [Korean] [Korean] [Korean] [Korean]
[Chinese]
2 [Korean] 2A 2B 2C 2D 2E 2F 2G WRITE-IN
Vote Once Thomas P. Harry Harry Harry Thomas P. Julia A. John
Vote una vez DiNapoli Wilson Wilson Wilson DiNapoli Willebrand Gaetani
[Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese]
[Korean] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese]
[Korean] [Korean] [Korean] [Korean] [Korean] [Korean] [Korean]
Training
[Korean] [Korean] [Korean] [Korean] [Korean] [Korean] [Korean]
A B C D E F G ✺
Attorney General Democratic Republican Independence Conservative Working Families Libertarian Freedom
[Spanish] [Chinese]
[Korean]
[Chinese]
[Korean]
[Chinese]
[Korean]
[Chinese]
[Korean]
[Chinese]
[Korean]
[Chinese]
[Korean]
[Chinese]
[Korean]
[Chinese]
3 [Korean] 3A 3B 3C 3D 3E 3F 3G WRITE-IN
Vote Once Eric T. Dan Eric T. Dan Eric T. Carl E. Ramon J.
Vote una vez
[Chinese]
Schneiderman Donovan Schneiderman Donovan Schneiderman Person Jimenez
[Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese]
[Korean]
[Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese]
[Korean] [Korean] [Korean] [Korean] [Korean] [Korean] [Korean]
[Korean] [Korean] [Korean] [Korean] [Korean] [Korean] [Korean]
A B C D E F G RDH H
Democratic Republican Independence Conservative Working Families Green Rent is 2 Damn Libertarian
[Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] High [Chinese]
38
[Korean] [Korean] [Korean] [Korean] [Korean] [Korean] [Chinese] [Korean]
[Korean]
4A 4B 4C 4D 4E 4F 4G 4H WRITE-IN
United States Kristen E. Joseph J. Kristen E. Joseph J. Kristen E. Cecile A. Joseph John
Gillibrand DioGuardi Gillibrand DioGuardi Gillibrand Lawrence Huff Clifton
Senator [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese]
[Spanish] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese] [Chinese]
[Chinese] [Korean] [Korean] [Korean] [Korean] [Korean] [Korean] [Korean] [Korean]
[Korean] [Korean] [Korean] [Korean] [Korean] [Korean] [Korean] [Korean]
4 [Korean]
K J I
Vote Once
Vote una vez
[Chinese]
[Korean]
Tax Revolt Taxpayers Anti-Prohibition
[Chinese] [Chinese] [Chinese]
[Korean] [Korean] [Korean]
4K 4J 4I
Bruce Joseph J. Viva
Blakeman DioGuardi Morgan
[Chinese] [Chinese] [Chinese]

39. 39

40. Recommended
03/08/2011$03:47:00pm$
You'filled'in'too'many'ovals'in'3'contests.'
These'votes'will'not'count:'
e d
t tl
s e
Return'your'ballot'U n Ignore'message.'Cast'ballot'
Get$your$ballot$back$to$correct$it.$Ask$a$ Ignore$this$message$and$cast$your$
poll$worker$for$a$new$one.$ ballot$with$votes$that$will$not$count.$
Return'Ballot' Cast'
40

41. Research and
standards
Election Assistance Commission and
National Institute of Standards and
Technology
Voluntary Voting System Guidelines -
VVSG 2.5

42. Brennan
Center
Design for
Democracy
42

43. Field Guides series
Research Research SOP and Usability Research
commissioned by commissioned by and Civic Life commissioned by
EAC, book NIST, book Project, book NIST, book
funded by funded by funded by funded by
Kickstarter Kickstarter Kickstarter Kickstarter

44. Coming soon
Effective election Delivering useful Effective design Designing multi-
department voter education for vote-by-mail language ballots
web sites
Research Research Funding yet to be Funding yet to be
sponsored by sponsored by secured secured
John D. & John D. &
Catherine T. Catherine T.
MacArthur MacArthur
Foundation Foundation

45. Field Guides
To Ensuring
Voter Intent
civicdesigning.org
/fieldguides

46. Be an
election worker

47. Observe
elections
HUGE PROJECT
on poll worker security!
We need participants
everywhere
47

48. Volunteer to
proofread

49. Use the
accessible
system
http://trace.wisc.edu/voting/ 49

50. Join our
projects
AIGA Design for
Democracy
UPA Usability in
Civic Life

51. Epilogue
51

52. Why can’t I vote on the internet?
Secrecy
Error tolerance
Security
Disagreement about the level of risk
52

53. Why aren’t all voters using the same
voting system?
53

54. The 10th Amendment
It’s a states rights issue.
54

55. Dana
Chisnell
dana@usabilityworks.net
www.usabilityworks.net
415.519.1148
@danachis