The document summarizes the CUTGroup (Civic User Testing Group), which pays Chicago residents to test civic apps. It discusses the origins of CUTGroup in helping make apps more usable. CUTGroup provides UX testing, develops digital skills in testers, and fosters community engagement. Tests involve recruiting diverse residents, collecting feedback, and sharing results to help developers improve apps. Examples provided include tests of the Foodborne Chicago illness reporting app and OpenStreetMap editor.

1. The CUTGroup
Daniel X. O’Neil
Sonja Marziano
Social Enterprise Class
Northwestern University
February 25, 2015
1
@danxoneil @smartchicago #CUTGroup

2. Agenda-cize
• My personal journey
• Smart Chicago
• Origins
• Components
• Tools
• Methods
• Examples
2
@smartchicago #CUTGroup

3. My personal journey
• Web worker since June 15, 1998
3
@smartchicago #CUTGroup

4. First
• I made my first civic app in 1999
• Text-based WAP systems were just
coming into general use on phones
• I tried to think of the most compelling use
case for this technology. In other words,
what can’t wait for you to get back to your
desk?
• The answer: Killer on the Loose
April 21, 2015 4@danxoneil

5. Most compelling wireless utility,
1999
April 21, 2015 5@danxoneil

6. I’ve worked with some developers
April 21, 2015 6@danxoneil

7. I’ve worked with some developers
April 21, 2015 7@danxoneil

8. April 21, 2015 8@danxoneil

9. April 21, 2015 9@danxoneil

10. April 21, 2015 10@danxoneil

11. Our principles
11
@smartchicago #CUTGroup

12. The Strategy
12@smartchicago

13. Access, Skills, and Data
• Access to the Internet for all
– Broadband at home
– Public computer centers and Community technology
centers
– Next generation stuff
• Skills once you’re on the Internet
– All on ramps
• Data so that there’s something worth looking at
once you’re on and know how to work it
– Apps
– Infrastructure
13
@smartchicago #CUTGroup

14. The Founders
14@smartchicago

15. Three Chicago Institutions
• The John D. and Catherine T. MacArthur Foundation
– A center for ideation and conception
– 2007 report, “The City that NETWorks”
– The primary source of funding
• Chicago Community Trust
– Provides housing for the organization and is fiscal agent
– At the center of funding and knowledge for the region
– Critical financial, human resources, and other support
• The City of Chicago
– The most important policy lever in the region
– Departmental relationships
– Getting things done
15
@smartchicago #CUTGroup

16. CUTGroup
16
@smartchicago #CUTGroup
• The Civic User Testing Group is a set of
regular Chicago residents who get paid to test
civic apps
– http://www.cutgroup.org/
• The CUTGroup Book: Civic User Testing
Group as a New Model for UX Testing, Digital
Skills Development, and Community
Engagement in Civic Tech
– http://www.cutgroupbook.org/

17. Origins
17
@smartchicago #CUTGroup
• Stemmed from Illinois Open Tech Challenge
(IOTC)
– Very successful in recruiting tech talent
– Not so successful in recruiting residents
– My experiences
• We basically came to the thought: “If it doesn’t
work for you, it doesn’t work.”

18. Three Main Components
18
@smartchicago #CUTGroup
• UX Testing
• Digital Skills
• Community
Engagement

19. Components: UX Testing
19
@smartchicago #CUTGroup
• Legitimate UX testing -- the developers who
work with us get specific, actionable feedback
from relevantly situated users
• Both quantitative and qualitative info
• We’ve found glaring bugs, discovered unique
insights, and helped people plan new product
releases.
• We stray from UX test design principles,
however, in a very key way: by requiring the
developer to participate in the test.

20. Components: Digital Skills
20
@smartchicago #CUTGroup
• Each test consists of a
mix of 10-20 people of
widely varying digital
skills, all learning and
teaching each other, in
the same room, out in
plain public view
• Developers are
learning how to
conduct UX tests and
to make their sites
better

21. Components: Community Engagement
21
@smartchicago #CUTGroup
• Biggest surprise to us
• Conduct tests in Connect Chicago locations
– More than 250 places in Chicago where you can
use a computer for free
– http://weconnectchicago.org/
• We were surprised to see how much people
like the CUTGroup.
– Same last two questions on every CUTGroup test
form:
• “Did you like this CUTGroup test?”
• “Anything else to add?”

22. Components: Community Engagement
22
@smartchicago #CUTGroup
• Diverse group of testers from different
backgrounds and neighborhoods

23. Tools
23
@smartchicago #CUTGroup

24. Tools
24
@smartchicago #CUTGroup
• Mailchimp: http://mailchimp.com/
– E-mail campaigns
• Wufoo forms: http://www.wufoo.com/
– Collect information
– Recruitment e-mails
– Tests
• Excel
• Patterns: software that helps collect data from
different systems; assists with segmenting

25. More Tools
25
@smartchicago #CUTGroup
• Devices
– Laptops & iPads
– Encourage testers to use their own devices
– We can segment based on tester devices
• Recording equipment:
– http://www.smartchicagocollaborative.org/hardware-
and-software-for-cutgroup-and-civic-hacker-events/
• Gift cards:
– http://www.smartchicagocollaborative.org/gift-cards-for-
cutgroup/
– $5 gift card for sign up, $20 gift for a test
• Candy!

26. Methods: Pre-Test
26
@smartchicago #CUTGroup
• Developer meeting
– Target testers, Type of test, Questions to ask
– What do we want to learn?
• Recruitment
– 1st e-mail campaign
– The more we ask of testers, the smarter we get about
testing!
• Segmenting
– 2nd e-mail campaign asking for time availability
• Scouting for test locations
– We want to be all over the city

27. Methods: During Test
27
@smartchicago #CUTGroup
• Proctoring
– Different test types:
direct proctoring, focus
groups, remote, self-
driven/in-person
– Depends on developer
meeting and what we
want to learn from the
test
• We use Wufoo forms
to ask test questions
with or without a
proctor

28. Methods: Post-Test
28
@smartchicago #CUTGroup
• Analysis
– In-depth analysis of different questions
• Publishing results for every test
• Follow Up
– Github Issues; example from CUTGroup #9:

29. Tests
29
@smartchicago #CUTGroup
We have conducted 12 tests to date:
• #1 – FreedomPop Router Test
• #2 – Go2School User Application Test
• #3 – Chicago Health Atlas
• #4 – EatSafe.co
• #5 – ChicagoWorksforYou.com
• #6 – OpenStreetMap Editor
• #7 – EveryBlock iPhone App
• #8 – Waitbot App
• #9 – Foodborne Chicago
• #10 – Build it! Bronzeville
• #11 – Expunge.io
• #12 – Roll with Me

30. Examples: EatSafe.co
• EatSafe.co Tester #8
– https://www.youtube.com/watch?v=rGjMhz3TIkI
30
@smartchicago #CUTGroup

31. Examples: OpenStreetMap
• OpenStreetMap Tester #5
– http://youtu.be/BWNZxVhV7Kk
31
@smartchicago #CUTGroup

32. Foodborne Chicago
• An app that searches Twitter
for tweets related to food
poisoning and helps report
these incidents to the
Chicago Department of
Public Health
• https://www.foodbornechicag
o.org/
32
@smartchicago @foodbornechi #CUTGroup

33. Foodborne Questions
Here is a list of questions we wanted to answer through this
test
• How do users feel about responding to a random tweet?
• Does the form require too much information?
• Would users feel better or worse if there was more
Chicago Department of Public Health or 311 visibility?
• Why might residents on the South side of Chicago not
respond as often as residents from other Chicago
neighborhoods?
33
@smartchicago @foodbornechi #CUTGroup

34. Foodborne Results
• How do users feel about responding to a
random tweet?
- Twitter is used as a private communications
network for the majority of our testers.
- Twitter handle & logo helps
- Tweets should have three elements:
compassion, question, and an official element.
• Does the form require too much information?
- Surprised that form wasn’t too long per testers
34
@smartchicago @foodbornechi #CUTGroup

35. Foodborne Results
• Would users feel better or worse if there was more
Chicago Department of Public Health or 311 visibility?
- Wanted stronger connection to City of Chicago
• Why do residents on the South side of Chicago not
respond as often as residents from other Chicago
neighborhoods?
- Twitter users see Twitter as a way to connect with
people they know, or are within in their circles
whether that be their friends, their work, or their
community.
35
@smartchicago @foodbornechi #CUTGroup

36. Improvements
• Change the Tweet
• URL Update
• Be more Official
• More about Process
• Being part of more Networks
36
@smartchicago @foodbornechi #CUTGroup

37. The CUTGroup Book
http://www.cutgroupbook.org
37

38. Be a Tester!
• Are you a Chicago Resident?
• Interested in testing?
• Sign Up Here  http://www.cutgroup.org/
38
@smartchicago #CUTGroup

39. Sign up to test yer stuff!
• Are you a developer?
• Want to do this?
• Sign Up Here 
http://www.cutgroup.org/developers.html
39
@smartchicago #CUTGroup

40. So ya thanks
• @SmartChicago
• @DanXONeil
• @SmartChgoKyla
• @SSMarziano
• www.SmartChicagoCollaborative.org
40