As part of Denver's 2019 Startup Week, Crownpeak's Director of UX, Ari Weissman and Lys Maitland, Experience Research Manager at a national healthcare organization, presented a joint session on "Choosing a direction: Learning how to test ideas and designs."
4. British Consulate
Charter / Spectrum
Checkr
The Commons on Champa
Cooley
CTRC
Ford Smart Mobility
Fremont Economic Development
General Assembly
Hilton Garden Inn
Nanno
Nix Media
CoastalCloud
Formidable
Granicus
Hogan Lovells
Iterable
Name.com
Plante Moran
Slifer Frampton
Starry
Swiftpage
Syntropy Partners
Yoonit Wine
HEADLINE SPONSORS
PARTNER SPONSORS MEMBER SPONSORS
Officescapes
Procare
Quizlet Ridg
Segment
SimpleBooth
Slalom
Tendril / Uplight
Test Double
Two Parts / The Passport
Program
6. The Agenda
1. Intro
2. What is UX Research?
3. The Design Process
4. Why do we test?
5. How to plan
6. $$$ vs. $
7. Testing Outputs
8. Engaging the whole team
9. What to do next
7. Lys Maitland
Experience Researcher Manager888.310.5327
• 15+ years experience
• Responsible for
research from product
inception to
implementation
• Led enterprise-level
research for clients like
Transamerica, Davita,
Scottrade, NextGear,
Securian
8. Ari Weissman
Director of UX @ Crownpeak888.310.5327
• 15+ years global
experience
• Responsible for all things
UX, from research
through experience
design
• Lead for clients including
TimeWarner Cable,
AMEX, PwC, FreemanCo,
Securian, …
10. To many people, conducting user research is a bit like
cleaning up the garage: they know it would be a good thing
to do and would bring some real benefits, yet somehow
they never quite get round to it.
— Arin Bhowmick
Vice President, Design at IBM
“
”
11. AUTHOR OF OBSERVING THE
USER EXPERIENCE
Mike Kuniaysky
What is UX Research?
“User research focuses on
understanding user behaviors, needs,
and motivations through observation
techniques, task analysis, and other
feedback methodologies…the process
of understanding the impact of design
on an audience.”
13. Double Diamond: Problem Space
The Problem Space
• This is where we test ideas
• Goal is to identify a strategy and
develop a plan
a.k.a. “Design the right thing.”
• This is where great user
experiences come from
14. Discovery Research (diverge)
Understanding the User and
Problem
• Desk Research: lit reviews, existing
research, domain research, google
scholar
• Stakeholder Research: workshops,
interviews, demonstrations
• User Research: interviews,
observation, surveys, contextual
inquiry
15. Definition Research (converge)
Aligning around understanding
• Target audience
• Desired user outcomes
• Desired business outcomes
• Risks and assumptions
• Method to measure success
• A hypothesis on how to solve the
problem
16. Vision without action is a
daydream. Action without
vision is a nightmare.
— Japanese Proverb
“
”
17. Double Diamond: Solution Space
The Solution Space
• This is where we test solutions
• Goal is to work through
tradeoffs to deliver an optimal
solution
a.k.a. “Design things right”
• This is also where great
usability comes from
18. Development Research (diverge)
Exploring Solutions
• Design: sketching, prototyping,
usability testing
• Information Architecture:
prototyping, card sorting, tree
tests, click tests
• Collaboration: design studio, co-
design, feedback workshop
• Desk Research: comparative
analysis, pattern review
19. Delivery Research (diverge)
Delivering an outcome
• Test and iterate on design and
content
• Consolidate design rationale
• Document experience / design
decisions
20. Don’t Forget to Measure!
Did you choose how to measure success before you began design?
Did you benchmark the current state to know if it's improved?
Is your measurement strategy based on user outcomes?
21. Why do we test?
UX research helps:
• Validate assumptions
• Understand the problem from the user’s perspective
• Learn about the ‘hidden’ things we didn’t know
• Measure the delta
• Remove opinions from the design process
• Create alignment
• Reduce the cost of delivering a successful product.
22. Some examples of testing
Testing an IA or design learnability
CLICK TEST
Testing flows and content
USABILITY TESTING
Testing and measuring current state
for redesign
BENCHMARKING
Closed card sort to validate an IA
CARD SORTING
Iterative testing of flows and concepts,
some generative research
ITERATIVE DESIGN TESTING
27. Who is your
audience?
888.310.5327
Some questions to ask:
- Who will use of the thing you’re
building?
- Are there different groups of users?
If so, which have the highest
priority?
- Where are they?
28. You can learn valuable things by asking the right
people the wrong questions. If you are talking to the
wrong people it doesn't matter what you ask.
— Erika Hall, author of Just Enough Research
“
”
30. Analysis and Synthesis
1 2 3 4
Trends
Look for trends across
all participants
Good and Bad
Document what
doesn't work AND
what does work
Prioritize
Choose the most
important findings to
share and focus on
Think big picture
Is the issue identified a
symptom of a bigger
problem?
5 6 7 8
Quotes
Identify quotes that
capture the essence of
your insights
Solutions
Collect solutions
identified by
participants
Opportunities
Do the outcomes
suggest good next
steps?
Centralize/
Share
Easy access and
redundancy.
32. Formal
Location
Quiet room with a table
Separate location for
observers
Quality phone if remote
Logistics
Schedule a location and
time for two hours
Recruit participants and
distribute incentive
Prep/share/revise
discussion guide
Create data capture
sheet
Audio/Video recording
Costs
Location
Recruiting and
incentives
Screen sharing/
Recording tool
Snacks and beverages
Transcriptions
Fidelity to test
Typically wireframe or
above
1 2 3 4
33. Quick and cheap
Location
Anywhere!
Logistics
Find a place
Find some people
Ask some questions
Write down answers
Costs
Minimal to none
Fidelity to test
Paper and pencil
1 2 3 4
Let’s talk about examples
40. I use not only all the brains I have, but
all the ones I can borrow
- Woodrow Wilson
“ ”
41. Engage your
team at every
step
- Choosing Goals and Audience
- Reviewing Discussion Guide
- Observing and Note Taking
- Debriefing
- Probing During Analysis
- Prioritizing Results
- Determining Next Steps
43. What to do
next?
- Build testing into your delivery process
- Set up consistent research cadence
- Keep a backlog of problems
- Join in the design and dev process
- Develop a pool of users
- Alternate hi-effort and low-effort activities
- Validate between qualitative and
quantitative
- Triangulate your data
