Más contenido relacionado La actualidad más candente (20) Similar a Introducing: The Lab Notebook - A tool for managing a LEAN UX process (20) Introducing: The Lab Notebook - A tool for managing a LEAN UX process 1. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Introducing: The Lab Notebook
A hypothesis-driven approach for
delivering value faster
#UXLabNotebook
Julie Casanave @jcasanave
Kerry Holeman @uxkerry
2. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
What’s this all about?
3. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Project completion can easily be
mistaken for project success.
4. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
A successful launch delivers
measurable customer value.
If users don’t adopt the solution
we need to evaluate and make a decision.
6. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Science!
7. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
What’s the most important
unanswered question?
What’s the smallest thing we can make
to answer that question?
-Steve Mulder, Senior Director of Audience Insights, NPR
8. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Create a Lab Report
Create
the experiment
Decide
what to do next
Define
the question
9. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Create a Lab Report
Create
the experiment
How will we
learn it?
Decide
what to do next
What do we do
with what we
learned?
Define
the question
What do we
want to learn?
10. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Define the Question
What do we want to learn?
11. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Problem Statement
WHAT ISSUE NEEDS SOLVING?
“I am [customer] trying to [accomplish goal].
But [insert problem] because [insert cause].
This makes me feel [emotion].
The consequence for [my team/company]
is [adverse business effect]”
12. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
I am a 60-year old trying to take a retirement withdrawal online,
because I need to cover the cost of a new car.
But I don’t feel confident doing it online because I don’t
understand if I’m subject to penalties or tax consequences.
This makes me feel anxious.
The consequence for Fidelity is costly customer service calls
and poor customer perception.
How might we…
“I am [customer] trying to [accomplish goal].
But [insert problem] because [insert cause].
This makes me feel [emotion].
The consequence for [my team/company] is …
13. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Exercise
WHAT ISSUE NEEDS SOLVING?
“I am [customer] trying to [accomplish goal].
But [insert problem] because [insert cause].
This makes me feel [emotion].
The consequence for [my team/company]
is [adverse business effect]”
14. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Define the Experiment
How will we learn?
15. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Define the Experiment
How the issue
will be addressed
What we’ll build
to address it
How we’ll know
it works
HYPOTHESIS
(solution strategy)
TEST
strategy
MEASUREMENT
strategy
+ +
“[This change] will
enable [who] to achieve
[what], leading to
[value].”
“The simplest way
to test this is to [test
strategy].”
“We will know we’ve
succeeded when we
observe [user
behavior/sentiment].
16. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Define the Experiment
HYPOTHESIS
(solution strategy)
TEST
strategy
MEASUREMENT
strategy
+ +
A Turbo-Tax-like withdrawal process, which guides people
through a decision path, will help customers withdraw retirement
funds with a higher degree of confidence, leading to fewer calls to
the call center.
[This change] will enable [who] to achieve [what], leading to [value].”
17. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Define the Experiment
HYPOTHESIS
(solution strategy)
TEST
strategy
MEASUREMENT
strategy
+ +
The simplest way to test this is with a moderated usability session
+ paper prototype which takes the customer down one sample
decision-making path to completion.
The simplest way to test this is to [test strategy].
18. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Define the Experiment
HYPOTHESIS
(solution strategy)
TEST
strategy
MEASUREMENT
strategy
+ +
We will know we’ve succeeded when we observe customers:
1. Successfully complete the process without indicating they’d
abandon.
2. Indicate that their level of confidence about completing the
transaction is higher than it is today.
We will know we’ve succeeded when we observe [user behavior/sentiment].
19. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Exercise: Hypothesis
HYPOTHESIS
(solution strategy)
TEST
strategy
MEASUREMENT
strategy
+ +
“[This change] will enable [who] to achieve [what],
leading to [value].”
20. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Decide on a Test Strategy
SENTIMENT
(Whatpeoplesay)
BEHAVIOR
(Whatpeopledo)
QUANTITATIVE
(Reveals the “what”)
QUALITATIVE
(Reveals the “why”)
Surveys
Moderated
Usability Study
A/B Testing
Self/Un-Moderated
Usability Study
Opinion Lab Verbatims
Focus Groups
Interview
Call Center Notes
Social Media Activity
Web Analytics
Emotional Reaction Study
Card Sort Treejack Study
First Click Study
Eye-Tracking Study
Pre-release
Post-release
PRO TIP
Be biased towards
behavioral methods
21. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Think creatively about what type of
experiment will give the results you need.
Deployed code is only one kind of experiment.
(and often the most expensive kind)
22. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Exercise: Test Strategy
HYPOTHESIS
(solution strategy)
TEST
strategy
MEASUREMENT
strategy
+ +
“The simplest way to test this is to [test strategy].”
23. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Decide How to Measure
HAPPINESS
ENGAGEMENT
ADOPTION
RETENTION
TASK
SUCCESS Source: Google Ventures, How to Choose the Right UX Metrics for Your Product, by Kerry Rodden
Measures of attitudes, often collected via survey
• Emotional reaction
• Perceived level of effort
• Sentiment rating
• Level of confidence
• Volume of opinion lab
comments
24. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Decide How to Measure
HAPPINESS
ENGAGEMENT
ADOPTION
RETENTION
TASK
SUCCESS Source: Google Ventures, How to Choose the Right UX Metrics for Your Product, by Kerry Rodden
Level of user involvement
• Bounce rate
• Channel switching
• Page visits
• Searches
• Click-throughs
• Eye-tracking
• Visitors
25. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Decide How to Measure
HAPPINESS
ENGAGEMENT
ADOPTION
RETENTION
TASK
SUCCESS Source: Google Ventures, How to Choose the Right UX Metrics for Your Product, by Kerry Rodden
Gaining new users of a product or feature
• Click rates
• Click-throughs
• New accounts
26. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Decide How to Measure
HAPPINESS
ENGAGEMENT
ADOPTION
RETENTION
TASK
SUCCESS Source: Google Ventures, How to Choose the Right UX Metrics for Your Product, by Kerry Rodden
The rate at which existing users are returning
• NA for our program
27. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Decide How to Measure
HAPPINESS
ENGAGEMENT
ADOPTION
RETENTION
TASK
SUCCESS Source: Google Ventures, How to Choose the Right UX Metrics for Your Product, by Kerry Rodden
Efficiency, effectiveness, and error rate
• Completion speed
• Error rate
• Abandonment
• Completion rate
• Pathing
• Re-searches
28. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Decide How to Measure
HAPPINESS
ENGAGEMENT
ADOPTION
RETENTION
TASK
SUCCESS Source: Google Ventures, How to Choose the Right UX Metrics for Your Product, by Kerry Rodden
• Emotional reaction
• Perceived level of effort
• Sentiment rating
• Bounce rate
• Channel switching
• Click rates
• Click-throughs
• Not applicable
• Completion speed
• Error rate
• Level of confidence
• Volume of opinion lab comments
• Page visits
• Searches
• Click-throughs
• Abandonment
• Completion rate
• Eye-tracking
• Visitors
• New accounts
• Pathing
• Re-searches
What metric matters most?
29. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Decide How to Measure
HAPPINESS
ENGAGEMENT
ADOPTION
RETENTION
TASK
SUCCESS Source: Google Ventures, How to Choose the Right UX Metrics for Your Product, by Kerry Rodden
• Emotional reaction
• Perceived level of effort
• Sentiment rating
• Bounce rate
• Channel switching
• Click rates
• Click-throughs
• Not applicable
• Completion speed
• Error rate
• Level of confidence
• Volume of opinion lab comments
• Page visits
• Searches
• Click-throughs
• Abandonment
• Completion rate
• Eye-tracking
• Visitors
• New accounts
• Pathing
• Re-searches
Does this experience give you a
higher or lower level of confidence
compared to today’s process?
30. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Decide How to Measure
HAPPINESS
ENGAGEMENT
ADOPTION
RETENTION
TASK
SUCCESS Source: Google Ventures, How to Choose the Right UX Metrics for Your Product, by Kerry Rodden
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
• ??
WHICH ONE MATTERS MOST?
WHAT’S YOUR SET OF METRICS?
31. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Exercise: Measurement Strategy
HYPOTHESIS
(solution strategy)
TEST
strategy
MEASUREMENT
strategy
+ +
“We will know we’ve succeeded when we observe [user
behavior/sentiment].
32. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Decide what to do next
What do we do with what we learned?
33. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
What We Learned Major findings, unanticipated
discoveries
Pivot? Or Persevere? Or Stop?
At what point was the conclusion
reached?
Time Lapsed
Since Launch
Conclusion
What will happen next?Next steps
DD Month YYYYLaunch Date
Documenting what was learned
Outcomes How did experiment do against HEART
metrics?
34. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
A disproven hypothesis
is evidence.
It is not failure.
“I have not failed.
I've just found 10,000 ways that won't work.”
-Thomas A. Edison
35. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
And when that experiment is done?
36. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
LATHER, RINSE, REPEAT
? !
? !
? !
? !
? !
? !
DESIGN RUN ITERATE
LABNOTEBOOK
LAB REPORT
LAB REPORT
LAB REPORT
LAB REPORT
37. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
The “grain of salt” slide
38. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
MITIGATE RISK
• Deliver value faster
• Reduce wasted time, effort,
money
FOCUSED EXPERIMENTS
• Clear learning agenda
• Simple test strategy
• Well defined metrics
39. © 2016 Proprietary material; do not use without permission of author | #UXLabNotebook
Thank You.
Julie Casanave | @jcasanave
Kerry Holeman | @UXKerry
Notas del editor This is a framework that lets us identify how we’re going to determine success. While the business will continue to have high-level KPIs and performance scorecards, this framework takes it down to a more granular level, and gives us some very concrete ideas about how to determine if we’ve delivered actual value. It lets us look both at sentiment and behavior, both of which tell us about how much people like using our stuff. This is a framework that lets us identify how we’re going to determine success. While the business will continue to have high-level KPIs and performance scorecards, this framework takes it down to a more granular level, and gives us some very concrete ideas about how to determine if we’ve delivered actual value. It lets us look both at sentiment and behavior, both of which tell us about how much people like using our stuff.