This document outlines Kate Rutter's presentation on measuring what matters from a UX perspective at UX Days Tokyo on April 19, 2015. The presentation guides attendees through an exercise to create a product snapshot, envision key user interactions, and identify metrics that measure progress on the most important interaction. The goal is to help product teams select metrics that are user-centered and will drive improvements based on real user needs and behaviors rather than just vanity metrics. Attendees work individually and in pairs to create snapshots of existing or hypothetical products and identify metrics focused on measuring a key interaction's success.
Top profile Call Girls In Mysore [ 7014168258 ] Call Me For Genuine Models We...
Measuring What Matters: A UX Approach to Metrics :: UX Days Tokyo [April 2015]
1. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
a UX Approach to Metrics
Measuring what Matters
Sunday, April 19, 2015 | Kate Rutter | @katerutter
2. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Letʼs get to know each other
At each table...
* name
* role
* your favorite superpower
Hello!
5 min
3. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
@katerutter
tweet-friendly
4. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
5. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
6. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
I am not an expert on your business.
You are.
My goal is to help you explore
how to measure how well your product
serves real humans...
so you can use metrics to design & improve
the products you build.
What to expect...
7. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
What to expect...
We are going to be working fast.
8. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
What to expect...
There is a purpose to this.
I will push you to get something
down rapidly. In 1 day, youʼll learn how to
create actionable, UX-centered metrics to
track progress of your product.
We are going to be working fast.
9. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
What to expect...
You will be working solo and in pairs.
Note that these techniques are
very powerful in teams.
Weʼll keep talking to a minimum.
(Happy to talk more after the workshop.)
10. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
What to expect...
You may not like the pace/process/
pressure/perspective, etc. etc.
That is okay.
All I ask is that you keep an open
mind and try it.
11. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
How do you
measure UX
now?
10 min
12. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
UX + metrics
=
<3
13. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
•Identify and design for key interactions that
deliver measurable value to your users.
•Have a clear set of numbers that indicate
progress.
•Know what interactions you need to track to
measure user behaviors.
Why?
14. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Why this matters:
“Over 10,000 downloads!”
“Avg time on site is 22 minutes!”
“+450 new sign-ins this week!”
15. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Why this matters:
“Over 10,000 downloads!”
“Avg time on site is 22 minutes!”
“+450 new sign-ins this week!”
FAIL
16. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Why this matters:
“Over 10,000 downloads!”
“Avg time on site is 22 minutes!”
“+450 new sign-ins this week!”is this good?
17. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Product
(black box)
Conversion
Funnel
Traffic
18. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Product
(black box)
Conversion
Funnel
Traffic
19. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
~ Alistair Croll
Don’t just ask questions. Know how
the answers to the questions will
change your behavior.
20. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
A good metric is...
Clear & specific
Comparative
Rate/Ratio
Changes your behavior
21. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
What we will make
Snapshot
What people?
What problems?
What product?
How will we
track progress?
Metric plan
What can a
user do with
our product?
Key use
What metric
matters?
Key metric
22. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Make a
product
snapshot
23. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Letʼs get warmed up
A product you work on
An idea for a new product
Pick an idea to
use for the day
2 min
Task-a-
Doodle
write it on a
stickynoteor
24. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Create a Product Snapshot
Snapshot
What people?
What problems?
What product?
What can a
user do with
our product?
Key use
How will we
track progress?
Metric plan
What metric
matters?
Key metric
25. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
a mobile app for
delegating tasks
Working parents
with with kids
need to know when
something’s done
Too much to do, not enough time
need to share
tasks on the go
Task-a-doodle 4 19 2015
10 min
Who are the people?
(the users)
What problems do
they have?
What is a proposed
solution?
Product Snapshot
BE SPECIFIC
26. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Is it clear?
Is it specific?
What can be
improved?
Product Snapshot
Share as a pair
5 min
(each)
27. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
What you have now...
Snapshot
What people?
What problems?
What product?
What can a
user do with
our product?
Key use
How will we
track progress?
Metric plan
What metric
matters?
Key metric
28. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
the
UX STACK
29. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
How do I
KNOW IT’S
WORKING?
I have a
product.
so...
30. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Who is it for?
Why does it matter
to them?
What can they do that
they couldnʼt do before?
What features do they
need to do that?
How do those features
fit together?
Users
Needs &
goals
Uses
Features
Product,
brand & voice
via @luxrco
UX elements
31. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Users
Needs &
goals
Uses
Features
Product,
brand & voice
Features
Users
Uses
Product
Needs
via @luxrco
UX stack
via @luxrco
32. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Users
Needs &
goals
Uses
Features
Product,
brand & voice
Features
Users
Uses
Product
Needs
Company
Purpose
via @luxrco
sketches,
prototypes,
pixels
UI
UX stack
via @luxrco
33. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Envision a
Key Use
34. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Envision a Key Use
Snapshot
What people?
What problems?
What product?
What can a
user do with
our product?
Key use
How will we
track progress?
Metric plan
What metric
matters?
Key metric
35. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
via @luxrco
Features
Users
Uses
Product
Needs
Company
Purpose
via @luxrco
sketches,
prototypes,
pixels
UI
UX stack
36. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
An example from Foodspotting
6-up uses
37. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Envision 6 uses
sketch!
Task-a-doodle, Mary
4 19 2015
38. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
39. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
30 sec
40. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Rapid Sketching
Star people!
People, Context & Emotion
41. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
You can draw a person in 4 simple steps....
Star People
42. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Star People in context
Use simple
shapes to put a
person in a place.
43. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
smile frown
up brows
neutral
down
brows
no brows
add a mouth...
add eyebrows...
Expressions Matrix
44. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
2 min
Speed-sketch
10 star people!
45. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Key Uses 6-up
OK to be obvious
Task-a-doodle, Mary
4 19 2015
•1 idea per panel
•6 different ideas
(not a sequence)
•Sketch AND write
What can someone
DO with your
product?
- Solves the problem
- Addresses a need
10 min
46. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Key Use
•This is info transfer; not
a new idea
•Make the sketch clean &
clear
Select one and redraw.
5 min
Task-a-doodle,
Mary
4 19 2015
Share tasks
with others.
What can someone
DO with your
product?
OK to be obvious
47. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
What you have now...
Snapshot
What people?
What problems?
What product?
What can a
user do with
our product?
Key use
How will we
track progress?
Metric plan
What metric
matters?
Key metric
48. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Lunch!
http://www.g4tv.com/attackoftheshow/blog/post/715478/cutest-bento-boxes-for-geeks
1 hr
49. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Identify a Key
Metric
50. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Identify a Key Metric
Snapshot
What people?
What problems?
What product?
What can a
user do with
our product?
Key use
How will we
track progress?
Metric plan
What metric
matters?
Key metric
51. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
52. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
a good metric... a great metric...
...makes you look at all the
other metrics and say “none of
those other numbers matter if
we donʼt get this right first.”
...measures the usage of your
product by a person. The usage
should be specific to features that
deliver value to your user.
vanity
total number
of registered
users
unhelpful
sign-ups
% of users
who share a
task 3+
times a day,
per week
awesomebetter
% of users
who sign in
3+ times a
day,
per week
good
% of new
users per
week
Metrics checklist
53. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Brainstorm 5 ideas for
metrics
5 min
What could you count to make sure
the Key Use is happening?
54. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Pick one to focus on
1 min
55. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
3. Is there an object
basis?
• “per user” [ ___/user]
• “per user per week” (fancy!)
• Examples: per user, per session,
or by interaction or item
(messages per file, emails
per thread)
* not required, but super-helpful
• “per week” [ ___/wk ]
• “per month” [ ___/mo]
2. Is there a time basis?
• “number of...” [ # ]
• “average number of...” [avg ]
• “percent of...” [ % ]
1. Does the metric
begin with a number?
Make it awesome 5 min
56. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Share your work
Review the snapshot &
use.
Take 3 minutes to
confirm clarity
Peers: make
recommendations
Share as a pair
5 min
(each)
Key Metric
57. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
10 min
Questions &
Discussion
58. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
% of users who wear a tie
within one week of receiving
it, per month.
% users who wear a received
tie 3+ times/week, per week.
% users who purchase a tie 3
shipments in a row, per week.
59. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Put it on your dashboard
1 min
60. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Pick another one
1 min
61. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
3. Is there an object
basis?
• “per user” [ ___/user]
• “per user per week” (fancy!)
• Examples: per user, per session,
or by interaction or item
(messages per file, emails
per thread)
* not required, but super-helpful
• “per week” [ ___/wk ]
• “per month” [ ___/mo]
2. Is there a time basis?
• “number of...” [ # ]
• “average number of...” [avg ]
• “percent of...” [ % ]
1. Does the metric
begin with a number?
Make it awesome 5 min
62. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Put it on your dashboard
63. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Break!
15 min
64. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
One more time
1 min
65. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
3. Is there an object
basis?
• “per user” [ ___/user]
• “per user per week” (fancy!)
• Examples: per user, per session,
or by interaction or item
(messages per file, emails
per thread)
* not required, but super-helpful
• “per week” [ ___/wk ]
• “per month” [ ___/mo]
2. Is there a time basis?
• “number of...” [ # ]
• “average number of...” [avg ]
• “percent of...” [ % ]
1. Does the metric
begin with a number?
Make it awesome 5 min
66. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Share your work
Review the snapshot &
use.
Take 3 minutes to
confirm clarity
Peers: make
recommendations
Share as a pair
5 min
(each)
Key Metric
67. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Put it on your dashboard
1 min
Pick ONE to work on next
68. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
What you have now...
Snapshot
What people?
What problems?
What product?
What can a
user do with
our product?
Key use
How will we
track progress?
Metric plan
What metric
matters?
Key metric
69. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Make a Plan
70. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Make a Metric Plan
Snapshot
What people?
What problems?
What product?
What can a
user do with
our product?
Key use
How will we
track progress?
Metric plan
What metric
matters?
Key metric
71. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Metrics Plan
•Set a goal
•Set a time span
•Think about how to
collect the data
How will you track
progress?
% of users
who share
a task 3+
times a day,
per week
Capture clicks on “share now”
- capture # of shares
- timestamp each
- capture by user ID
(dump into raw data file)
- use total # of users as base
1. Calculate by day, # of users with
3+ shares from 12:00am - 11:59pm
2. Use threshold of 3+ to order list
3. # of users above threshold / Total #
of users x 100 = %
Week starts Monday @ 12:00am,
ends Sunday @ 11:59pm
30%
8 1 2015
5%
4 19 2015
72. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Metrics Plan
•Set a goal
•Set a time span
•Think about how to
collect the data
How will you track
progress?
% of users
who share
a task 3+
times a day,
per week
Capture clicks on “share now”
- capture # of shares
- timestamp each
- capture by user ID
(dump into raw data file)
- use total # of users as base
1. Calculate by day, # of users with
3+ shares from 12:00am - 11:59pm
2. Use threshold of 3+ to order list
3. # of users above threshold / Total #
of users x 100 = %
Week starts Monday @ 12:00am,
ends Sunday @ 11:59pm
30%
8 1 2015
5%
4 19 2015
Capture clicks on “share now”
- capture # of shares
- timestamp each
- capture by user ID
(dump into raw data file)
- use total # of users as base
1. Calculate by day, # of users with
3+ shares from 12:00am - 11:59pm
2. Use threshold of 3+ to order list
3. # of users above threshold / Total #
of users x 100 = %
Week starts Monday @ 12:00am,
ends Sunday @ 11:59pm
73. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Three ways to gather data:
• Non-technical
• Technical “by hand”
• Instrumented with software
Instrumentation
process of setting up the technology to capture the
numbers that matter to you. This often means using
an analytics package or writing custom code.
74. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Metrics Plan
•Set a goal
•Set a time span
•Think about how to
collect the data
20 min
How will you track
progress?
% of users
who share
a task 3+
times a day,
per week
Capture clicks on “share now”
- capture # of shares
- timestamp each
- capture by user ID
(dump into raw data file)
- use total # of users as base
1. Calculate by day, # of users with
3+ shares from 12:00am - 11:59pm
2. Use threshold of 3+ to order list
3. # of users above threshold / Total #
of users x 100 = %
Week starts Monday @ 12:00am,
ends Sunday @ 11:59pm
30%
8 1 2015
5%
4 19 2015
75. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Pass your work
1-minute “quiet read”
5 minutes of questions
to confirm clarity and
then feedback.
Peers: Offer help and
advice.
Share as a pair
15 min
(total)
Metrics Plan
76. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
What you have now...
Snapshot
What people?
What problems?
What product?
What can a
user do with
our product?
Key use
How will we
track progress?
Metric plan
What metric
matters?
Key metric
77. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Tracking over
time
78. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Track over time
79. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
80. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
81. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Pull it all together.
Tell the story.
82. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
via @luxrco
Features
Users
Uses
Product
Needs
Company
Purpose
via @luxrco
sketches,
prototypes,
pixels
UI
UX stack
83. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
What you have now...
Snapshot
What people?
What problems?
What product?
What can a
user do with
our product?
Key use
How will we
track progress?
Metric plan
What metric
matters?
Key metric
84. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
This is [product name] .
* Minimum Viable Product
10 minMetrics Gallery
The metric
& plan:
The value:
The idea...
85. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
10 min
Questions &
Discussion
86. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Geek out & learn tons more
87. UX Days Tokyo | Measuring What Matters | @katerutter | APRIL 19, 2015
Kate Rutter
@katerutter
http://intelleto.com
Thanks!
88. UX Days Tokyo | Measuring What Matters | @katerutter | April 2015
Handouts
a UX Approach to Metrics
Measuring what Matters
89. UX Days Tokyo | Measuring What Matters | @katerutter | April 2015
Features
Users
Uses
Product
Needs
Company
Purpose
sketches,
prototypes,
pixels
UI
UX Stack
93. 6
a great metric...
...makes you look at all the other metrics
and say “none of those other numbers
matter if we donʼt get this right first.”
a good metric...
...measures the usage of the product by
a person. The usage should be specific to
features that deliver value to the user.
Making metrics actionable
vanity good betterunhelpful
total number
of registered
users
% of users
who share a
task 3+
times a day,
per week
sign-ups
awesome
% of users
who sign in
3+ times a
day,
per week
% of new
users per
week
Does the metric begin
with a number?
• “percent of...” [ % ]
• “average number of...” [avg ]
• “number of...” [ # ]
Is there a time basis?
Is it comparative?
• “per week” [ ___/wk ]
• “per month” [ ___/mo]
Is there an object basis?
* not required, but super-helpful
• “per user” [ ___/user]
• “per user per week” (fancy!)
1
Metrics Check
2 3
95. by
Goal
/ /
Key Metric plan date
/ /
To assess progress, we
will measure this metric:
Measured
Baseline
Daily Weekly Monthly
Hereʼs how weʼll collect the data:
metric