This document summarizes a presentation on product excellence by Google. It discusses key concepts in product management like understanding users, mapping critical user journeys, prioritizing features, avoiding common pitfalls, establishing shared principles of excellence, measuring success metrics, and iterating to improve the product. The document provides examples and exercises to illustrate these concepts. The overall message is that achieving product excellence is an ongoing process of aligning teams, empathizing with users, building the right things, and continuously measuring and improving.
17. Exercise: Where do you stand?
Statements
It’s important to have a deep understanding of how users engage with my
product/service.
1a.
AGREE DISAGREE
18. Exercise: Where do you stand?
Statements
It’s important to have a deep understanding of how users engage with my
product/service.
1a.
I have a deep understanding of how users engage with my product/service.1b.
AGREE DISAGREE
19. Exercise: Where do you stand?
Statements
It’s important to have a deep understanding of how users engage with my
product/service.
1a.
I have a deep understanding of how users engage with my product/service.1b.
The first step in developing a product/service is observing and interviewing a
diverse set of users.
2.
AGREE DISAGREE
20. Exercise: Where do you stand?
Statements
It’s important to have a deep understanding of how users engage with my
product/service.
1a.
I have a deep understanding of how users engage with my product/service.1b.
The first step in developing a product/service is observing and interviewing a
diverse set of users.
2.
Companies should prioritize the needs of the paying users above the non-
paying users.
3.
AGREE DISAGREE
39. PE Framework at Google
How do we achieve a culture of PE?
ALIGN
EMPATHIZEBUILD
MEASURE
40. PE, one step at a time:
/ 1: Know your users
/ 2: Critical user journeys
/ 3: Prioritization
/ 4: Pitfalls
/ 5: PE principles
/ 6: Measurement
/ 7: Bringing it all together
ALIGN
EMPATHIZEBUILD
MEASURE
41. ALIGN
EMPATHIZEBUILD
MEASURE
/ 1: Know your users
/ 2: Critical user journeys
/ 3: Prioritization
/ 4: Pitfalls
/ 5: PE principles
/ 6: Measurement
/ 7: Bringing it all together
42. Roberto [Mexico], Samsung Galaxy S4
“When I left the zoo my phone told me all my data was
gone. Because G+ uploaded all the pictures, it spent
all my data. I was really angry.”
What assumption was made here?
Example - G+ auto upload
43. How do we fall prey to the CURSE OF KNOWLEDGE?
● Assumptions
● Unconscious bias
Know your users
The curse of knowledge - be aware of bias
44. User research modes
Ways to answer your questions
ANALYZE
Is there existing data we
can consult?
- Logs
- Research reports
- Market analyses
OBSERVE
Is there info we could glean
from watching users?
- Usability studies
- Diary studies
- Field visits
ASK
Is there info we could glean
from speaking to users?
- Surveys
- Interviews
- Focus groups
45. ALIGN
EMPATHIZEBUILD
MEASURE
/ 1: Know your users
/ 2: Critical user journeys
/ 3: Prioritization
/ 4: Pitfalls
/ 5: PE principles
/ 6: Measurement
/ 7: Bringing it all together
47. YouTube Sign Up ChromeSearch
Tap video result and open
web view
YouTube
Toggle back to browser to
complete ‘add to playlist’
Find song
Add to playlist,
go to signup flow Finish watching
Critical user journeys
What we care about
48. Example: Write a CUJ for this example
Uploading a YouTube video for the first time
A 50 step process!
50. I want to Build a home.
I build the most beautiful homes.
believe me!
51. We draw the blueprint, delivering to the future tenant desires and
preferences, evaluate the budget, and start building.Delegation is the key.
The most beautiful key.
52. Look at each piece.
They are the most beautiful pieces.
53. Create a Blue Print, a Journey Map.
The most beautiful Journey Map.
62. ALIGN
EMPATHIZEBUILD
MEASURE
/ 1: Know your users
/ 2: Critical user journeys
/ 3: Prioritization
/ 4: Pitfalls
/ 5: PE principles
/ 6: Measurement
/ 7: Bringing it all together
63. / Principles are a SHARED VOCABULARY that:
Defines excellence
Offers a verbal shorthand for discussing excellence
/ Analogous to Design Patterns in engineering
Principles
What are they for?
68. Measurement
How to create a metric
GOAL: How do you want users to
engage with your product/service?
SIGNAL: What changes in user behavior
or opinion would indicate goal success?
69. Measurement
A metric that worked
/ How do we evaluate quality of a search result?
GOAL: Users find what they need on the first try
70. Measurement
A metric that worked
/ How do we evaluate quality of a search result?
GOAL: Users find what they need on the first try
SIGNAL: Length of time before coming
back to Search after clicking on search result
GOAL: Users find what they need on the first try
71. Measurement
SUPER
Most business/developer user products
S – Serviceability & System Health
U – User Sentiment
P – Productivity
E – Engagement
R – Revenue
HEART
Most consumer products
H – Happiness
E – Engagement
A – Adoption
R – Retention
T – Task success
72. ALIGN
EMPATHIZEBUILD
MEASURE
/ 1: Know your users
/ 2: Critical user journeys
/ 3: Prioritization
/ 4: Pitfalls
/ 5: PE principles
/ 6: Measurement
/ 7: Bringing it all together
97. Part-time Product Management Courses in
San Francisco, Silicon Valley, Los Angeles, New
York, Austin, Boston, Seattle, Chicago, Denver,
London, Toronto
www.productschool.com
Notas del editor
Remember the Puzzle Analogy we user earlier?
To do this right, we need Wisdom
Excellence is a shared responsibility for all of us. It’s about taking pride in our work and in order to be successful, we need to make sure it’s built into the heart of everything we do.
The problem we’re trying to solve is incredibly complex. It’s messy, because at its core, the problem is about culture. It is a Google-wide problem.
For larger or stablished organizations, “Accelerate: Building Strategic Agility for a Faster-Moving World” By Dr. John Kotter, presents a practical and effective methodology to initiate Cultural Change.
The core principles of an effective organizational cahnge are well represented here. Google organizational change methodology wasn’t identical to this model, however, this model is a great start for initiating change.
Excellence is a shared responsibility for everyone. We need to make sure it’s built into the heart of everything we do.
This is where we’ll be spending time today.
We'll be discussing different aspects of the framework individually before tying them all together.
Goal: Intro to section
Duration: 0:00:30
Start: 11:19 AM
End: 11:20 AM
"CUJs" section takes roughly 54 min total.
Remember automatic photo uploading? How many of you would say that’s been a positive feature for you?
Roberto is on a monthly data plan which renews on the 21st each month. On the 23rd he went to the zoo and took a bunch of pictures. [click]
When he left for the day, he received a notification that his entire month's data had been used up
Robert told us felt really Angry.
So we need to ask ourselves how we fall victim to the Curse of Knowledge as we build products and services for users that aren’t us
From Wikipedia: “The curse of knowledge is a cognitive bias that occurs when, in predicting others' forecasts or behaviors, individuals are unable to ignore the knowledge they have that others do not have, or when they are unable to disregard information already processed.” You’ve got it for your product.
What are some assumptions we make about our users? (cultural, accessibility, access to resources, etc.)
Let’s look at some examples where we weren’t cognizant of the songs in our heads
Unconscious bias: Face recognition doesn’t works for black faces as well as for lighter skin colors
You can Ask
What information could you learn by interviewing or surveying your users?
Tying in what we know about unconscious bias, how can we gather data from the diverse users we have?
Goal: Intro to section
Duration: 0:00:30
Start: 11:19 AM
End: 11:20 AM
"CUJs" section takes roughly 54 min total.
Goal: Participants understand definition of CUJ
Duration: 0:01:00
Start: 11:26 AM
End: 11:27 AM
So what are CUJ?
CUJ describe the path a user takes to complete a goal or accomplish a task.
A CUJ is either very common, or very important to get right, or both.
Why are they useful?
CUJ are a way of reconciling your intentions for your users with their own
How might you use them?
Can be used to "audit" existing feature sets
AND can be used to design & define new ways to help your user achieve that goal
Goal: Understands that a user cares only about accomplishing her goal, not what’s happening behind the scenes.
Duration: 0:01:00
Start: 11:28 AM
End: 11:29 AM
Facilitator: [ talk through the scenario and make it as overly complex as possible, e.g., “A user starts by doing a voice query on their Nexus 5x running stock Android M and that triggers a recognizer based on their language settings on the device which returns the result to the GFE which triggers oneboxen, one of which is hopefully the youtube video, and the user clicks that, which triggers an intent, which the user will choose the YouTube app, and… ” ]
The user’s task might span several PAs. She might start in search and end up in YouTube
We’re showing our org structure to our users in situations like this.
The User doesn’t care what’s happening behind the scenes which enable her to accomplish her goal.
But it’s our responsibility to be the ones who care. We need to see the big picture of CUJs and work across teams to make these journeys work well
There are multiple ways to achieve the user's goal--some of which might already exist.
The goal is to look at the journey with a fresh set of eyes, focusing on the goal or intent, rather than the constraints around existing implementation
How might we look at this journey through the perspective of achieving the goal without the curse of knowing all the steps needed to achieve that goal?
[Transition] So how do we craft journeys and ensure they work well?
Goal: First attempt at writing a CUJ - Here’s an example of a full CUJ
Duration: 0:01:00
Start: 11:29 AM
End: 11:30 AM
Another great example - go/cloudsql-connect-cuj
This is a front-end example, but the point here applies more broadly. "This is more than a front end problem - changes to billing, latency, authentication, and overall process often require backend and frontend changes to be built in tandem." - Scott Ellsworth
YouTube team sat down and evaluated the experience of uploading a video to YouTube.
Full end-to-end experience; Discovered it took 50 steps!
Included steps other teams were responsible for (auth, browser, system UI, …)
Functionally, this process worked. And all the teams responsible for making different features of this journey did their job.
But no one considered what this experience was for a user. Nobody counted all the steps or evaluating the entire journey (until now!)
Check out the Google Insider article at go/youtube-50-1 with more details (they eliminated about 30 steps as of that writing).
Option: Play video @ go/pe-youtube - The YT team talked about how critical this process was for them, so for the rest of this section, we'll look at how to go about writing CUJs.
Every CUJ (starts with a) consists of a Goal and a Role
Goal = what is the users trying to do (“I want to remember an upcoming meeting”)
Role = who is the user and what attributes about her might influence the way she accomplishes this journey (“A grandmother who just bought first smartphone”)
CUJs in terms of a Goal and some number of Roles
We’re doing it this way to give you a concrete way to define CUJs.
Your team may use a different methodology, such as personas and use cases, but the end goal is the same.
You need to understand what your users want to do and what characteristics of theirs might influence how they want to do it.
Keep in mind that CUJ are permanent. They are the identity of your Product. Users use your product as they want to achieve the CUJ goal, based on the role they have. If your product CUJ are broken, don’t work well or are complex, users won’t use your product. If the goal you assumed for user, doesn’t align with users goal, why would users use your product?
Sometimes users use the existing CUJ to achieve another goal that you haven’t intended to solve. Be aware of them and make an educated decision on how to proceed and if you like to solve for this new goal.
Goal: Intro to section
Duration: 0:00:30
Start: 12:17 PM
End: 12:17 PM
The "Prioritization" Section takes roughly 21 min total.
Imagine your product is a network of interconnected roads, with each road representing a set of features or a distinct piece of the product
Imagine you just finished evaluating your CUJ – the blue line – and found a bunch of potholes – things that broke or hindered the CUJ.
The reality is, in addition to these issues, your product is littered with a whole bunch of other issues
You only have time to fix a fraction of the potholes.
How do you decide which ones to fix?
Mobile Search team did a GAR (Google Accessibility Rating) audit and found a whole bunch accessibility issues. Needed to decide how to prioritize issues to fix.
Selected a couple key CUJs to prioritize issues to fix, among them “I want to find the opening hours of a particular store.”
Issues: To a sighted user, the store hours were visible just “below the fold” and took seconds to find. But a screen reader user had to navigate through ads, the map, and most of the first result, along with hidden javascript which is spoken, adding time and cognitive load to the task.
Solution: Worked with >10 search features teams to build many improvements, including more meaningful item groupings in a display block, invisible text removal, voice commands for expanding knowledge panels, screen-readable phone number links, and text labels for major images and buttons.
Goal: Intro to section
Duration: 0:00:30
Start: 1:44 PM
End: 1:44 PM
The "Pitfalls" Section takes roughly 12 minutes total.
It’s good to look out for particular categories of problems that products & services tend to encounter over and over.
We call these common categories “pitfalls”. They’re not just bugs. Pitfalls are any issue that negatively impacts the user experience.
Pitfalls are helpful in that they give us a common language to talk about what’s not working
Let’s quickly walk through the 11 major pitfalls we’ve identified
Notes on the examples we’ll show:
Dated: These may have been fixed or are outdated, but we’re using them because they were great examples
Mobile: These are all mobile examples, but don’t get hung up on that. Why? When these pitfalls were originally documented, mobile examples were the easiest to find.
Blameless: Let’s all remember the rule that we should assume everyone here is intelligent, well intentioned, and is making reasonable decisions. We’re not blaming anyone. Pitfalls are created by innocent oversights, yet still damaging.
Goal: Intro to section
Duration: 0:00:30
Start: 1:56 PM
End: 1:57 PM
The "PE Principles" Section takes roughly 25 min total
Principles serve two functions
They help us define what excellence is. They provide aspirational goals for our products/services
They enable us to have conversations around excellence
Analogous to Design Patterns for engineers
Design Patterns provide a shorthand for engineers to conceptualize and discuss about complex engineering topics
One of the most common examples is the Singleton pattern (https://en.wikipedia.org/wiki/Singleton_pattern): “In software engineering, the singleton pattern is a design pattern that restricts the instantiation of a workshop to one object. This is useful when exactly one object is needed to coordinate actions across the system. The concept is sometimes generalized to systems that operate more efficiently when only one object exists, or that restrict the instantiation to a certain number of objects. The term comes from the mathematical concept of a singleton.”
Two engineers often discuss their software designs in terms of these patterns because it makes it easier to describe complex ideas. When one engineers says, “We should model the keyboard as a singleton”, this is clearly understood by another engineer educated on this shared vocabulary (which is common).
A shared vocabulary (principles) helps us collectively drive towards a shared vision (PE)
The PE Principles are organized into three pillars; focused utility, simple design, and crafted execution. The emphasis is on balance; even as we value our ability to launch and iterate at speed, it's important to do so in a way that focuses on each of these dimensions of the product experience, without sacrificing one for another.
Focused Utility
Product provides simple value that is easily recognize
Is it the right thing?
Simple Design
Product feels effortless to adopt and use
Crafted Execution
Did we get the details right?
Ask: How many of you would be thrilled to report this after a launch?
Have you heard the phrase "you are what you measure"?
Over time you only improve the metrics you choose to measure. The decisions you make end up driving those metrics. Those decisions do not necessarily drive the underlying changes you want, even if you thought those changes were why you chose the metrics in the first place.
It might be the case that this graph is good news. But it might not tell the full story. Maybe the number of new users has increased, but your base of existing users diminished
30 day actives: If we say this matters, we’ll work towards improving this.
Well intentioned metrics that indirectly measure what we think is in users’ best interest can become wrong incentive for us
Ask: How might you measure "Happiness" or another metric that is aligned with users' needs?
(Optional) Example: Google Maps for Mobile (GMM) Android active users
GMM Android active users growing
BUT less quickly than Android activations
1-day-active to 30-day-active ratio less than other core Android Apps like YouTube & Gmail
Result: Shifted focus to everyday tasks, e.g., commuting. Led to launch of traffic alerts.
Every metric needs two basic elements: a goal and 1 or more signals
Goals
These are your high-level aspirations of the product, feature, or journey.
Decide what you hope users will do.
Signals
What would provide evidence that a goal has been met?
Almost always proxies for user thought and behavior.
You might need to aggregate multiple signals in order to determine if your goal’s been met
Larry and Sergey wanted to evaluate the effectiveness of search results, and they needed a good user-centric metric
None of the signals they were accustomed to measuring offered them the information they needed about their goal
Goal was for users to find what they need on the first try
Ask: What signal could they use?
“Long click”
It measured how long users took to come back after clicked on a search result
Wanted to measure a signal of quality, that was how long till someone came back
Unobvious metric
Doesn’t measure what happens on the site
Measures activity that does not happen on the site
Here, we're introducing the terms for "Minimum Viable Products" and "Minimum Delightful Products" to be explicit about the intent of the product you're about to launch.
Given your resources, time, and CUJs, are you launching an MVP or an MDP? What are the potential consequences or impacts of one versus the other?
If you had a choice, what would you rather want to launch? (answer: MDP)
You want to be proud of every version
v1 is really important today, so “launch and iterate” doesn't work quite the way it used to
So if we start with an MDP, each version can improve on all dimensions, so users have a delightful experience with each version.
If we start with an MVP, how might the previous version influence what changes we make to the next?
One last parting thought…
As you sail off into the sunset and head back to your teams with all wonderful thoughts of how you're going to change the world, tackle all your problems and begin shipping products which are truly excellent from the outset, there's one problem…
The kraken will be there to drag you down
Email, meetings, CLs, bugs… it is easy to get dragged down and just revert back to everything you did before
The Android and Identity teams work really hard to create a great initial device setup flow
But when we go into the field here's what we see: https://www.youtube.com/watch?v=4BUtA7mMLKA#t=20s
You cannot predict these behaviors without going to our users and learning on the ground.
During interview with packers, table height never came up as an issue. It was only through observation that this latent need emerged.
Ultimately, this insight influenced the design of future packing stations. (“After” picture to the right.)