Shek Viswanathan, Product Manager at Qualtrics who previously built Words with Friends at Zynga, explains the product development loop and growth framework. His talk gives examples of north star metrics and other metrics to drive growth.
7. How do you know if
your
The Big Question
players,
customers,
users
><
8. How do you know if
your
enjoy
The Big Question
players,
customers,
users
playing,
using
><
><
9. How do you know if
your
enjoy
your
The Big Question
players,
customers,
users
playing,
using
game,
product,
service
?><
><
><
10. Understanding and improving your product health
comes down to breaking customer signals into a suite
of metrics and identifying key levers that drive growth.
12. Understand the Core Loop
Growth Playbook
Step 4.
Prioritize product features,
build and iterate
Step 3.
Identify which KPI influence growth
Step 2.
Build a Growth Model based on core KPI
from the product
Step 1.
Establish the core loop.
Measure Core Loop Engagement
13. Understand the Core Loop
Model for Growth
Growth Playbook
Step 4.
Prioritize product features,
build and iterate
Step 3.
Identify which KPI influence growth
Step 2.
Build a Growth Model based on core KPI
from the product
Step 1.
Establish the core loop.
Measure Core Loop Engagement
14. Understand the Core Loop
Model for Growth
Growth Playbook
Step 4.
Prioritize product features,
build and iterate
Step 3.
Identify which KPI influence growth
Step 2.
Build a Growth Model based on core KPI
from the product
Step 1.
Establish the core loop.
Measure Core Loop Engagement
15. Understand the Core Loop
Model for Growth
Identify Leverage
Growth Playbook
Step 4.
Prioritize product features,
build and iterate
Step 3.
Identify which KPI influence growth
Step 2.
Build a Growth Model based on core KPI
from the product
Step 1.
Establish the core loop.
Measure Core Loop Engagement
16. Understand the Core Loop
Model for Growth
Identify Leverage
Growth Playbook
Step 4.
Prioritize product features,
build and iterate
Step 3.
Identify which KPI influence growth
Step 2.
Build a Growth Model based on core KPI
from the product
Step 1.
Establish the core loop.
Measure Core Loop Engagement
17. Understand the Core Loop
Model for Growth
Identify Leverage
Growth Playbook
Prioritize
and Build
Step 4.
Prioritize product features,
build and iterate
Step 3.
Identify which KPI influence growth
Step 2.
Build a Growth Model based on core KPI
from the product
Step 1.
Establish the core loop.
Measure Core Loop Engagement
18. Understand the Core Loop
Model for Growth
Identify Leverage
Growth Playbook
Prioritize
and Build
Step 4.
Prioritize product features,
build and iterate
Step 3.
Identify which KPI influence growth
Step 2.
Build a Growth Model based on core KPI
from the product
Step 1.
Establish the core loop.
Measure Core Loop Engagement
19. Understand the Core Loop
Model for Growth
Identify Leverage
1. Understand the Core Loop
Prioritize
and Build
Step 4.
Prioritize product features,
build and iterate
Step 3.
Identify which KPI influence growth
Step 2.
Build a Growth Model based on core KPI
from the product
Step 1.
Establish the core loop.
Measure Core Loop Engagement
28. What constitutes a Core Loop?
Playing Words
Playing Hands
Posting
Grammin'
29. Completing the core loop is correlated to long term
retention.
Someone who plays a Word in 24 hours
after install, is more likely to return next
week than someone who doesn’t.
User with 7 friends in 10 days likely to
retain forever
Measuring core loop engagement is key to understanding product health
30. Only core action will not do. Other features that
complement the core loop need to be monitored.
31. Only core action will not do. Other features that
complement the core loop need to be monitored.
Creating a game with others
32. Only core action will not do. Other features that
complement the core loop need to be monitored.
Creating a game with others
Adding Friends, posting pictures, status updates, etc
34. What is a Product Core Loop?
Inner Loop
Low Investment
Daily Habit
35. What is a Product Core Loop?
Outer Loop
Medium/High Investment
Inner Loop
Low Investment
Daily Habit
36. What is a Product Core Loop?
Inner Loop
Outer Loop
Low Investment
Daily Habit
Medium/High Investment
Minor
Reward
Major
Reward
37. What is a Product Core Loop?
Inner Loop
Outer Loop
Low Investment
Daily Habit
Medium/High Investment
Trigger
Notification,
User Initiated
Minor
Reward
Major
Reward
38. What is a Product Core Loop?
Inner Loop
Outer Loop
Low Investment
Daily Habit
Medium/High Investment
Trigger
Notification,
User Initiated
Activity
Passive,
Active
Minor
Reward
Major
Reward
39. What is a Product Core Loop?
Inner Loop
Outer Loop
Low Investment
Daily Habit
Medium/High Investment
Trigger
Notification,
User Initiated
Activity
Passive,
Active
Start
Task
User Initiated
Minor
Reward
Major
Reward
40. What is a Product Core Loop?
Inner Loop
Outer Loop
Low Investment
Daily Habit
Medium/High Investment
Trigger
Notification,
User Initiated
Activity
Passive,
Active
Start
Task
User Initiated
Complete
Cross a
milestone
Minor
Reward
Major
Reward
41. 2. Modeling for Growth
Understand the Core Loop
Model for Growth
Identify Leverage
Prioritize
and Build
Step 4.
Prioritize product features,
build and iterate
Step 3.
Identify which KPI influence growth
Step 2.
Build a Growth Model based on core KPI
from the product
Step 1.
Establish the core loop.
Measure Core Loop Engagement
42. Now that we know what core loop action to measure,
how do we build a model for growth?
44. Modeling Audience Growth v1: Installs + Retention = DAU
Modeling Audience Growth v2: Over time WAU * Engagement = DAU
Modeling Revenue Growth v1: DAU * % Converting * ARPPU = Revenue
Building the model
45. Modeling Audience Growth v1: Installs + Retention = DAU
Modeling Audience Growth v2: Over time WAU * Engagement = DAU
Modeling Revenue Growth v1: DAU * % Converting * ARPPU = Revenue
Building the model
46. Modeling Audience Growth v1: Installs + Retention = DAU
Modeling Audience Growth v2: Over time WAU * Engagement = DAU
Modeling Revenue Growth v1: DAU * % Converting * ARPPU = Revenue
Building the model
47. Modeling Audience Growth v1: Installs + Retention = DAU
Modeling Audience Growth v2: Over time WAU * Engagement = DAU
Modeling Revenue Growth v1: DAU * % Converting * ARPPU = Revenue
Building the model
Similar models can be built for lower frequency products such as Travel or Payments
48. Modeling Audience Growth v1: Installs + Retention = DAU
Modeling Audience Growth v2: Over time WAU * Engagement = DAU
Modeling Revenue Growth v1: DAU * % Converting * ARPPU = Revenue
Building the model
Similar models can be built for lower frequency products such as Travel or Payments
Choosing a right platform to measure and instrument metrics
is as important as the act of measurement
49. A model can look like this over time…
Retention
Return Rates
Activations
Lapsers
Reactivations
Engagement
Cohorts
New Users Returning Users
Payers
High Value Payers
High, Med, Low
Engaged
Txn size, Conversion
50. So, now you know how to measure the core loop, and
also have a model for growth. How do you iterate
towards success state?
52. Success state is usually a single metric (often termed the Northstar
Metric) or a suite of metrics that are directly correlated with growth.
Product teams set goals for success state and rally the team around
those goals.
Example: Grow Active Users by x% YoY
Success State
53. Success state is usually a single metric (often termed the Northstar
Metric) or a suite of metrics that are directly correlated with growth.
Product teams set goals for success state and rally the team around
those goals.
Example: Grow Active Users by x% YoY
Success State
54. Success state is usually a single metric (often termed the Northstar
Metric) or a suite of metrics that are directly correlated with growth.
Product teams set goals for success state and rally the team around
those goals.
Example: Grow Active Users by x% YoY
Success State
55. 3. Identify Leverage
Understand the Core Loop
Model for Growth
Identify Leverage
Prioritize
and Build
Step 4.
Prioritize product features,
build and iterate
Step 3.
Identify which KPI influence growth
Step 2.
Build a Growth Model based on core KPI
from the product
Step 1.
Establish the core loop.
Measure Core Loop Engagement
56. Understanding where your product is in its life cycle is
important to establish what levers to pull to inflect
growth.
58. Early Stage
Product Life Cycle
Activation
Core Loop Engagement
Early Retention & Monetization
Time
Audience/
Revenue
59. Early Stage
Product Life Cycle
Growth Stage
Activation
Core Loop Engagement
Early Retention & Monetization
Stability in core metrics
Long Term Retention
Monetization
Virtuous Loops & Virality
Time
Audience/
Revenue
60. Early Stage
Product Life Cycle
Mature Product
Growth Stage
Activation
Core Loop Engagement
Early Retention & Monetization
Stability in core metrics
Long Term Retention
Monetization
Virtuous Loops & Virality
Identify Leverage
Optimize
Grow LTV
Time
Audience/
Revenue
62. Product A adds 100k new users every day and retains 5% of users after
30 days.
Product team has two ideas for features
Feature A - Increasing virality (installs)
Feature B - Increasing retention (D30)
What would you do?
63. Product A adds 100k new users every day and retains 5% of users after
30 days.
Product team has two ideas for features
Feature A - Increasing virality (installs)
Feature B - Increasing retention (D30)
What would you do?
64. Product A adds 100k new users every day and retains 5% of users after
30 days.
Product team has two ideas for features
Feature A - Increasing virality (installs)
Feature B - Increasing retention (D30)
What would you do?
65. In 1 Quarter
Feature A:
Hits 2.5M Users, with
5% D30 retention
Feature B:
Hits 2.1M Users, with
15% D30 retention
66. In 1 Year
Feature A:
Product Hits 4M
Users, with 5% D30
retention
Feature B:
Product Hits 5.5M
Users, with 15% D30
retention
67. Once you’ve identified leverage points, switch your hat
to that of a ruthless prioritizer and help the team
prioritize and build the product roadmap.
68. 4. Building Product Roadmap
Understand the Core Loop
Model for Growth
Identify Leverage
Prioritize
and Build
Step 4.
Prioritize product features,
build and iterate
Step 3.
Identify which KPI influence growth
Step 2.
Build a Growth Model based on core KPI
from the product
Step 1.
Establish the core loop.
Measure Core Loop Engagement
70. We talked about 4 things…
Understand the Core Loop
Model for Growth
Identify Leverage
Prioritize
and Build
Step 4.
Prioritize product features,
build and iterate
Step 3.
Identify which KPI influence growth
Step 2.
Build a Growth Model based on core KPI
from the product
Step 1.
Establish the core loop.
Measure Core Loop Engagement
71. Question: Did all of this add up for the product over
time? How do we measure this?
72. Most important health metric is LTV…
Retention
How long do your users retain? Is
this improving with every cohort?
Average Revenue per DAU
What % of your users convert?
How much do they spend per
day/month?
Lifetime Value
Revenue that a customer will bring
to your company over their life time
73. Is the core loop engagement getting stronger over
time? Will manifest in stronger LTVs over time.
Measuring LTV over time reveals changes in…
If retention is getting stronger - each same sized
cohort will generate more revenue over time.
Engagement
Retention
Monetization
Are users converting at the same levels over time?
If average revenue per daily active user stabilizes
or grows, it will show in increasing LTV.
74. Ultimately, breaking customer signals into a suite of
metrics and identifying key leverage points is vital for
strategizing and building a roadmap for growth.