In 2004, with only 150,000 registered users and very little revenue, Facebook raised $500,000 and less than a year later raised another $12.7 million. It was not the most popular social network nor was it even the most popular college social network. So what metrics were early indicators of it's future success?
How to Measure the Metrics that Determine Real Progress
1. How to Measure the
Metrics that Determine
Real Progress
by Trevor Owens
leanstartupmachine
Sunday, October 2, 2011
2. About Me
5 failed startups (vanity metric)
Web design agency
Imported electric scooters from China
Now working on Lean Startup Machine
P/M fit in 6 months
leanstartupmachine
Sunday, October 2, 2011
3. Lean Startup Machine
Metrics-Driven Startup Competition
Our winner is the team with the most
validated learning
Penultimate Goal: remove all subjectivity
Ultimate Goal: predict future success
leanstartupmachine
Sunday, October 2, 2011
5. Lean Startups Big Idea #1
The vast majority of startups fail NOT because
they couldnʼt build a product, but because no
one wanted the product.
One of Steve Blankʼs (many) Big Ideas from The Four Steps to the
Epiphany
Thanks to Brant Cooper & Patrick Vlaskovits for this content
Sunday, October 2, 2011
7. Lean Startups Big Idea #2
“The vast majority of successful [startups]
abandoned their initial plans and learned what
would and would not work in the market.”
-Clayton Christensen, The Innovatorʼs Dilemma
Thanks to Brant Cooper & Patrick Vlaskovits for this content
Sunday, October 2, 2011
8. Agenda
Introductions
Vanity vs. Good Metrics
Funnel & Cohort Metrics
Welcome to Innovation Accounting
Qualitative vs. Quantitative Metrics
leanstartupmachine
Sunday, October 2, 2011
10. Why do you want to
learn about metrics?
Sunday, October 2, 2011
11. Vanity Metrics
Google Analytics
There will always be something up and to
the right
Anything you read on TechCrunch
They make your competitors feel bad and
they don’t tell you anything about the
company
leanstartupmachine
Sunday, October 2, 2011
12. Vanity Metrics pt. 2
From TechCrunch:
“mDialog Raises $5 Million from
BlackBerry Partners”
“GroupMe Is Now Sending One Million
Texts Every Day”
“Google Plus hit 10M users in 2 weeks”
leanstartupmachine
Sunday, October 2, 2011
13. What Metrics Matter?
Facebook Example
150,000 users, little revenue
Many superior competitors
What would indicate their future success?
leanstartupmachine
Sunday, October 2, 2011
14. What Metrics Matter?
pt. 2
Facebook Example
75% of users visit one or more times per
day
Within one month of launching on a new
campus, can acquire 90% of the student
body
leanstartupmachine
Sunday, October 2, 2011
15. What Metrics Matter?
pt. 3
Value Hypothesis
How do we know we’re adding more
value to our customers lives?
Growth Hypothesis
Through what engine will we acquire
new customers?
leanstartupmachine
Sunday, October 2, 2011
16. What Metrics Matter?
pt. 4
Good Metrics (Three A’s)
Are Actionable
Can be Audited
Are Accessible
leanstartupmachine
Sunday, October 2, 2011
17. Assignment 1
Groups of 4 (10 minutes)
Research metrics on:
Google Plus
Groupon
Twitter
Explain which are vanity and
which are predictive
leanstartupmachine
Sunday, October 2, 2011
18. Tools
Funnel Metrics - AARRR
Acquisition - user visits
Activation - user registers
Retention - user comes back
Referral - friend of user signs up
Revenue - user pays for something
leanstartupmachine
Sunday, October 2, 2011
19. Tools pt. 2
Split Testing
Which performs better, A or B?
Wrong Uses
Red button or blue button?
Right Uses
Do users need to try the product
before paying for it?
leanstartupmachine
Sunday, October 2, 2011
20. Tools pt. 3
Cohort Analysis
Buckets users into the month or week
they joined
NEVER look at users in aggregate
Fact: New users are more engaged
leanstartupmachine
Sunday, October 2, 2011
21. Tools pt. 4
Funnel Metrics - AARRR
Acquisition
Activation
Retention
Referral
Revenue
leanstartupmachine
Sunday, October 2, 2011
22. Assignment 2
Groups of 4 (10 minutes)
Give an example of using split-
testing or cohort analysis to get
actionable metrics on:
Acquisition, Activation
Retention, Referral, Revenue
leanstartupmachine
Sunday, October 2, 2011
23. Innovation Accounting:
Tying It Together
The ultimate measure of progress in any
startup is rate of validated learning.
Innovation Accounting determines whether
your experiments are productive or not.
1. Set the baseline MVP
2.Optimize the MVP
3.Pivot or Persevere
leanstartupmachine
Sunday, October 2, 2011
24. Baseline MVP
What’s an MVP?
An experiment that starts with a
falsifiable hypothesis
Your first MVP may not be a product
Concierge MVPs Kick Ass
leanstartupmachine
Sunday, October 2, 2011
25. Optimization
Qualitative Metrics
Data expressed in people, words, &
feelings around actions
Small, detailed data set
Quantitative Metrics
Data expressed around actions
Large data set
leanstartupmachine
Sunday, October 2, 2011
26. Pivot or Persevere
What is a pivot?
Change in strategy based on learning from
market feedback
When do you pivot?
When each additional experiment leads to
less progress
leanstartupmachine
Sunday, October 2, 2011
28. Top 10 Ways
Entrepreneurs Pivot a
Lean Startup (Forbes)
Sunday, October 2, 2011
29. When’s It Enough?
Product Market Fit
At least 40% of users saying they
would be “very disappointed” without
your product.
Arbitrary, but defined after comparing
results across nearly 100 startups.
Those that struggle for traction are
always under 40%.
leanstartupmachine
Sunday, October 2, 2011
30. Thank You
Recommended Software
Unbounce
KISS Metrics
Survey.io
Wufoo
LinkShiftr
leanstartupmachine
Sunday, October 2, 2011