SlideShare una empresa de Scribd logo
1 de 143
Descargar para leer sin conexión
Practical 
Lean Analytics 
Growthcon 
October 24, 2014 
@acroll
Kevin Costner is a lousy entrepreneur. 
Don’t sell what you can make. Make what you can sell.
The core of Lean 
is iteration.
Most startups don’t know what they’ll 
be when they grow up. 
Hotmail 
was a 
database 
company 
Flickr 
was going to 
be an MMO 
Twitter 
was a 
podcasting 
company 
Autodesk 
made 
desktop 
automation 
Paypal 
first built for 
Palmpilots 
Freshbooks 
was invoicing 
for a web 
design firm 
Wikipedia 
was to be 
written by 
experts only 
Mitel 
was a 
lawnmower 
company
Waterfall, agile, and lean 
(Why the old ways don’t work.)
Waterfall approach 
You know the problem and the solution.
Known set of 
requirements 
Known ways to satisfy 
them 
Spec Build Test Launch
Agile methodologies 
Know the problem, find the solution
Known set of 
requirements 
Unclear how to satisfy 
them 
Problem Build Test Viable? Launch 
statement 
Sprints 
Adjust 
Unknown set of
Lean approach 
First, know that you don’t know.
Product/market 
hypothesis Trial startup 
Possible problem 
space 
Product/ 
market 
hypothesis 
Trial startup 
Product/ 
market 
hypothesis 
Trial startup 
Trial startup 
Product/market 
hypothesis 
You are 
here 
PIVOT
Why now? 
First: High rate of change of digital 
technologies & channels.
Arbitron and radio data
Times a song in “heavy 
rotation” is played daily 
30 
15 
0 
6 26 
2007 2012
Why now? 
Second: It’s no longer about whether 
you can build it—it’s about whether 
anyone will care.
The Attention Economy 
“What information consumes 
is rather obvious: it consumes 
the attention of its recipients. 
Hence a wealth of information 
creates a poverty of attention, and a 
need to allocate that attention efficiently 
among the overabundance of 
information sources that might 
consume it.” 
(Computers, Communications and the Public Interest, pages 40-41, 
Herbert Simon Martin Greenberger, ed., The Johns Hopkins Press, 1971.)
Lit motors tests the risky part
Unfortunately, it is hard to be honest 
with ourselves.
Everyone’s idea is 
the best right? 
People love 
this part! 
(but that’s not always 
a good thing) 
No data, no 
learning. 
This is where 
things fall apart.
Analytics can help.
Analytics is the measurement of 
movement towards your business 
goals.
In a startup, the purpose of analytics is 
to iterate to product/market fit 
before the money runs out.
Some fundamentals.
A good metric is: 
Understandable 
If you’re busy 
explaining the 
data, you won’t 
be busy acting 
on it. 
Comparative 
Comparison is 
context. 
A ratio or rate 
The only way to 
measure 
change and roll 
up the tension 
between two 
metrics (MPH) 
Behavior 
changing 
What will you 
do differently 
based on the 
results you 
collect?
The 
simplest 
rule 
If a metric won’t change how 
you behave, it’s a 
bad 
metric. 
h"p://www.flickr.com/photos/circasassy/7858155676/
Metrics help you know yourself. 
Acquisition 
Hybrid 
Loyalty 
You are 
just like 
70% 
of retailers 
20% 
of retailers 
10% 
of retailers 
Customers that 
buy >1x in 90d 
Your customers 
will buy from you 
Once 
2-2.5 
per year 
>2.5 
per year 
Then you are 
in this mode 
1-15% 
15-30% 
>30% 
Focus on 
Low acquisition 
cost, high checkout 
Increasing return 
rates, market share 
Loyalty, selection, 
inventory size 
(Thanks to Kevin Hillstrom for this.)
Qualitative 
Unstructured, anecdotal, 
revealing, hard to 
aggregate, often too 
positive & reassuring. 
Warm and fuzzy. 
Quantitative 
Numbers and stats. 
Hard facts, less insight, 
easier to analyze; often 
sour and disappointing. 
Cold and hard.
Exploratory 
Speculative. Tries to find 
unexpected or 
interesting insights. 
Source of unfair 
advantages. 
Cool. 
Reporting 
Predictable. Keeps you 
abreast of the normal, 
day-to-day operations. 
Can be managed by 
exception. 
Necessary.
Rumsfeld on Analytics 
Things we 
know 
don’t 
know 
(Or rather, Avinash Kaushik channeling Rumsfeld) 
we know Are facts which may be wrong and 
should be checked against data. 
we don’t 
know 
Are questions we can answer by 
reporting, which we should baseline 
& automate. 
we know 
Are intuition which we should 
quantify and teach to improve 
effectiveness, efficiency. 
we don’t 
know 
Are exploration which is where 
unfair advantage and interesting 
epiphanies live.
Slicing and dicing data 
Feb Mar Apr May 
5,000 
Active users 
0 
Jan 
Cohort: 
Comparison of 
similar groups 
along a timeline. 
(this is the April cohort) 
A/B test: 
Changing one thing 
(i.e. color) and 
measuring the 
result (i.e. revenue.) 
Multivariate 
analysis 
Changing several 
things at once to 
see which correlates 
with a result. 
☀☁☀☁ 
Segment: 
Cross-sectional 
comparison of all 
people divided by 
some attribute (age, 
gender, etc.) 
☀ 
☁
Which of these two companies 
is doing better?
January February March April May 
Is this company Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50 
growing or stagnating? 
Cohort 1 2 3 4 5 
January $5 $3 $2 $1 $0.5 
February $6 $4 $2 $1 
March $7 $6 $5 
April $8 $7 
May $9 
How about 
this one?
Cohort 1 2 3 4 5 
January $5 $3 $2 $1 $0.5 
February $6 $4 $2 $1 
March $7 $6 $5 
April $8 $7 
May $9 
Averages $7 $5 $3 $1 $0.5 
Look at the 
same data 
in cohorts
Lagging 
Historical. Shows you 
how you’re doing; 
reports the news. 
Example: sales. 
Explaining the 
past. 
Leading 
Forward-looking. 
Number today that 
predicts tomorrow; 
reports the news. 
Example: pipeline. 
Predicting the 
future.
Some examples 
A Facebook user reaching 7 friends within 10 days of signing up 
(Chamath Palihapitiya) 
If someone comes back to Zynga a day after signing up for a game, 
they’ll probably become an engaged, paying user (Nabeel Hyatt) 
A Dropbox user who puts at least one file in one folder on one device 
(ChenLi Wang) 
Twitter user following a certain number of people, and a certain 
percentage of those people following the user back (Josh Elman) 
A LinkedIn user getting to X connections in Y days (Elliot Schmukler) 
(From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
Which means it’s time to talk 
about correlation.
10000 
1000 
100 
10 
1 
Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec 
Ice cream consumption Drownings
Correlated 
Two variables that are 
related (but may be 
dependent on 
something else.) 
Ice cream & 
drowning. 
Causal 
An independent variable 
that directly impacts a 
dependent one. 
Summertime & 
drowning.
A leading, causal metric 
is a superpower. 
h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
Why is Nigerian spam so badly 
written?
Experienced scammers expect a “strike rate” of 1 or 2 replies per 1,000 messages 
emailed; they expect to land 2 or 3 “Mugu” (fools) each week. 
One scammer boasted “When you get a reply it’s 70% sure you’ll get the money” 
“By sending an email that repels all but the most gullible,” says [Microsoft Researcher 
Corman] Herley, “the scammer gets the most promising marks to self-select, and tilts 
the true to false positive ratio in his favor.” 
This would be horribly 
inefficient since 
humans are involved. 
Good language (10% conversion) 
Not-gullible (.07% conversion) 
Aunshul Rege of Rutgers University, USA in 2009 
1000 emails 
Bad language (0.1% conversion) 
1-2 responses 
Gullible (70% conversion) 
1 fool and their money, parted. 
1000 emails 
100 responses 
1 fool and their money, parted.
Turns out the word “Nigeria” is the best 
way to identify promising prospects.
Nigerian spammers 
really understand their target market. 
They see past vanity metrics.
The Lean Analytics framework.
Sustainable growth comes through the 
actions of your customers. 
- Eric Ries
Eric’s three engines of growth 
Virality 
Make people 
invite friends. 
How many they 
tell, how fast they 
tell them. 
Price 
Spend money to 
get customers. 
Customers are 
worth more than 
they cost. 
Stickiness 
Keep people 
coming back. 
Approach 
Get customers 
faster than you 
lose them. 
Math that 
matters
Dave’s Pirate Metrics 
AARRR Acquisition 
How do your users become aware of you? 
SEO, SEM, widgets, email, PR, campaigns, blogs ... 
Activation 
Do drive-by visitors subscribe, use, etc? 
Features, design, tone, compensation, affirmation ... 
Retention 
Does a one-time user become engaged? 
Notifications, alerts, reminders, emails, updates... 
Revenue 
Do you make money from user activity? 
Transactions, clicks, subscriptions, DLC, analytics... 
Referral 
Do users promote your product? 
Email, widgets, campaigns, likes, RTs, affiliates...
Gate 
Stage 
EMPATHY I’ve found a real, poorly-met need that a 
reachable market faces. 
STICKINESS I’ve figured out how to solve the problem in a 
way they will keep using and pay for. 
VIRALITY I’ve found ways to get them to tell their friends, 
either intrinsically or through incentives. 
REVENUE The users and features fuel growth organically 
and artificially. 
SCALE I’ve found a sustainable, scalable business with 
the right margins in a healthy ecosystem. 
The five stages
Empathy stage: 
Localmind hacks Twitter 
Needed to find out if a core assumption—strangers answering 
questions—was valid. 
Ran Twitter experiment instead of writing code 
Asked senders of geolocated Tweets from Times Square random 
questions; counted response rate 
Conclusion: high enough to proceed
LikeBright’s mechanical turk 
Used Mechanical Turk, Google Voice to speak w/ 
100 single women; paid $2. The interviews 
lasted typically around 10-15 minutes. 
Simple interview script with open-ended 
questions, since he was digging into the problem 
validation stage of his startup. 
Founder Nick Soman: “I was amazed at the 
feedback I got. We were able to speak with one 
hundred single women that met our criteria in 
four hours on one evening.” 
Went back to TechStars and got accepted. 
LikeBright’s website is now live with a 50% 
female user base, and recently raised a round of 
funding. 
“Since that first foray into interviewing customers, 
I’ve probably spoken with over a thousand 
people through Mechanical Turk,”
How to avoid leading the witness 
Avoid biased wording, preconceptions, or 
a giveaway appearance. Word your 
surveys carefully to be neutral. 
Get them to purchase. Ask them to pay. Demand real 
Ask “why” several times. Leave lingering, uncomfortable pauses 
Don’t tip your hand 
Make the question real 
Keep digging 
Look for other clues Have a colleague make notes of when they react, or of their body
Stickiness stage: 
qidiq streamlines invites 
Survey owner adds recipient to group 
Survey owner asks question 
Recipient reads survey question 
Recipient responds to question 
Recipient sees survey results 
(Later, if needed…) 
Recipient visits site; no password! 
Recipient does password recovery 
One-time link sent to email 
Recipient creates password 
Recipient can edit profile, etc. 
Survey owner adds recipient to group 
Survey owner asks question 
Recipient gets invite 
Recipient installs mobile app 
Recipient creates account, profile 
Recipient can edit profile, etc. 
Recipient reads survey question 
Recipient responds to question 
Recipient sees survey results 
10-25% RESPONSE RATE 
70-90% RESPONSE RATE
1200 
1000 
800 
600 
400 
200 
0 
January February 
1 2 3 4 5 6 7 8 9 
Days since last engagement 
25000 
20000 
15000 
10000 
5000 
0 
Disengaged 
(>10 days) 
Number of users 
A better approach to engagement 
This is a 
good thing.
Who is worth more? 
A Lifetime: 
B Lifetime: 
Today 
$200 
$200 
Roberto Medri, Etsy 
Visits
Virality stage: Timehop focuses 
on content sharing 
Focused on percent of daily active users that share their content 
Aiming for 20-30% of DAU sharing 
“All that matters now is virality. Everything else—be 
it press, publicity stunts or something else—is like 
pushing a rock up a mountain: it will never scale. 
But being viral will.” 
- Jonathan Wegener, co-founder
------------------------------------------------------ 
Get your free private email at http://www.hotmail.com 
------------------------------------------------------
v ≠ 1, pt = δp0 (1 – vt+1) / (1 – v) + p0 
http://robert.zubek.net/blog/2008/01/30/viral-coefficient-calculation/ 
Viral coefficient
Or simpler 
x - > 1 
Users Viral 
coefficient 
Churn & 
abandonment
Revenue stage: Backupify’s 
Customer Acquisition Payback 
Initially focused on site visitors 
Then focused on trials 
Then switched to signups 
Today, MRR 
In early 2010, CAC was $243 and ARPU was only $39 
Pivoted to target business users 
CLV-to-CAC today is 5-6x 
Now they track Customer Acquisition Payback 
Target is less than 12 months
Scale stage: 
Incremental order cost 
Marginal costs 
Fixed costs
Six business model archetypes. 
E-commerce SaaS Mobile Media 
app 
User-gen 
content 
2-sided 
market 
The business you’re in
(Which means eye 
charts like these.) 
Customer Acquisition Cost 
paid direct search wom inherent 
virality 
VISITOR 
Freemium/trial offer 
Enrollment 
User 
Disengaged User 
Freemium 
churn 
Cancel 
Engaged User 
Free user 
disengagement 
Reactivate 
Trial abandonment 
Cancel 
rate 
Invite Others 
Upselling 
rate Upselling 
Paying Customer 
Reactivation 
rate 
Paid 
conversion 
FORMER USERS 
User Lifetime Value 
Reactivate 
Capacity Limit 
Support data 
FORMER CUSTOMERS 
Customer Lifetime Value 
Viral coefficient 
Viral rate 
Resolution 
Account Cancelled Billing Info Exp. 
Paid Churn Rate 
Tiering 
Trial Over Disengaged Dissatisfied
Acquisition 
Customer Acquisition Cost 
paid direct search wom inherent 
virality 
VISITOR 
Freemium/trial offer 
Enrollment 
User 
Disengaged User 
Freemium 
churn 
Cancel 
Engaged User 
Free user 
disengagement 
Reactivate 
Trial abandonment 
Cancel 
rate 
Invite Others 
Upselling 
rate Upselling 
Paying Customer 
Reactivation 
rate 
Paid 
conversion 
FORMER USERS 
User Lifetime Value 
Reactivate 
Capacity Limit 
Support data 
FORMER CUSTOMERS 
Customer Lifetime Value 
Viral coefficient 
Viral rate 
Resolution 
Account Cancelled Billing Info Exp. 
Paid Churn Rate 
Tiering 
Trial Over Disengaged Dissatisfied
Activation 
Customer Acquisition Cost 
paid direct search wom inherent 
virality 
VISITOR 
Freemium/trial offer 
Enrollment 
User 
Disengaged User 
Freemium 
churn 
Cancel 
Engaged User 
Free user 
disengagement 
Reactivate 
Trial abandonment 
Cancel 
rate 
Invite Others 
Upselling 
rate Upselling 
Paying Customer 
Reactivation 
rate 
Paid 
conversion 
FORMER USERS 
User Lifetime Value 
Reactivate 
Capacity Limit 
Support data 
FORMER CUSTOMERS 
Customer Lifetime Value 
Viral coefficient 
Viral rate 
Resolution 
Account Cancelled Billing Info Exp. 
Paid Churn Rate 
Tiering 
Trial Over Disengaged Dissatisfied
Retention 
Customer Acquisition Cost 
paid direct search wom inherent 
virality 
VISITOR 
Freemium/trial offer 
Enrollment 
User 
Disengaged User 
Freemium 
churn 
Cancel 
Engaged User 
Free user 
disengagement 
Reactivate 
Trial abandonment 
Cancel 
rate 
Invite Others 
Upselling 
rate Upselling 
Paying Customer 
Reactivation 
rate 
Paid 
conversion 
FORMER USERS 
User Lifetime Value 
Reactivate 
Capacity Limit 
Support data 
FORMER CUSTOMERS 
Customer Lifetime Value 
Viral coefficient 
Viral rate 
Resolution 
Account Cancelled Billing Info Exp. 
Paid Churn Rate 
Tiering 
Trial Over Disengaged Dissatisfied
Revenue 
Customer Acquisition Cost 
paid direct search wom inherent 
virality 
VISITOR 
Freemium/trial offer 
Enrollment 
User 
Disengaged User 
Freemium 
churn 
Cancel 
Engaged User 
Free user 
disengagement 
Reactivate 
Trial abandonment 
Cancel 
rate 
Invite Others 
Upselling 
rate Upselling 
Paying Customer 
Reactivation 
rate 
Paid 
conversion 
FORMER USERS 
User Lifetime Value 
Reactivate 
Capacity Limit 
Support data 
FORMER CUSTOMERS 
Customer Lifetime Value 
Viral coefficient 
Viral rate 
Resolution 
Account Cancelled Billing Info Exp. 
Paid Churn Rate 
Tiering 
Trial Over Disengaged Dissatisfied
Revenue 
Customer Acquisition Cost 
paid direct search wom inherent 
virality 
VISITOR 
Freemium/trial offer 
Enrollment 
User 
Disengaged User 
Freemium 
churn 
Cancel 
Engaged User 
Free user 
disengagement 
Reactivate 
Trial abandonment 
Cancel 
rate 
Invite Others 
Upselling 
rate Upselling 
Paying Customer 
Reactivation 
rate 
Paid 
conversion 
FORMER USERS 
User Lifetime Value 
Reactivate 
Capacity Limit 
Support data 
FORMER CUSTOMERS 
Customer Lifetime Value 
Viral coefficient 
Viral rate 
Resolution 
Account Cancelled Billing Info Exp. 
Paid Churn Rate 
Tiering 
Trial Over Disengaged Dissatisfied
Model + Stage = One Metric That Matters. 
The business you’re in 
E-Com SaaS Mobile 2-Sided Media UCG 
One Metric 
That Matters. 
Empathy 
Stickiness 
Virality 
Revenue 
Scale 
The stage you’re at
Really? Just one?
Yes, one.
In a startup, focus is hard to achieve.
Having only one metric 
addresses this problem.
www.theeastsiderla.com
Moz cuts down on metrics 
SaaS-based SEO toolkit in the scale stage. Focused on net adds. 
Was a marketing campaign successful? 
Were customer complaints lowered? 
Was a product upgrade valuable? 
Net adds up: 
Can we acquire more valuable customers? 
What product features can increase engagement? 
Can we improve customer support? 
Net adds flat: 
Are the new customers not the right segment? 
Did a marketing campaign fail? 
Did a product upgrade fail somehow? 
Is customer support falling apart? 
Net adds down:
Metrics are like squeeze toys. 
http://www.flickr.com/photos/connortarter/4791605202/
Empathy 
Stickiness 
Virality 
Revenue 
Scale 
E-commerce 
Mobile 
app 
User-gen 
content 
SaaS Media 
2-sided 
market 
Interviews; qualitative results; quantitative scoring; surveys 
Loyalty, 
conversion 
CAC, shares, 
reactivation 
(Money from transactions) 
Transaction, 
CLV 
Affiliates, 
white-label 
Engagement, 
churn 
Inherent 
virality, CAC 
(Money from active users) 
Upselling, 
CAC, CLV 
API, magic #, 
mktplace 
Content, 
spam 
Invites, 
sharing 
(Money from ad clicks) 
Ads, 
donations 
Analytics, 
user data 
Inventory, 
listings 
SEM, sharing 
Transactions, 
commission 
Other 
verticals 
Downloads, 
churn, virality 
WoM, app 
ratings, CAC 
CLV, 
ARPDAU 
Spinoffs, 
publishers 
Traffic, visits, 
returns 
Content 
virality, SEM 
CPE, affiliate 
%, eyeballs 
Syndication, 
licenses
Better: bit.ly/BigLeanTable
Drawing some lines in the sand.
A company loses a quarter of its 
customers every year. 
Is this good or bad?
Baseline: 
2-5% monthly churn 
• The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s 
on the job. 
• Get below a 5% monthly churn rate before you know you’ve got a 
business that’s ready to grow (Mark MacLeod) and around 2% 
before you really step on the gas (David Skok) 
• Last-ditch appeals and reactivation can have a big impact. 
Facebook’s “don’t leave” reduces attrition by 7%.
Not knowing what normal is 
makes you do unwise things.
Baseline: 
5-7% growth a week 
“A good growth rate during YC 
is 5-7% a week,” he says. “If 
you can hit 10% a week you're 
doing exceptionally well. If you 
can only manage 1%, it's a sign 
you haven't yet figured out 
what you're doing.” At revenue 
stage, measure growth in 
revenue. Before that, measure 
growth in active users. 
Paul Graham, Y Combinator 
• Are there enough people who really care 
enough to sustain a 5% growth rate? 
• Don’t strive for a 5% growth at the expense 
of really understanding your customers 
and building a meaningful solution 
• Once you’re a pre-revenue startup at or 
near product/market fit, you should have 
5% growth of active users each week 
• Once you’re generating revenues, they 
should grow at 5% a week
Baseline: 
10% visitor engagement/day 
30% of users/month use web or mobile app 
10% of users/day use web or mobile app 
1%of users/day use it concurrently 
Fred Wilson’s social ratios
Baseline: 
Calculating customer lifetime 
25% 
5% 
monthly churn 
monthly churn 
100/25=4 
100/5=20 
The average 
The average 
customer lasts 
customer lasts 
4 months 
20 months 
2% 
monthly churn 
100/2=50 
The average 
customer lasts 
50 months
Baseline: 
CAC under 1/3 of CLV 
• CLV is wrong. CAC Is probably wrong, too. 
• Time kills all plans: It’ll take a long time to find 
out whether your churn and revenue projections 
are right 
• Cashflow: You’re basically “loaning” the 
customer money between acquisition and CLV. 
• It keeps you honest: Limiting yourself to a 
CAC of only a third of your CLV will forces you 
to verify costs sooner. 
Lifetime of 20 mo. 
$30/mo. per 
customer 
$600 CLV 
1/3 spend 
$200 CAC 
Now segment 
those users!
Etsy 
• Online store for creative types, founded 2005 
• $525M Gross Merchandise Sales in 2011, with 
19,000,000 members and 800,000 active 
shops offering 15,000,000 items for sale 
• 1.4B pageviews per month ~2M iPhone app 
downloads 
• Thin revenues: Etsy makes only $0.20 or 3.5% 
margin 
• Heavy focus on Customer Lifetime Value (buyer 
and seller) 
• Actually residual lifetime value; they take this 
pretty seriously.
Etsy 
• The best customers to target are 
• Recent high-profile customers 
• Old-time best customers about to 
churn or just churned 
• Tiered campaigns 
• Bronze/silver customers: reinforcement, 
nudges 
• Gold customers: premium services 
• Platinum customers: recognition 
• What they watch: 
• Growth of individual product categories 
• Time to first sale by a user 
• Average order value 
• Percentage of visits that convert to a 
sale 
• Percentage of return buyers 
• Distinct sellers within a product 
category 
• Time-to-first-sale and average order 
value by product category 
Roberto Medri, Etsy
The Lean Analytics cycle
Pick a KPI Draw a line 
Draw a new line 
Pivot or 
give up 
Try again 
Success! 
Did we move the 
needle? 
Measure 
the results 
Design a test 
Make changes 
in production 
Find a potential 
improvement 
With data: 
find a 
commonality 
Without data: 
make a good 
guess 
Hypothesis
Do AirBnB hosts 
get more business 
if their property is 
professionally 
photographed?
Gut instinct (hypothesis) 
Professional photography helps AirBnB’s business 
Candidate solution (MVP) 
20 field photographers posing as employees 
Measure the results 
Compare photographed listings to a control group 
Make a decision 
Launch photography as a new feature for all hosts
5,000 shoots per month 
by February 2012
Hang on a second.
REALLY? 
Gut instinct (hypothesis) 
Professional photography helps AirBnB’s business
Pick a KPI Draw a line 
Draw a new line 
Pivot or 
give up 
Try again 
Success! 
Did we move the 
needle? 
Measure 
the results 
Design a test 
Make changes 
in production 
Find a potential 
improvement 
With data: 
find a 
commonality 
Without data: 
make a good 
guess 
Hypothesis
“Gee, those 
houses that do 
well look really 
nice.” 
Maybe it’s the 
camera. 
With data: 
find a commonality 
“Computer: What 
do all the 
highly rented 
houses have in 
common?” 
Camera model. 
Without data: make a 
good guess
Circle of Moms: Not enough engagement 
• Too few people were 
actually using the 
product 
• Less than 20% of any 
circles had any activity 
after their initial creation 
• A few million monthly 
uniques from 10M 
registered users, but no 
sustained traction 
• They found moms were far more engaged 
• Their messages to one another were on average 50% longer 
• They were 115% more likely to attach a picture to a post they wrote 
• They were 110% more likely to engage in a threaded (i.e. deep) 
conversation 
• Circle owners’ friends were 50% more likely to engage with the circle 
• They were 75% more likely to click on Facebook notifications 
• They were 180% more likely to click on Facebook news feed items 
• They were 60% more likely to accept invitations to the app 
• Pivoted to the new market, including a name change 
• By late 2009, 4.5M users and strong engagement 
• Sold to Sugar, inc. in early 2012
Landing page design A/B testing 
Cohort analysis General analytics 
URL shortening 
Funnel analytics 
Influencer Marketing 
Publisher analytics 
SaaS analytics 
Gaming analytics 
User analytics Spying on users 
User interaction Customer User segmentation satisfaction KPI dashboards
Growth hacking 
(is a word you should hate but will hear a lot about.)
Growth hacking, demystified. 
Find 
correlation 
Test 
causality 
Optimize the 
causal factor 
Pick a metric 
to change
http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/ 
Is social action a leading 
indicator of donation?
http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/ 
Is mobile use?
Guerrilla 
marketing 
Data-driven 
learning 
GROWTH 
HACKING 
Subversiveness
AirBnB and Craigslist
Take baby steps.
Netflix
Tesla 
http://www.hdwallpapersinn.com/wp-content/uploads/2012/12/600-tesla.jpg
Twitter’s 140-character 
limit isn’t arbitrary. It’s 
constrained by the size 
http://i.i.cbsi.com/cnwk.1d/i/tim/2011/11/18/ 
sms_screen_twitter_activity_stream_270x405.png
http://www.flickr.com/photos/bootbearwdc/1243690099/ 
Think subversively.
To summarize:
1. Define your business model
2. Draw a 
system 
diagram
3. Decide what 
stage you’re at
4. Identify the One Metric That Matters 
(usually the one that is most broken)
5. Use the cycle to 
experiment until 
you’ve achieved 
the desired result.
6. Set up monitoring for this metric in 
case it breaks, and choose a new 
OMTM
Conclusions
“The most important figures that one 
needs for management are unknown 
or unknowable, but successful 
management must nevertheless take 
account of them.” 
Lloyd S. Nelson
Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
ARCHIMEDES 
HAD TAKEN 
BATHS BEFORE.
Once, a leader convinced others 
in the absence of data.
Now, a leader knows 
what questions to ask.
Ben Yoskovitz 
byosko@gmail.com 
@byosko 
Alistair Croll 
acroll@gmail.com 
@acroll
The mobile app! 
customer lifecycle! 
Ratings 
Reviews 
Search 
Leaderboards 
Purchases 
App store! 
App sales 
Downloads 
Installs 
Play 
Disengagement 
Reactivation 
Uninstallation 
Disengagement 
Account" 
creation 
Virality 
Downloads," 
Gross revenue 
ARPU 
Activation 
Churn, CLV 
In-app" 
purchases 
Legitimate 
Incentivized 
Fraudulent 
Ratings!
Building message maps
Build a message map. 
1. Understand the stages a buyer goes through 2. Create benefits; mitigate objections 3. Target the message to the stage the audience is at
Everyone in the world 
A. I need a car 
I should buy 
B. a car 
It should be 
C. a hybrid 
I should buy 
D. a Honda Civic
Everyone in the world 
A. I need a car 
People who want to drive 
I should buy 
B. a car 
Prospective car buyers 
It should be 
C. a hybrid 
People looking for a hybrid 
I should buy 
D. a Honda Civic 
Honda Civic Hybrid owners
“Isn’t it time you got out of the 
city?” campaign showing how cars 
make nature accessible & ridiculing 
urban hipsters. 
Ads showing how cars are needed 
any time (pregnancy, errands, urgent 
business) and how a car is a 
“personal assistant.” 
Urgency (“every time you drive a 
non-hybrid car you kill the planet a 
little”) and testimonials from buyers 
who’ve saved money. 
Honda branding ads and model-specific 
promotions. 
Follow-up satisfaction campaign to 
encourage buyers to tell their friends 
Everyone in the world 
A. I need a car 
People who want to drive 
“I need a vehicle to get 
around, be productive, and 
enjoy my life.” 
I should buy 
B. a car 
Prospective car buyers 
“I want to own a car because it’s 
convenient; it’s a personal 
relationship; I don’t trust others.” 
It should be 
C. a hybrid 
People looking for a hybrid 
“I want to save money and fuel. I 
also care about the environment 
and want to be seen as ‘green’.” 
I should buy 
D. a Honda Civic 
Honda Civic Hybrid owners
Everyone in the world 
People who want to drive 
“I need a vehicle to get 
around, be productive, and 
enjoy my life.” 
Prospective car buyers 
“I want to own a car because it’s 
convenient; it’s a personal 
relationship; I don’t trust others.” 
People looking for a hybrid 
“I want to save money and fuel. I 
also care about the environment 
and want to be seen as ‘green’.” 
Honda Civic Hybrid owners 
Those who don’t need cars 
• I’m too young to drive 
• I’m too old to drive 
• I can walk or take public 
transit 
Car users who won’t buy 
• It’s too expensive for me 
• I will use a shared car service 
• It’ll get stolen 
Those who won’t buy hybrids 
• Hybrids are gutless 
• Batteries are toxic & explosive 
• In the end it costs more than 
it saves 
I will buy another brand 
• I buy domestic 
• I’ve always driven a VW 
• Toyotas are reliable 
• I want something prestigious 
A. I need a car 
I should buy 
B. a car 
It should be 
C. a hybrid 
I should buy 
D. a Honda Civic
Everyone in the world 
People who want to drive 
“I need a vehicle to get 
around, be productive, and 
enjoy my life.” 
Prospective car buyers 
“I want to own a car because it’s 
convenient; it’s a personal 
relationship; I don’t trust others.” 
People looking for a hybrid 
“I want to save money and fuel. I 
also care about the environment 
and want to be seen as ‘green’.” 
Honda Civic Hybrid owners 
Those who don’t need cars 
• I’m too young to drive 
• I’m too old to drive 
• I can walk or take public 
transit 
Car users who won’t buy 
• It’s too expensive for me 
• I will use a shared car service 
• It’ll get stolen 
Those who won’t buy hybrids 
• Hybrids are gutless 
• Batteries are toxic & explosive 
• In the end it costs more than 
it saves 
I will buy another brand 
• I buy domestic 
• I’ve always driven a VW 
• Toyotas are reliable 
• I want something prestigious 
Sponsor a driving school 
“Give the gift of driving” 
campaign for grandparents. 
PR on dangers of commuting, 
pedestrian deaths 
Financing, cashback 
Sell to carshares; 
underscore their limitations 
Theft warranty, tracking 
services, high-end locks 
Independent tests, 
standard metrics (0-60 in X) 
Lab research, studies 
ROI calculator; 
replacement programs 
Prove Honda hires US workers 
“Time to leave Germany” ads 
Spontaneous accel. stories 
Premium brand (Acura) 
A. I need a car 
I should buy 
B. a car 
It should be 
C. a hybrid 
I should buy 
D. a Honda Civic
“Isn’t it time you got out of the 
city?” campaign showing how cars 
make nature accessible & ridiculing 
urban hipsters. 
Ads showing how cars are needed 
any time (pregnancy, errands, urgent 
business) and how a car is a 
“personal assistant.” 
Urgency (“every time you drive a 
non-hybrid car you kill the planet a 
little”) and testimonials from buyers 
who’ve saved money. 
Honda branding ads and model-specific 
promotions. 
Follow-up satisfaction campaign to 
encourage buyers to tell their friends 
Everyone in the world 
People who want to drive 
“I need a vehicle to get 
around, be productive, and 
enjoy my life.” 
Prospective car buyers 
“I want to own a car because it’s 
convenient; it’s a personal 
relationship; I don’t trust others.” 
People looking for a hybrid 
“I want to save money and fuel. I 
also care about the environment 
and want to be seen as ‘green’.” 
Honda Civic Hybrid owners 
Those who don’t need cars 
• I’m too young to drive 
• I’m too old to drive 
• I can walk or take public 
transit 
Car users who won’t buy 
• It’s too expensive for me 
• I will use a shared car service 
• It’ll get stolen 
Those who won’t buy hybrids 
• Hybrids are gutless 
• Batteries are toxic & explosive 
• In the end it costs more than 
it saves 
I will buy another brand 
• I buy domestic 
• I’ve always driven a VW 
• Toyotas are reliable 
• I want something prestigious 
Sponsor a driving school 
“Give the gift of driving” 
campaign for grandparents. 
PR on dangers of commuting, 
pedestrian deaths 
Financing, cashback 
Sell to carshares; 
underscore their limitations 
Theft warranty, tracking 
services, high-end locks 
Independent tests, 
standard metrics (0-60 in X) 
Lab research, studies 
ROI calculator; 
replacement programs 
Prove Honda hires US workers 
“Time to leave Germany” ads 
Spontaneous accel. stories 
Premium brand (Acura) 
A. I need a car 
I should buy 
B. a car 
It should be 
C. a hybrid 
I should buy 
D. a Honda Civic

Más contenido relacionado

Más de Yusuke Takahashi, PhD

Introducing ChatCenter Ai (version 20190305) JA
Introducing ChatCenter Ai (version 20190305) JAIntroducing ChatCenter Ai (version 20190305) JA
Introducing ChatCenter Ai (version 20190305) JAYusuke Takahashi, PhD
 
Introducing ChatCenter Ai (version 20190129)
Introducing ChatCenter Ai (version 20190129)Introducing ChatCenter Ai (version 20190129)
Introducing ChatCenter Ai (version 20190129)Yusuke Takahashi, PhD
 
(Conference) Artificial Intelligence for 
Supporting Decisions in Businesses
(Conference) Artificial Intelligence for 
Supporting Decisions in Businesses(Conference) Artificial Intelligence for 
Supporting Decisions in Businesses
(Conference) Artificial Intelligence for 
Supporting Decisions in BusinessesYusuke Takahashi, PhD
 
- (Conference) Artificial Intelligence for 
Supporting Decisions in Businesses
- (Conference) Artificial Intelligence for 
Supporting Decisions in Businesses- (Conference) Artificial Intelligence for 
Supporting Decisions in Businesses
- (Conference) Artificial Intelligence for 
Supporting Decisions in BusinessesYusuke Takahashi, PhD
 
(Training) Artificial Intelligence for 
Supporting Decisions in Businesses
(Training) Artificial Intelligence for 
Supporting Decisions in Businesses(Training) Artificial Intelligence for 
Supporting Decisions in Businesses
(Training) Artificial Intelligence for 
Supporting Decisions in BusinessesYusuke Takahashi, PhD
 
Invited Lecture at GnB Accelerator in Jakarta
Invited Lecture at GnB Accelerator in JakartaInvited Lecture at GnB Accelerator in Jakarta
Invited Lecture at GnB Accelerator in JakartaYusuke Takahashi, PhD
 
Case Studies of Awesome Growth Hacks
Case Studies of Awesome Growth HacksCase Studies of Awesome Growth Hacks
Case Studies of Awesome Growth HacksYusuke Takahashi, PhD
 
NIKKEI COMPUTER’s Seminar (07.17.2014)
NIKKEI COMPUTER’s Seminar (07.17.2014)NIKKEI COMPUTER’s Seminar (07.17.2014)
NIKKEI COMPUTER’s Seminar (07.17.2014)Yusuke Takahashi, PhD
 
Appsocially Deck for New Context Conference 2013, San Francisco (Nov 04, 2013)
Appsocially Deck for New Context Conference 2013, San Francisco (Nov 04, 2013)Appsocially Deck for New Context Conference 2013, San Francisco (Nov 04, 2013)
Appsocially Deck for New Context Conference 2013, San Francisco (Nov 04, 2013)Yusuke Takahashi, PhD
 
How NOT to Suck at App Distribution - Quick Start Guide - Appsocially's Growt...
How NOT to Suck at App Distribution - Quick Start Guide - Appsocially's Growt...How NOT to Suck at App Distribution - Quick Start Guide - Appsocially's Growt...
How NOT to Suck at App Distribution - Quick Start Guide - Appsocially's Growt...Yusuke Takahashi, PhD
 
vidyard @ Y Combinator's Ad Innovation Conference
vidyard @ Y Combinator's Ad Innovation Conferencevidyard @ Y Combinator's Ad Innovation Conference
vidyard @ Y Combinator's Ad Innovation ConferenceYusuke Takahashi, PhD
 
論文の読み方・書き方(第1回講義資料、10/3、2011年度前期c2-19、高橋雄介)
論文の読み方・書き方(第1回講義資料、10/3、2011年度前期c2-19、高橋雄介)論文の読み方・書き方(第1回講義資料、10/3、2011年度前期c2-19、高橋雄介)
論文の読み方・書き方(第1回講義資料、10/3、2011年度前期c2-19、高橋雄介)Yusuke Takahashi, PhD
 

Más de Yusuke Takahashi, PhD (20)

Introducing ChatCenter Ai (version 20190305) JA
Introducing ChatCenter Ai (version 20190305) JAIntroducing ChatCenter Ai (version 20190305) JA
Introducing ChatCenter Ai (version 20190305) JA
 
Introducing ChatCenter Ai (version 20190129)
Introducing ChatCenter Ai (version 20190129)Introducing ChatCenter Ai (version 20190129)
Introducing ChatCenter Ai (version 20190129)
 
Introduction - Yusuke Takahashi PhD
Introduction - Yusuke Takahashi PhDIntroduction - Yusuke Takahashi PhD
Introduction - Yusuke Takahashi PhD
 
(Conference) Artificial Intelligence for 
Supporting Decisions in Businesses
(Conference) Artificial Intelligence for 
Supporting Decisions in Businesses(Conference) Artificial Intelligence for 
Supporting Decisions in Businesses
(Conference) Artificial Intelligence for 
Supporting Decisions in Businesses
 
- (Conference) Artificial Intelligence for 
Supporting Decisions in Businesses
- (Conference) Artificial Intelligence for 
Supporting Decisions in Businesses- (Conference) Artificial Intelligence for 
Supporting Decisions in Businesses
- (Conference) Artificial Intelligence for 
Supporting Decisions in Businesses
 
(Training) Artificial Intelligence for 
Supporting Decisions in Businesses
(Training) Artificial Intelligence for 
Supporting Decisions in Businesses(Training) Artificial Intelligence for 
Supporting Decisions in Businesses
(Training) Artificial Intelligence for 
Supporting Decisions in Businesses
 
Invited Lecture at GnB Accelerator in Jakarta
Invited Lecture at GnB Accelerator in JakartaInvited Lecture at GnB Accelerator in Jakarta
Invited Lecture at GnB Accelerator in Jakarta
 
ChatCenter Aiのご紹介
ChatCenter Aiのご紹介ChatCenter Aiのご紹介
ChatCenter Aiのご紹介
 
ChatCenter Aiのご紹介
ChatCenter Aiのご紹介ChatCenter Aiのご紹介
ChatCenter Aiのご紹介
 
Introducing ChatCenter Ai
Introducing ChatCenter AiIntroducing ChatCenter Ai
Introducing ChatCenter Ai
 
Case Studies of Awesome Growth Hacks
Case Studies of Awesome Growth HacksCase Studies of Awesome Growth Hacks
Case Studies of Awesome Growth Hacks
 
NIKKEI COMPUTER’s Seminar (07.17.2014)
NIKKEI COMPUTER’s Seminar (07.17.2014)NIKKEI COMPUTER’s Seminar (07.17.2014)
NIKKEI COMPUTER’s Seminar (07.17.2014)
 
Appsocially Deck for New Context Conference 2013, San Francisco (Nov 04, 2013)
Appsocially Deck for New Context Conference 2013, San Francisco (Nov 04, 2013)Appsocially Deck for New Context Conference 2013, San Francisco (Nov 04, 2013)
Appsocially Deck for New Context Conference 2013, San Francisco (Nov 04, 2013)
 
About AppSocially
About AppSociallyAbout AppSocially
About AppSocially
 
How NOT to Suck at App Distribution - Quick Start Guide - Appsocially's Growt...
How NOT to Suck at App Distribution - Quick Start Guide - Appsocially's Growt...How NOT to Suck at App Distribution - Quick Start Guide - Appsocially's Growt...
How NOT to Suck at App Distribution - Quick Start Guide - Appsocially's Growt...
 
AppSocially: Product Milestone
AppSocially: Product MilestoneAppSocially: Product Milestone
AppSocially: Product Milestone
 
Why we are working on AppSocially
Why we are working on AppSociallyWhy we are working on AppSocially
Why we are working on AppSocially
 
AppSociail.ly Oveview
AppSociail.ly OveviewAppSociail.ly Oveview
AppSociail.ly Oveview
 
vidyard @ Y Combinator's Ad Innovation Conference
vidyard @ Y Combinator's Ad Innovation Conferencevidyard @ Y Combinator's Ad Innovation Conference
vidyard @ Y Combinator's Ad Innovation Conference
 
論文の読み方・書き方(第1回講義資料、10/3、2011年度前期c2-19、高橋雄介)
論文の読み方・書き方(第1回講義資料、10/3、2011年度前期c2-19、高橋雄介)論文の読み方・書き方(第1回講義資料、10/3、2011年度前期c2-19、高橋雄介)
論文の読み方・書き方(第1回講義資料、10/3、2011年度前期c2-19、高橋雄介)
 

Último

20220816-EthicsGrade_Scorecard-JP_Morgan_Chase-Q2-63_57.pdf
20220816-EthicsGrade_Scorecard-JP_Morgan_Chase-Q2-63_57.pdf20220816-EthicsGrade_Scorecard-JP_Morgan_Chase-Q2-63_57.pdf
20220816-EthicsGrade_Scorecard-JP_Morgan_Chase-Q2-63_57.pdfChris Skinner
 
Go for Rakhi Bazaar and Pick the Latest Bhaiya Bhabhi Rakhi.pptx
Go for Rakhi Bazaar and Pick the Latest Bhaiya Bhabhi Rakhi.pptxGo for Rakhi Bazaar and Pick the Latest Bhaiya Bhabhi Rakhi.pptx
Go for Rakhi Bazaar and Pick the Latest Bhaiya Bhabhi Rakhi.pptxRakhi Bazaar
 
Excvation Safety for safety officers reference
Excvation Safety for safety officers referenceExcvation Safety for safety officers reference
Excvation Safety for safety officers referencessuser2c065e
 
digital marketing , introduction of digital marketing
digital marketing , introduction of digital marketingdigital marketing , introduction of digital marketing
digital marketing , introduction of digital marketingrajputmeenakshi733
 
business environment micro environment macro environment.pptx
business environment micro environment macro environment.pptxbusiness environment micro environment macro environment.pptx
business environment micro environment macro environment.pptxShruti Mittal
 
Onemonitar Android Spy App Features: Explore Advanced Monitoring Capabilities
Onemonitar Android Spy App Features: Explore Advanced Monitoring CapabilitiesOnemonitar Android Spy App Features: Explore Advanced Monitoring Capabilities
Onemonitar Android Spy App Features: Explore Advanced Monitoring CapabilitiesOne Monitar
 
WSMM Media and Entertainment Feb_March_Final.pdf
WSMM Media and Entertainment Feb_March_Final.pdfWSMM Media and Entertainment Feb_March_Final.pdf
WSMM Media and Entertainment Feb_March_Final.pdfJamesConcepcion7
 
Fordham -How effective decision-making is within the IT department - Analysis...
Fordham -How effective decision-making is within the IT department - Analysis...Fordham -How effective decision-making is within the IT department - Analysis...
Fordham -How effective decision-making is within the IT department - Analysis...Peter Ward
 
Unveiling the Soundscape Music for Psychedelic Experiences
Unveiling the Soundscape Music for Psychedelic ExperiencesUnveiling the Soundscape Music for Psychedelic Experiences
Unveiling the Soundscape Music for Psychedelic ExperiencesDoe Paoro
 
GUIDELINES ON USEFUL FORMS IN FREIGHT FORWARDING (F) Danny Diep Toh MBA.pdf
GUIDELINES ON USEFUL FORMS IN FREIGHT FORWARDING (F) Danny Diep Toh MBA.pdfGUIDELINES ON USEFUL FORMS IN FREIGHT FORWARDING (F) Danny Diep Toh MBA.pdf
GUIDELINES ON USEFUL FORMS IN FREIGHT FORWARDING (F) Danny Diep Toh MBA.pdfDanny Diep To
 
EUDR Info Meeting Ethiopian coffee exporters
EUDR Info Meeting Ethiopian coffee exportersEUDR Info Meeting Ethiopian coffee exporters
EUDR Info Meeting Ethiopian coffee exportersPeter Horsten
 
Driving Business Impact for PMs with Jon Harmer
Driving Business Impact for PMs with Jon HarmerDriving Business Impact for PMs with Jon Harmer
Driving Business Impact for PMs with Jon HarmerAggregage
 
1911 Gold Corporate Presentation Apr 2024.pdf
1911 Gold Corporate Presentation Apr 2024.pdf1911 Gold Corporate Presentation Apr 2024.pdf
1911 Gold Corporate Presentation Apr 2024.pdfShaun Heinrichs
 
Entrepreneurship lessons in Philippines
Entrepreneurship lessons in  PhilippinesEntrepreneurship lessons in  Philippines
Entrepreneurship lessons in PhilippinesDavidSamuel525586
 
How Generative AI Is Transforming Your Business | Byond Growth Insights | Apr...
How Generative AI Is Transforming Your Business | Byond Growth Insights | Apr...How Generative AI Is Transforming Your Business | Byond Growth Insights | Apr...
How Generative AI Is Transforming Your Business | Byond Growth Insights | Apr...Hector Del Castillo, CPM, CPMM
 
Memorándum de Entendimiento (MoU) entre Codelco y SQM
Memorándum de Entendimiento (MoU) entre Codelco y SQMMemorándum de Entendimiento (MoU) entre Codelco y SQM
Memorándum de Entendimiento (MoU) entre Codelco y SQMVoces Mineras
 
NAB Show Exhibitor List 2024 - Exhibitors Data
NAB Show Exhibitor List 2024 - Exhibitors DataNAB Show Exhibitor List 2024 - Exhibitors Data
NAB Show Exhibitor List 2024 - Exhibitors DataExhibitors Data
 
trending-flavors-and-ingredients-in-salty-snacks-us-2024_Redacted-V2.pdf
trending-flavors-and-ingredients-in-salty-snacks-us-2024_Redacted-V2.pdftrending-flavors-and-ingredients-in-salty-snacks-us-2024_Redacted-V2.pdf
trending-flavors-and-ingredients-in-salty-snacks-us-2024_Redacted-V2.pdfMintel Group
 
Introducing the Analogic framework for business planning applications
Introducing the Analogic framework for business planning applicationsIntroducing the Analogic framework for business planning applications
Introducing the Analogic framework for business planning applicationsKnowledgeSeed
 

Último (20)

WAM Corporate Presentation April 12 2024.pdf
WAM Corporate Presentation April 12 2024.pdfWAM Corporate Presentation April 12 2024.pdf
WAM Corporate Presentation April 12 2024.pdf
 
20220816-EthicsGrade_Scorecard-JP_Morgan_Chase-Q2-63_57.pdf
20220816-EthicsGrade_Scorecard-JP_Morgan_Chase-Q2-63_57.pdf20220816-EthicsGrade_Scorecard-JP_Morgan_Chase-Q2-63_57.pdf
20220816-EthicsGrade_Scorecard-JP_Morgan_Chase-Q2-63_57.pdf
 
Go for Rakhi Bazaar and Pick the Latest Bhaiya Bhabhi Rakhi.pptx
Go for Rakhi Bazaar and Pick the Latest Bhaiya Bhabhi Rakhi.pptxGo for Rakhi Bazaar and Pick the Latest Bhaiya Bhabhi Rakhi.pptx
Go for Rakhi Bazaar and Pick the Latest Bhaiya Bhabhi Rakhi.pptx
 
Excvation Safety for safety officers reference
Excvation Safety for safety officers referenceExcvation Safety for safety officers reference
Excvation Safety for safety officers reference
 
digital marketing , introduction of digital marketing
digital marketing , introduction of digital marketingdigital marketing , introduction of digital marketing
digital marketing , introduction of digital marketing
 
business environment micro environment macro environment.pptx
business environment micro environment macro environment.pptxbusiness environment micro environment macro environment.pptx
business environment micro environment macro environment.pptx
 
Onemonitar Android Spy App Features: Explore Advanced Monitoring Capabilities
Onemonitar Android Spy App Features: Explore Advanced Monitoring CapabilitiesOnemonitar Android Spy App Features: Explore Advanced Monitoring Capabilities
Onemonitar Android Spy App Features: Explore Advanced Monitoring Capabilities
 
WSMM Media and Entertainment Feb_March_Final.pdf
WSMM Media and Entertainment Feb_March_Final.pdfWSMM Media and Entertainment Feb_March_Final.pdf
WSMM Media and Entertainment Feb_March_Final.pdf
 
Fordham -How effective decision-making is within the IT department - Analysis...
Fordham -How effective decision-making is within the IT department - Analysis...Fordham -How effective decision-making is within the IT department - Analysis...
Fordham -How effective decision-making is within the IT department - Analysis...
 
Unveiling the Soundscape Music for Psychedelic Experiences
Unveiling the Soundscape Music for Psychedelic ExperiencesUnveiling the Soundscape Music for Psychedelic Experiences
Unveiling the Soundscape Music for Psychedelic Experiences
 
GUIDELINES ON USEFUL FORMS IN FREIGHT FORWARDING (F) Danny Diep Toh MBA.pdf
GUIDELINES ON USEFUL FORMS IN FREIGHT FORWARDING (F) Danny Diep Toh MBA.pdfGUIDELINES ON USEFUL FORMS IN FREIGHT FORWARDING (F) Danny Diep Toh MBA.pdf
GUIDELINES ON USEFUL FORMS IN FREIGHT FORWARDING (F) Danny Diep Toh MBA.pdf
 
EUDR Info Meeting Ethiopian coffee exporters
EUDR Info Meeting Ethiopian coffee exportersEUDR Info Meeting Ethiopian coffee exporters
EUDR Info Meeting Ethiopian coffee exporters
 
Driving Business Impact for PMs with Jon Harmer
Driving Business Impact for PMs with Jon HarmerDriving Business Impact for PMs with Jon Harmer
Driving Business Impact for PMs with Jon Harmer
 
1911 Gold Corporate Presentation Apr 2024.pdf
1911 Gold Corporate Presentation Apr 2024.pdf1911 Gold Corporate Presentation Apr 2024.pdf
1911 Gold Corporate Presentation Apr 2024.pdf
 
Entrepreneurship lessons in Philippines
Entrepreneurship lessons in  PhilippinesEntrepreneurship lessons in  Philippines
Entrepreneurship lessons in Philippines
 
How Generative AI Is Transforming Your Business | Byond Growth Insights | Apr...
How Generative AI Is Transforming Your Business | Byond Growth Insights | Apr...How Generative AI Is Transforming Your Business | Byond Growth Insights | Apr...
How Generative AI Is Transforming Your Business | Byond Growth Insights | Apr...
 
Memorándum de Entendimiento (MoU) entre Codelco y SQM
Memorándum de Entendimiento (MoU) entre Codelco y SQMMemorándum de Entendimiento (MoU) entre Codelco y SQM
Memorándum de Entendimiento (MoU) entre Codelco y SQM
 
NAB Show Exhibitor List 2024 - Exhibitors Data
NAB Show Exhibitor List 2024 - Exhibitors DataNAB Show Exhibitor List 2024 - Exhibitors Data
NAB Show Exhibitor List 2024 - Exhibitors Data
 
trending-flavors-and-ingredients-in-salty-snacks-us-2024_Redacted-V2.pdf
trending-flavors-and-ingredients-in-salty-snacks-us-2024_Redacted-V2.pdftrending-flavors-and-ingredients-in-salty-snacks-us-2024_Redacted-V2.pdf
trending-flavors-and-ingredients-in-salty-snacks-us-2024_Redacted-V2.pdf
 
Introducing the Analogic framework for business planning applications
Introducing the Analogic framework for business planning applicationsIntroducing the Analogic framework for business planning applications
Introducing the Analogic framework for business planning applications
 

AppSocially主催GrowthCon Tokyo #1: "緊急開催!実践リーンアナリティクスワークショップ" by Alistair Croll / October 24, 2014

  • 1. Practical Lean Analytics Growthcon October 24, 2014 @acroll
  • 2. Kevin Costner is a lousy entrepreneur. Don’t sell what you can make. Make what you can sell.
  • 3. The core of Lean is iteration.
  • 4. Most startups don’t know what they’ll be when they grow up. Hotmail was a database company Flickr was going to be an MMO Twitter was a podcasting company Autodesk made desktop automation Paypal first built for Palmpilots Freshbooks was invoicing for a web design firm Wikipedia was to be written by experts only Mitel was a lawnmower company
  • 5. Waterfall, agile, and lean (Why the old ways don’t work.)
  • 6. Waterfall approach You know the problem and the solution.
  • 7.
  • 8. Known set of requirements Known ways to satisfy them Spec Build Test Launch
  • 9. Agile methodologies Know the problem, find the solution
  • 10.
  • 11. Known set of requirements Unclear how to satisfy them Problem Build Test Viable? Launch statement Sprints Adjust Unknown set of
  • 12. Lean approach First, know that you don’t know.
  • 13. Product/market hypothesis Trial startup Possible problem space Product/ market hypothesis Trial startup Product/ market hypothesis Trial startup Trial startup Product/market hypothesis You are here PIVOT
  • 14. Why now? First: High rate of change of digital technologies & channels.
  • 16. Times a song in “heavy rotation” is played daily 30 15 0 6 26 2007 2012
  • 17. Why now? Second: It’s no longer about whether you can build it—it’s about whether anyone will care.
  • 18. The Attention Economy “What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention, and a need to allocate that attention efficiently among the overabundance of information sources that might consume it.” (Computers, Communications and the Public Interest, pages 40-41, Herbert Simon Martin Greenberger, ed., The Johns Hopkins Press, 1971.)
  • 19. Lit motors tests the risky part
  • 20. Unfortunately, it is hard to be honest with ourselves.
  • 21. Everyone’s idea is the best right? People love this part! (but that’s not always a good thing) No data, no learning. This is where things fall apart.
  • 23. Analytics is the measurement of movement towards your business goals.
  • 24. In a startup, the purpose of analytics is to iterate to product/market fit before the money runs out.
  • 26. A good metric is: Understandable If you’re busy explaining the data, you won’t be busy acting on it. Comparative Comparison is context. A ratio or rate The only way to measure change and roll up the tension between two metrics (MPH) Behavior changing What will you do differently based on the results you collect?
  • 27. The simplest rule If a metric won’t change how you behave, it’s a bad metric. h"p://www.flickr.com/photos/circasassy/7858155676/
  • 28. Metrics help you know yourself. Acquisition Hybrid Loyalty You are just like 70% of retailers 20% of retailers 10% of retailers Customers that buy >1x in 90d Your customers will buy from you Once 2-2.5 per year >2.5 per year Then you are in this mode 1-15% 15-30% >30% Focus on Low acquisition cost, high checkout Increasing return rates, market share Loyalty, selection, inventory size (Thanks to Kevin Hillstrom for this.)
  • 29. Qualitative Unstructured, anecdotal, revealing, hard to aggregate, often too positive & reassuring. Warm and fuzzy. Quantitative Numbers and stats. Hard facts, less insight, easier to analyze; often sour and disappointing. Cold and hard.
  • 30. Exploratory Speculative. Tries to find unexpected or interesting insights. Source of unfair advantages. Cool. Reporting Predictable. Keeps you abreast of the normal, day-to-day operations. Can be managed by exception. Necessary.
  • 31. Rumsfeld on Analytics Things we know don’t know (Or rather, Avinash Kaushik channeling Rumsfeld) we know Are facts which may be wrong and should be checked against data. we don’t know Are questions we can answer by reporting, which we should baseline & automate. we know Are intuition which we should quantify and teach to improve effectiveness, efficiency. we don’t know Are exploration which is where unfair advantage and interesting epiphanies live.
  • 32. Slicing and dicing data Feb Mar Apr May 5,000 Active users 0 Jan Cohort: Comparison of similar groups along a timeline. (this is the April cohort) A/B test: Changing one thing (i.e. color) and measuring the result (i.e. revenue.) Multivariate analysis Changing several things at once to see which correlates with a result. ☀☁☀☁ Segment: Cross-sectional comparison of all people divided by some attribute (age, gender, etc.) ☀ ☁
  • 33. Which of these two companies is doing better?
  • 34. January February March April May Is this company Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50 growing or stagnating? Cohort 1 2 3 4 5 January $5 $3 $2 $1 $0.5 February $6 $4 $2 $1 March $7 $6 $5 April $8 $7 May $9 How about this one?
  • 35. Cohort 1 2 3 4 5 January $5 $3 $2 $1 $0.5 February $6 $4 $2 $1 March $7 $6 $5 April $8 $7 May $9 Averages $7 $5 $3 $1 $0.5 Look at the same data in cohorts
  • 36. Lagging Historical. Shows you how you’re doing; reports the news. Example: sales. Explaining the past. Leading Forward-looking. Number today that predicts tomorrow; reports the news. Example: pipeline. Predicting the future.
  • 37. Some examples A Facebook user reaching 7 friends within 10 days of signing up (Chamath Palihapitiya) If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) A Dropbox user who puts at least one file in one folder on one device (ChenLi Wang) Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) A LinkedIn user getting to X connections in Y days (Elliot Schmukler) (From the 2012 Growth Hacking conference. http://growthhackersconference.com/)
  • 38. Which means it’s time to talk about correlation.
  • 39. 10000 1000 100 10 1 Jan Feb Mar Apr May Jun Jul Aug Sept Oct Nov Dec Ice cream consumption Drownings
  • 40. Correlated Two variables that are related (but may be dependent on something else.) Ice cream & drowning. Causal An independent variable that directly impacts a dependent one. Summertime & drowning.
  • 41. A leading, causal metric is a superpower. h"p://www.flickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/
  • 42. Why is Nigerian spam so badly written?
  • 43. Experienced scammers expect a “strike rate” of 1 or 2 replies per 1,000 messages emailed; they expect to land 2 or 3 “Mugu” (fools) each week. One scammer boasted “When you get a reply it’s 70% sure you’ll get the money” “By sending an email that repels all but the most gullible,” says [Microsoft Researcher Corman] Herley, “the scammer gets the most promising marks to self-select, and tilts the true to false positive ratio in his favor.” This would be horribly inefficient since humans are involved. Good language (10% conversion) Not-gullible (.07% conversion) Aunshul Rege of Rutgers University, USA in 2009 1000 emails Bad language (0.1% conversion) 1-2 responses Gullible (70% conversion) 1 fool and their money, parted. 1000 emails 100 responses 1 fool and their money, parted.
  • 44. Turns out the word “Nigeria” is the best way to identify promising prospects.
  • 45. Nigerian spammers really understand their target market. They see past vanity metrics.
  • 46. The Lean Analytics framework.
  • 47. Sustainable growth comes through the actions of your customers. - Eric Ries
  • 48. Eric’s three engines of growth Virality Make people invite friends. How many they tell, how fast they tell them. Price Spend money to get customers. Customers are worth more than they cost. Stickiness Keep people coming back. Approach Get customers faster than you lose them. Math that matters
  • 49. Dave’s Pirate Metrics AARRR Acquisition How do your users become aware of you? SEO, SEM, widgets, email, PR, campaigns, blogs ... Activation Do drive-by visitors subscribe, use, etc? Features, design, tone, compensation, affirmation ... Retention Does a one-time user become engaged? Notifications, alerts, reminders, emails, updates... Revenue Do you make money from user activity? Transactions, clicks, subscriptions, DLC, analytics... Referral Do users promote your product? Email, widgets, campaigns, likes, RTs, affiliates...
  • 50. Gate Stage EMPATHY I’ve found a real, poorly-met need that a reachable market faces. STICKINESS I’ve figured out how to solve the problem in a way they will keep using and pay for. VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives. REVENUE The users and features fuel growth organically and artificially. SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem. The five stages
  • 51. Empathy stage: Localmind hacks Twitter Needed to find out if a core assumption—strangers answering questions—was valid. Ran Twitter experiment instead of writing code Asked senders of geolocated Tweets from Times Square random questions; counted response rate Conclusion: high enough to proceed
  • 52. LikeBright’s mechanical turk Used Mechanical Turk, Google Voice to speak w/ 100 single women; paid $2. The interviews lasted typically around 10-15 minutes. Simple interview script with open-ended questions, since he was digging into the problem validation stage of his startup. Founder Nick Soman: “I was amazed at the feedback I got. We were able to speak with one hundred single women that met our criteria in four hours on one evening.” Went back to TechStars and got accepted. LikeBright’s website is now live with a 50% female user base, and recently raised a round of funding. “Since that first foray into interviewing customers, I’ve probably spoken with over a thousand people through Mechanical Turk,”
  • 53. How to avoid leading the witness Avoid biased wording, preconceptions, or a giveaway appearance. Word your surveys carefully to be neutral. Get them to purchase. Ask them to pay. Demand real Ask “why” several times. Leave lingering, uncomfortable pauses Don’t tip your hand Make the question real Keep digging Look for other clues Have a colleague make notes of when they react, or of their body
  • 54. Stickiness stage: qidiq streamlines invites Survey owner adds recipient to group Survey owner asks question Recipient reads survey question Recipient responds to question Recipient sees survey results (Later, if needed…) Recipient visits site; no password! Recipient does password recovery One-time link sent to email Recipient creates password Recipient can edit profile, etc. Survey owner adds recipient to group Survey owner asks question Recipient gets invite Recipient installs mobile app Recipient creates account, profile Recipient can edit profile, etc. Recipient reads survey question Recipient responds to question Recipient sees survey results 10-25% RESPONSE RATE 70-90% RESPONSE RATE
  • 55. 1200 1000 800 600 400 200 0 January February 1 2 3 4 5 6 7 8 9 Days since last engagement 25000 20000 15000 10000 5000 0 Disengaged (>10 days) Number of users A better approach to engagement This is a good thing.
  • 56. Who is worth more? A Lifetime: B Lifetime: Today $200 $200 Roberto Medri, Etsy Visits
  • 57. Virality stage: Timehop focuses on content sharing Focused on percent of daily active users that share their content Aiming for 20-30% of DAU sharing “All that matters now is virality. Everything else—be it press, publicity stunts or something else—is like pushing a rock up a mountain: it will never scale. But being viral will.” - Jonathan Wegener, co-founder
  • 58. ------------------------------------------------------ Get your free private email at http://www.hotmail.com ------------------------------------------------------
  • 59.
  • 60. v ≠ 1, pt = δp0 (1 – vt+1) / (1 – v) + p0 http://robert.zubek.net/blog/2008/01/30/viral-coefficient-calculation/ Viral coefficient
  • 61. Or simpler x - > 1 Users Viral coefficient Churn & abandonment
  • 62. Revenue stage: Backupify’s Customer Acquisition Payback Initially focused on site visitors Then focused on trials Then switched to signups Today, MRR In early 2010, CAC was $243 and ARPU was only $39 Pivoted to target business users CLV-to-CAC today is 5-6x Now they track Customer Acquisition Payback Target is less than 12 months
  • 63. Scale stage: Incremental order cost Marginal costs Fixed costs
  • 64. Six business model archetypes. E-commerce SaaS Mobile Media app User-gen content 2-sided market The business you’re in
  • 65. (Which means eye charts like these.) Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Freemium churn Cancel Engaged User Free user disengagement Reactivate Trial abandonment Cancel rate Invite Others Upselling rate Upselling Paying Customer Reactivation rate Paid conversion FORMER USERS User Lifetime Value Reactivate Capacity Limit Support data FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Trial Over Disengaged Dissatisfied
  • 66. Acquisition Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Freemium churn Cancel Engaged User Free user disengagement Reactivate Trial abandonment Cancel rate Invite Others Upselling rate Upselling Paying Customer Reactivation rate Paid conversion FORMER USERS User Lifetime Value Reactivate Capacity Limit Support data FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Trial Over Disengaged Dissatisfied
  • 67. Activation Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Freemium churn Cancel Engaged User Free user disengagement Reactivate Trial abandonment Cancel rate Invite Others Upselling rate Upselling Paying Customer Reactivation rate Paid conversion FORMER USERS User Lifetime Value Reactivate Capacity Limit Support data FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Trial Over Disengaged Dissatisfied
  • 68. Retention Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Freemium churn Cancel Engaged User Free user disengagement Reactivate Trial abandonment Cancel rate Invite Others Upselling rate Upselling Paying Customer Reactivation rate Paid conversion FORMER USERS User Lifetime Value Reactivate Capacity Limit Support data FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Trial Over Disengaged Dissatisfied
  • 69. Revenue Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Freemium churn Cancel Engaged User Free user disengagement Reactivate Trial abandonment Cancel rate Invite Others Upselling rate Upselling Paying Customer Reactivation rate Paid conversion FORMER USERS User Lifetime Value Reactivate Capacity Limit Support data FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Trial Over Disengaged Dissatisfied
  • 70. Revenue Customer Acquisition Cost paid direct search wom inherent virality VISITOR Freemium/trial offer Enrollment User Disengaged User Freemium churn Cancel Engaged User Free user disengagement Reactivate Trial abandonment Cancel rate Invite Others Upselling rate Upselling Paying Customer Reactivation rate Paid conversion FORMER USERS User Lifetime Value Reactivate Capacity Limit Support data FORMER CUSTOMERS Customer Lifetime Value Viral coefficient Viral rate Resolution Account Cancelled Billing Info Exp. Paid Churn Rate Tiering Trial Over Disengaged Dissatisfied
  • 71. Model + Stage = One Metric That Matters. The business you’re in E-Com SaaS Mobile 2-Sided Media UCG One Metric That Matters. Empathy Stickiness Virality Revenue Scale The stage you’re at
  • 74. In a startup, focus is hard to achieve.
  • 75. Having only one metric addresses this problem.
  • 77. Moz cuts down on metrics SaaS-based SEO toolkit in the scale stage. Focused on net adds. Was a marketing campaign successful? Were customer complaints lowered? Was a product upgrade valuable? Net adds up: Can we acquire more valuable customers? What product features can increase engagement? Can we improve customer support? Net adds flat: Are the new customers not the right segment? Did a marketing campaign fail? Did a product upgrade fail somehow? Is customer support falling apart? Net adds down:
  • 78. Metrics are like squeeze toys. http://www.flickr.com/photos/connortarter/4791605202/
  • 79. Empathy Stickiness Virality Revenue Scale E-commerce Mobile app User-gen content SaaS Media 2-sided market Interviews; qualitative results; quantitative scoring; surveys Loyalty, conversion CAC, shares, reactivation (Money from transactions) Transaction, CLV Affiliates, white-label Engagement, churn Inherent virality, CAC (Money from active users) Upselling, CAC, CLV API, magic #, mktplace Content, spam Invites, sharing (Money from ad clicks) Ads, donations Analytics, user data Inventory, listings SEM, sharing Transactions, commission Other verticals Downloads, churn, virality WoM, app ratings, CAC CLV, ARPDAU Spinoffs, publishers Traffic, visits, returns Content virality, SEM CPE, affiliate %, eyeballs Syndication, licenses
  • 81. Drawing some lines in the sand.
  • 82. A company loses a quarter of its customers every year. Is this good or bad?
  • 83. Baseline: 2-5% monthly churn • The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s on the job. • Get below a 5% monthly churn rate before you know you’ve got a business that’s ready to grow (Mark MacLeod) and around 2% before you really step on the gas (David Skok) • Last-ditch appeals and reactivation can have a big impact. Facebook’s “don’t leave” reduces attrition by 7%.
  • 84. Not knowing what normal is makes you do unwise things.
  • 85. Baseline: 5-7% growth a week “A good growth rate during YC is 5-7% a week,” he says. “If you can hit 10% a week you're doing exceptionally well. If you can only manage 1%, it's a sign you haven't yet figured out what you're doing.” At revenue stage, measure growth in revenue. Before that, measure growth in active users. Paul Graham, Y Combinator • Are there enough people who really care enough to sustain a 5% growth rate? • Don’t strive for a 5% growth at the expense of really understanding your customers and building a meaningful solution • Once you’re a pre-revenue startup at or near product/market fit, you should have 5% growth of active users each week • Once you’re generating revenues, they should grow at 5% a week
  • 86. Baseline: 10% visitor engagement/day 30% of users/month use web or mobile app 10% of users/day use web or mobile app 1%of users/day use it concurrently Fred Wilson’s social ratios
  • 87. Baseline: Calculating customer lifetime 25% 5% monthly churn monthly churn 100/25=4 100/5=20 The average The average customer lasts customer lasts 4 months 20 months 2% monthly churn 100/2=50 The average customer lasts 50 months
  • 88. Baseline: CAC under 1/3 of CLV • CLV is wrong. CAC Is probably wrong, too. • Time kills all plans: It’ll take a long time to find out whether your churn and revenue projections are right • Cashflow: You’re basically “loaning” the customer money between acquisition and CLV. • It keeps you honest: Limiting yourself to a CAC of only a third of your CLV will forces you to verify costs sooner. Lifetime of 20 mo. $30/mo. per customer $600 CLV 1/3 spend $200 CAC Now segment those users!
  • 89. Etsy • Online store for creative types, founded 2005 • $525M Gross Merchandise Sales in 2011, with 19,000,000 members and 800,000 active shops offering 15,000,000 items for sale • 1.4B pageviews per month ~2M iPhone app downloads • Thin revenues: Etsy makes only $0.20 or 3.5% margin • Heavy focus on Customer Lifetime Value (buyer and seller) • Actually residual lifetime value; they take this pretty seriously.
  • 90. Etsy • The best customers to target are • Recent high-profile customers • Old-time best customers about to churn or just churned • Tiered campaigns • Bronze/silver customers: reinforcement, nudges • Gold customers: premium services • Platinum customers: recognition • What they watch: • Growth of individual product categories • Time to first sale by a user • Average order value • Percentage of visits that convert to a sale • Percentage of return buyers • Distinct sellers within a product category • Time-to-first-sale and average order value by product category Roberto Medri, Etsy
  • 92. Pick a KPI Draw a line Draw a new line Pivot or give up Try again Success! Did we move the needle? Measure the results Design a test Make changes in production Find a potential improvement With data: find a commonality Without data: make a good guess Hypothesis
  • 93. Do AirBnB hosts get more business if their property is professionally photographed?
  • 94. Gut instinct (hypothesis) Professional photography helps AirBnB’s business Candidate solution (MVP) 20 field photographers posing as employees Measure the results Compare photographed listings to a control group Make a decision Launch photography as a new feature for all hosts
  • 95. 5,000 shoots per month by February 2012
  • 96. Hang on a second.
  • 97. REALLY? Gut instinct (hypothesis) Professional photography helps AirBnB’s business
  • 98. Pick a KPI Draw a line Draw a new line Pivot or give up Try again Success! Did we move the needle? Measure the results Design a test Make changes in production Find a potential improvement With data: find a commonality Without data: make a good guess Hypothesis
  • 99. “Gee, those houses that do well look really nice.” Maybe it’s the camera. With data: find a commonality “Computer: What do all the highly rented houses have in common?” Camera model. Without data: make a good guess
  • 100. Circle of Moms: Not enough engagement • Too few people were actually using the product • Less than 20% of any circles had any activity after their initial creation • A few million monthly uniques from 10M registered users, but no sustained traction • They found moms were far more engaged • Their messages to one another were on average 50% longer • They were 115% more likely to attach a picture to a post they wrote • They were 110% more likely to engage in a threaded (i.e. deep) conversation • Circle owners’ friends were 50% more likely to engage with the circle • They were 75% more likely to click on Facebook notifications • They were 180% more likely to click on Facebook news feed items • They were 60% more likely to accept invitations to the app • Pivoted to the new market, including a name change • By late 2009, 4.5M users and strong engagement • Sold to Sugar, inc. in early 2012
  • 101. Landing page design A/B testing Cohort analysis General analytics URL shortening Funnel analytics Influencer Marketing Publisher analytics SaaS analytics Gaming analytics User analytics Spying on users User interaction Customer User segmentation satisfaction KPI dashboards
  • 102. Growth hacking (is a word you should hate but will hear a lot about.)
  • 103. Growth hacking, demystified. Find correlation Test causality Optimize the causal factor Pick a metric to change
  • 106. Guerrilla marketing Data-driven learning GROWTH HACKING Subversiveness
  • 108.
  • 109.
  • 110.
  • 114. Twitter’s 140-character limit isn’t arbitrary. It’s constrained by the size http://i.i.cbsi.com/cnwk.1d/i/tim/2011/11/18/ sms_screen_twitter_activity_stream_270x405.png
  • 117. 1. Define your business model
  • 118. 2. Draw a system diagram
  • 119. 3. Decide what stage you’re at
  • 120. 4. Identify the One Metric That Matters (usually the one that is most broken)
  • 121. 5. Use the cycle to experiment until you’ve achieved the desired result.
  • 122. 6. Set up monitoring for this metric in case it breaks, and choose a new OMTM
  • 124. “The most important figures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them.” Lloyd S. Nelson
  • 125. Pic by Twodolla on Flickr. http://www.flickr.com/photos/twodolla/3168857844
  • 126. ARCHIMEDES HAD TAKEN BATHS BEFORE.
  • 127. Once, a leader convinced others in the absence of data.
  • 128. Now, a leader knows what questions to ask.
  • 129. Ben Yoskovitz byosko@gmail.com @byosko Alistair Croll acroll@gmail.com @acroll
  • 130.
  • 131.
  • 132. The mobile app! customer lifecycle! Ratings Reviews Search Leaderboards Purchases App store! App sales Downloads Installs Play Disengagement Reactivation Uninstallation Disengagement Account" creation Virality Downloads," Gross revenue ARPU Activation Churn, CLV In-app" purchases Legitimate Incentivized Fraudulent Ratings!
  • 133.
  • 134.
  • 135.
  • 137. Build a message map. 1. Understand the stages a buyer goes through 2. Create benefits; mitigate objections 3. Target the message to the stage the audience is at
  • 138. Everyone in the world A. I need a car I should buy B. a car It should be C. a hybrid I should buy D. a Honda Civic
  • 139. Everyone in the world A. I need a car People who want to drive I should buy B. a car Prospective car buyers It should be C. a hybrid People looking for a hybrid I should buy D. a Honda Civic Honda Civic Hybrid owners
  • 140. “Isn’t it time you got out of the city?” campaign showing how cars make nature accessible & ridiculing urban hipsters. Ads showing how cars are needed any time (pregnancy, errands, urgent business) and how a car is a “personal assistant.” Urgency (“every time you drive a non-hybrid car you kill the planet a little”) and testimonials from buyers who’ve saved money. Honda branding ads and model-specific promotions. Follow-up satisfaction campaign to encourage buyers to tell their friends Everyone in the world A. I need a car People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” I should buy B. a car Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” It should be C. a hybrid People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” I should buy D. a Honda Civic Honda Civic Hybrid owners
  • 141. Everyone in the world People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” Honda Civic Hybrid owners Those who don’t need cars • I’m too young to drive • I’m too old to drive • I can walk or take public transit Car users who won’t buy • It’s too expensive for me • I will use a shared car service • It’ll get stolen Those who won’t buy hybrids • Hybrids are gutless • Batteries are toxic & explosive • In the end it costs more than it saves I will buy another brand • I buy domestic • I’ve always driven a VW • Toyotas are reliable • I want something prestigious A. I need a car I should buy B. a car It should be C. a hybrid I should buy D. a Honda Civic
  • 142. Everyone in the world People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” Honda Civic Hybrid owners Those who don’t need cars • I’m too young to drive • I’m too old to drive • I can walk or take public transit Car users who won’t buy • It’s too expensive for me • I will use a shared car service • It’ll get stolen Those who won’t buy hybrids • Hybrids are gutless • Batteries are toxic & explosive • In the end it costs more than it saves I will buy another brand • I buy domestic • I’ve always driven a VW • Toyotas are reliable • I want something prestigious Sponsor a driving school “Give the gift of driving” campaign for grandparents. PR on dangers of commuting, pedestrian deaths Financing, cashback Sell to carshares; underscore their limitations Theft warranty, tracking services, high-end locks Independent tests, standard metrics (0-60 in X) Lab research, studies ROI calculator; replacement programs Prove Honda hires US workers “Time to leave Germany” ads Spontaneous accel. stories Premium brand (Acura) A. I need a car I should buy B. a car It should be C. a hybrid I should buy D. a Honda Civic
  • 143. “Isn’t it time you got out of the city?” campaign showing how cars make nature accessible & ridiculing urban hipsters. Ads showing how cars are needed any time (pregnancy, errands, urgent business) and how a car is a “personal assistant.” Urgency (“every time you drive a non-hybrid car you kill the planet a little”) and testimonials from buyers who’ve saved money. Honda branding ads and model-specific promotions. Follow-up satisfaction campaign to encourage buyers to tell their friends Everyone in the world People who want to drive “I need a vehicle to get around, be productive, and enjoy my life.” Prospective car buyers “I want to own a car because it’s convenient; it’s a personal relationship; I don’t trust others.” People looking for a hybrid “I want to save money and fuel. I also care about the environment and want to be seen as ‘green’.” Honda Civic Hybrid owners Those who don’t need cars • I’m too young to drive • I’m too old to drive • I can walk or take public transit Car users who won’t buy • It’s too expensive for me • I will use a shared car service • It’ll get stolen Those who won’t buy hybrids • Hybrids are gutless • Batteries are toxic & explosive • In the end it costs more than it saves I will buy another brand • I buy domestic • I’ve always driven a VW • Toyotas are reliable • I want something prestigious Sponsor a driving school “Give the gift of driving” campaign for grandparents. PR on dangers of commuting, pedestrian deaths Financing, cashback Sell to carshares; underscore their limitations Theft warranty, tracking services, high-end locks Independent tests, standard metrics (0-60 in X) Lab research, studies ROI calculator; replacement programs Prove Honda hires US workers “Time to leave Germany” ads Spontaneous accel. stories Premium brand (Acura) A. I need a car I should buy B. a car It should be C. a hybrid I should buy D. a Honda Civic