Product management is about creating change. Metrics are the guideposts that help us ensure that the changes we make are leading to the results we want. They let us forecast, provide early warning signals, and create incentives for action. In this talk Meghan Cochran talked about designing good metrics, gaining alignment among a broad range of stakeholders, and communicating progress effectively. She discussed the trade-offs of looking at rates & ratios vs absolute numbers, and talked about funnels, cohorts, and other fascinating and exciting measures of success.

1. Measure Your Way to Success by Sephora’s
former Director of Product
www.productschool.com

2. FREE INVITE
Join 12,000+ Product Managers on
productschool.com/slack-community

3. Product
Management
2-month part-time Courses

4. Coding
for Managers
2-month part-time Courses

5. Data Analytics
for Managers
2-month part-time Courses

6. Include @productschool and #prodmgmt at the end of your
tweet
Tweet to get a free ticket
for our next Event!

7. Meghan Cochran
Tonight’s Speaker

9. Product management is the discipline of
making changes likely to lead to success.

10. We measure
to understand
to guide
to inspire
to prove
to predict
to succeed

11. What we measure, and how we measure it,
will change our focus & our outcomes.
These days, we can measure almost
anything. Which metrics matter?

12. profit
margin
engagement
revenue
per user
Loyalty
Conversion Acquisition
USE lift unique visits
Shift from paid to organic
response time
lower churn
ACTIVE
USERS
Lifetime Value

14. Let’s take a brief moment to talk about
correlation & causality...

15. We have two variables, A & B.
Correlation
A & B move together historically
Causality
Change in A causes a change in B
Sometimes, it can be hard to tell the difference...

16. An online retail store has seen a seasonal dip in
January sales for the last three years.
The team decides to run a site-wide sale &
email campaign Jan 15-18, investing heavily in
compelling creative and messaging.
It’s a disaster: Email click rates are flat & sales
dip even lower than historical seasonality.
The team is surprised and disappointed.
They scrap the creative direction.

18. Purchases x% y%

19. First, we need to
understand our
FUNNEL
series of
repeatable
actions leading to
observed action
Success = purchases
Click
#
%
%
%
Customers
Visit the site
Engage with site
Make purchase

20. Purchases x% y%
Clicks
Site visits
Engagement a% b%

21. There are two
levers in a
FUNNEL
(1) Increase volume
(2) Improve conversion rates
Click
Visit the site
Engage with site
Make purchase
#
%
%
%

22. SEGMENTATION
provides insight
into how to move
those levers
Examine the data for variations by
- Source
- Demographics
- Date & time
- Content (ads, landing pages, etc)
Click on an ad
%
#

24. Communicate
outcomes
CONSISTENTLY
(1) We increased volume
Click
(2) We improved completion rates
# ↑ x%
% ↑ x%
% ↑ x%
% ↑ x%
Purchases # ↑ x%, with Revenue ↑ x%
Visit the site
Engage with site
Make purchase

26. COHORTS
Show variations
between groups
at the same stage
in the customer
lifecycle over time
Purchases made by new customers
Nov Dec Jan
# New 250,000 400,000 225,000
Click rate 10% 17% 11%
Engage rate 35% 64% 37%
Purchase rate 3% 5% 3%
# Purchases 263 1,920 275

27. LOCAL
metrics
Important, but not
actionable
without further
research
● User satisfaction
● Likelihood to recommend
● Search quality
● Page-level engagement
● Cost of acquisition

28. VALUES
v.
RATIOS
Values
● Specific volumes or amounts
● Show growth in absolute
terms
Ratios & percentages
● Relationships
● Comparisons over time

29. Search
QUALITY
For purchases with
source = search
Number of sales from top 5
Percent of sales from top 5

30. setting
GOALS
S pecific
M easurable
A chievable
R ealistic
T imebound
+M eaningful

31. Concerned about acquisition costs, the team
made a Q1 goal to double organic traffic.
The new content strategy worked.
Organic traffic & purchases doubled.
Cost of acquisition remained flat.
ORGANIC monthly Before After
Organic as % traffic 3.6% 4.3%
# Organic unique users 9,026 17,818
Conversion rate 0.13% 0.12%
Organic purchases 12 21

33. Product management is about creating change.
Metrics are the guideposts that help us ensure that
the changes we make are leading to the results we
want. They let us forecast, provide early warning
signals, and create incentives for action.
This talk is about designing good metrics, gaining
alignment among a broad range of stakeholders,
and communicating progress effectively. We'll
discuss the trade-offs of looking at rates & ratios vs
absolute numbers, and we'll talk about funnels,
cohorts, and other fascinating and exciting
measures of success.
Quotation attributed to W. Edwards Deming
Meghan Cochran
meghanpcochran@gmail.com
Product School - 10/4/17

34. Part-time Product Management Courses in
San Francisco, Silicon Valley, Los Angeles
and New York
www.productschool.com