1. Learn-Measure-Build
Craft Useful Experiments By
Running Backwards
Janice Fraser
December 2013
Monday, December 16, 13
1

2. © LUXR.CO 2013
Monday, December 16, 13
2

3. Luxr.co
Radically helpful workshops for
startup founders
First month free...use discount code:
TEHCARD
© LUXR.CO 2013
Monday, December 16, 13
3

4. TWEET!
@clevergirl
@luxrco
#LeanStartup
© LUXR.CO 2013
Monday, December 16, 13
4

5. • Learn the difference between an experiment and a solution test.
• Discover the seven steps to crafting an experiment with a clear
result.
• Practice identifying assumptions and defining testable
hypotheses.
• Understand the anatomy of an experiment.
• Design an experiment to test a real-life hypothesis.
• "Get out of the classroom" to run a real-world experiment on
your fellow conference-goers.
• Define a result and make a go-forward decision.
• Refine the experiment based on cost, time, and manpower.
• Identify the four characteristics of successful experiments.
© LUXR.CO 2013
Monday, December 16, 13
5

6. © 2013 LUXr Incorporated, www.luxr.co
Monday, December 16, 13
6

7. June & Pete’s
Wedding Cake
© 2013 LUXr Incorporated, www.luxr.co
Monday, December 16, 13
6

8. http://saptstrength.com/
© 2013 LUXr Incorporated, www.luxr.co
Monday, December 16, 13
7

9. June
Pete
http://saptstrength.com/
© 2013 LUXr Incorporated, www.luxr.co
Monday, December 16, 13
7

10. http://saptstrength.com/
© 2013 LUXr Incorporated, www.luxr.co
Monday, December 16, 13
8

11. April
May
June
July
August
http://saptstrength.com/
© 2013 LUXr Incorporated, www.luxr.co
Monday, December 16, 13
8

12. sedonaweddingcakes.com
© 2013 LUXr Incorporated, www.luxr.co
Monday, December 16, 13
9

13. freebestpictures.blogspot.com
© 2013 LUXr Incorporated, www.luxr.co
Monday, December 16, 13
10

14. thekitchn.com
© 2013 LUXr Incorporated, www.luxr.co
Monday, December 16, 13
11

15. Zoe Clark, The Cake Parlour
© 2013 LUXr Incorporated, www.luxr.co
Monday, December 16, 13
12

16. Zoe Clark, The Cake Parlour
© 2013 LUXr Incorporated, www.luxr.co
Monday, December 16, 13
12

17. sedonaweddingcakes.com
© 2013 LUXr Incorporated, www.luxr.co
Monday, December 16, 13
13

18. sedonaweddingcakes.com
Zoe Clark, The Cake Parlour
© 2013 LUXr Incorporated, www.luxr.co
Monday, December 16, 13
13

19. LEAN
Startup
© 2013 LUXr Incorporated, www.luxr.co
Monday, December 16, 13
14

20. LEAN
Startup
© LUXR.CO 2013
Monday, December 16, 13
15

21. Build
measure
learn!
© LUXR.CO 2013
Monday, December 16, 13
16

22. © LUXR.CO 2013
Monday, December 16, 13
17

23. © LUXR.CO 2013
Monday, December 16, 13
17

24. Lean Startup is NOT
© LUXR.CO 2013
Monday, December 16, 13
18

25. Lean Startup is NOT
Cheap Startup
© LUXR.CO 2013
Monday, December 16, 13
18

26. Lean Startup is NOT
Cheap Startup
Fast Startup
© LUXR.CO 2013
Monday, December 16, 13
18

27. Lean Startup is NOT
Cheap Startup
Fast Startup
Shortcut Startup
© LUXR.CO 2013
Monday, December 16, 13
18

28. Lean Startup is...
© LUXR.CO 2013
Monday, December 16, 13
19

29. Lean Startup is...
An approach for building companies that are
creating new products and services in
situations of extreme uncertainty.
© LUXR.CO 2013
Monday, December 16, 13
19

30. Lean Startup is...
An approach for building companies that are
creating new products and services in
situations of extreme uncertainty.
The approach advocates creating small
products that test the entrepreneur’s
assumptions, and using customer feedback to
evolve the product, thereby reducing waste.
© LUXR.CO 2013
Monday, December 16, 13
19

31. 1.
2.
3.
4.
List your assumptions.
Understand your customers.
Experiment efficiently.
Adjust direction based on evidence.
© LUXR.CO 2013
Monday, December 16, 13
20

32. 1.
2.
3.
4.
List your assumptions.
Understand your customers.
Experiment efficiently.
Adjust direction based on evidence.
© LUXR.CO 2013
Monday, December 16, 13
21

33. Go backward to go forward.
Learn
Build
Measure
© LUXR.CO 2013
Monday, December 16, 13
22

34. Go backward to go forward.
Learn
Build
Measure
© LUXR.CO 2013
Monday, December 16, 13
22

35. Go backward to go forward.
Learn
Build
Measure
© LUXR.CO 2013
Monday, December 16, 13
22

36. Go backward to go forward.
Learn
Build
Measure
© LUXR.CO 2013
Monday, December 16, 13
22

37. Go backward to go forward.
Learn
Build
Measure
© LUXR.CO 2013
Monday, December 16, 13
22

38. Think of it like this...
© LUXR.CO 2013
Monday, December 16, 13
23

39. Think of it like this...
TDD
© LUXR.CO 2013
Monday, December 16, 13
23

40. Think of it like this...
TDD
A Lean Startup is a test-driven
COMPANY
© LUXR.CO 2013
Monday, December 16, 13
23

41. Write the test first
© LUXR.CO 2013
Monday, December 16, 13
24

42. RISK = UNVALIDATED EFFORT
Plot the difference
MAKE
MAKE
MAKE
release
release
release
THINK
TIME
© LUXR.CO 2013
Monday, December 16, 13
25

43. RISK = UNVALIDATED EFFORT
Lots of little wiggles
TIME
© LUXR.CO 2013
Monday, December 16, 13
26

44. RISK = UNVALIDATED EFFORT
Lots of little wiggles
TIME
© LUXR.CO 2013
Monday, December 16, 13
26

45. RISK = UNVALIDATED EFFORT
Each wiggle is a learning cycle.
MEASURE
MAKE
BUILD
LEARN
release
BUILD
TIME
© LUXR.CO 2013
Monday, December 16, 13
27

46. There’s a pattern to
startup work:
ideation
decision
making
ideation etc.
© LUXR.CO 2013
Monday, December 16, 13
28

47. In foolproof
experiments, the
results should be selfevident.
© LUXR.CO 2013
Monday, December 16, 13
29

48. 7 steps to foolproof experiments
1.
Choose the problem to tackle. Clarify what’s
uncertain, what you need to learn.
2.
Understand the possible answers.
3.
Brainstorm “what you could do to prove it”.
4.
Choose the most broadly differentiated set of
indicators.
5.
Run the experiment.
6.
Write down the results...on a sheet of paper.
7.
Project the results into your future.
© LUXR.CO 2013
Monday, December 16, 13
30

49. The Workshop
© LUXR.CO 2013
Monday, December 16, 13
31

50. Box Me Up
Provides you with plastic storage boxes
Tranports your stuff to storage
Pay to get your stuff back
Per-box per year
1 round trip per transaction
Is it profitable?
Storage facility “outside the city” that they have a long-term lease
They keep your stuff for a year
Fully secured; all sorts of insurance; 24-hr armed security.
3 sizes of box
Target customer is Young Families, 2 incomes, urban, 2 kids, owns home
Full “Bill of Lading” Which you fill out online.
Max insurance is $500/box.
Next-day pickup and delivery
COGS: $12/box, avg., $6/box per storage,
Assumption 40% capacity yr 2.
Within 15 of the city
If no-pay, stuff gets auctioned after 90 days of no-pay.
Not a Moving company.
Drop-off and pickup are same location.
Nothing prevents business use
RIght now margin is negative, because we’re not at scale.
No union issues.
Breaks even in year 3. No special handling boxes.
Raised $5m series A from Sand Hill Road VCs
150 customers in SF and 42 in NY.
Size = boxbee size
Committed to pay $28/box/yr.
Cost of customer acquisition?
Nobody is paying up front.
© LUXR.CO 2013
Monday, December 16, 13
32

51. How Products Work
© LUXR.CO
2013
Monday, December 16, 13
33

52. How Products Work
Users
© LUXR.CO
2013
Monday, December 16, 13
33

53. How Products Work
Users
Needs
© LUXR.CO
1. I need...
2. I want...
3. My goal is...
2013
Monday, December 16, 13
33

54. How Products Work
Users
Needs
Uses
© LUXR.CO
1. I need...
2. I want...
3. My goal is...
Mary can...
2013
Monday, December 16, 13
33

55. How Products Work
Users
Needs
Uses
1. I need...
2. I want...
3. My goal is...
Mary can...
Features
© LUXR.CO
2013
Monday, December 16, 13
33

56. How Products Work
Users
Needs
Uses
1. I need...
2. I want...
3. My goal is...
Mary can...
Features
Prototypes
User Stories
Themed Releases
© LUXR.CO
This Week
2013
Monday, December 16, 13
33

57. How Products Work
Users
why
what
how
Needs
1. I need...
2. I want...
3. My goal is...
Business thinking goes here
Uses
Mary can...
Features
Prototypes
User Stories
Themed Releases
© LUXR.CO
This Week
2013
Monday, December 16, 13
33

58. How Products Work
Users
why
what
how
Needs
1. I need...
2. I want...
3. My goal is...
Business thinking goes here
Uses
Mary can...
Features
Sketches, wireframes, pixels
Prototypes
User Stories
Themed Releases
© LUXR.CO
This Week
2013
Monday, December 16, 13
33

59. How Products Work
Users
why
what
how
Needs
1. I need...
2. I want...
3. My goal is...
Business thinking goes here
Uses
Mary can...
Features
Sketches, wireframes, pixels
Prototypes
User Stories
Themed Releases
© LUXR.CO
This Week
2013
Monday, December 16, 13
33

60. How Products Work
Users
why
what
how
Needs
1. I need...
2. I want...
3. My goal is...
Business thinking goes here
Uses
Mary can...
Features
Sketches, wireframes, pixels
Prototypes
User Stories
Themed Releases
© LUXR.CO
This Week
2013
Monday, December 16, 13
33

61. How Products Work
THINK
MAKE
CHECK
Users
why
what
how
Needs
1. I need...
2. I want...
3. My goal is...
Business thinking goes here
Uses
Mary can...
BUILD
MEASURE
LEARN
Features
Sketches, wireframes, pixels
Prototypes
User Stories
Themed Releases
© LUXR.CO
This Week
2013
Monday, December 16, 13
33

62. 4 Characteristics of
Successful Experiments
1. High-stakes learning goal.
2. Extremely clear, focused hypothesis with
controlled variables.
3. Broadly differentiated indicators
covering all possible outcomes.
4. Right size and scale.
© LUXR.CO 2013
Monday, December 16, 13
34

63. Thank You!
@clevergirl
@luxrco
#LeanStartup
© LUXR.CO 2013
Monday, December 16, 13
35