Lean Analytics for Intrapreneurs (Lean Startup Conf 2013)

1. Lean Analytics for Intrapreneurs Lean Startup Conference 2013 @acroll

2. When you’re a startup your goal is to ﬁnd a sustainable, repeatable business model. When you’re a big company your goal is to perpetuate one.

3. Lean Analytics: Use data to build a better business faster.

4. Intrapreneur: Someone working to produce disruptive change in an organization that has already found a sustainable, repeatable business model.

5. (Before we get into Lean Analytics, 2 key lessons.) Lesson one: Companies die because they fail to move to new business models.

6. $1 8” 5.25” 3.5” Clay Christensen, The Innovator’s Dilemma Time eb oo k $1000 N ot op kt De s ic in M $100 ai nf ra om m e pu te r $10 M Cost per MB 14”

7. Technologies outstrip what the market needs, driven by feedback from the “best” current customer. $1 8” 5.25” $10 end High er stom cu end Low mer usto c $100 $1000 Clay Christensen, The Innovator’s Dilemma Time

8. The new market has different criteria for success, which are uninteresting to incumbents. $1 $10 $100 Storage capacity Portability $1000 Clay Christensen, The Innovator’s Dilemma Time

9. Sometimes this has unintended consequences $1 Smaller disc size means less vibration impact, leading to greater density, increasing storage capacity $10 $100 $1000 Clay Christensen, The Innovator’s Dilemma Time

10. Three kinds of innovation Improve along current metrics... Switch to a new value model Change the business model entirely ...or alter the rate of improvement Sustain/core Innovate/adjacent Disrupt/transformative (optimizing for more of the same) (introduce nearby product, market, or method) (Fundamentally changing the business model)

11. Lesson two: The difference between a rogue agent and a special operative is permission.

12. It’s not your job to prove you’re clever. It’s your job to change outcomes.

13. A quick introduction to Lean Analytics

14. Kevin Costner is a lousy entrepreneur. Don’t sell what you can make. Make what you can sell.

15. The core of Lean is iteration.

16. Unfortunately, we’re all liars.

17. Everyone’s idea is the best right? No data, no learning. People love this part! (but that’s not always a good thing) This is where things fall apart.

18. Analytics can help.

19. Analytics is the measurement of movement towards your business goals.

20. In a startup, the purpose of analytics is to iterate to product/market ﬁt before the money runs out.

21. In a big company, analytics replaces opinion with fact.

22. It helps you innovate on product, market, and method.

23. Some fundamentals.

24. A good metric is: Understandable Comparative If you’re busy explaining the data, you won’t be busy acting on it. Comparison is context. A ratio or rate The only way to measure change and roll up the tension between two metrics (MPH) Behavior changing If you’re busy explaining the data, you won’t be busy acting on it.

25. The simplest rule If a metric won’t change how you behave, it’s a bad metric. h"p://www.ﬂickr.com/photos/circasassy/7858155676/

26. Metrics help you know yourself. Customers that buy >1x in 90d 1-15% 15-30% >30% Then you are in this mode Your customers will buy from you You are just like 70% Acquisition Once Hybrid 2-2.5 20% per year of retailers Loyalty >2.5 10% per year of retailers of retailers Focus on Low acquisition cost, high checkout Increasing return rates, market share Loyalty, selection, inventory size (Thanks to Kevin Hillstrom for this.)

27. Qualitative Quantitative Unstructured, anecdotal, revealing, hard to aggregate, often too positive & reassuring. Numbers and stats. Hard facts, less insight, easier to analyze; often sour and disappointing. Warm and fuzzy. Cold and hard.

28. Exploratory Reporting Speculative. Tries to ﬁnd unexpected or interesting insights. Source of unfair advantages. Predictable. Keeps you abreast of the normal, day-to-day operations. Can be managed by exception. Cool. Necessary.

29. Rumsfeld on Analytics we know Are facts which may be wrong and should be checked against data. we don’t know Are questions we can answer by reporting, which we should baseline & automate. we know Are intuition which we should quantify and teach to improve effectiveness, efﬁciency. we don’t know Are exploration which is where unfair advantage and interesting epiphanies live. know Things we don’t know (Or rather, Avinash Kaushik channeling Rumsfeld)

30. Slicing and dicing data Active users 5,000 Cohort: Comparison of similar groups along a timeline. 0 Jan (this is the April cohort) Feb Segment: Cross-sectional comparison of all people divided by some attribute (age, gender, etc.) Mar ☀ ☁ Apr May A/B test: Changing one thing (i.e. color) and measuring the result (i.e. revenue.) ☀ ☁ ☀ ☁ Multivariate analysis Changing several things at once to see which correlates with a result.

31. Which of these two companies is doing better?

32. March April May Rev/customer $5.00 $4.50 $4.33 $4.25 $4.50 1 2 3 4 5 January How about this one? February Cohort Is this company growing or stagnating? January $5 $3 $2 $1 $0.5 $6 $4 $2 $1 March $7 $6 $5 April $8 $7 $9 February May

33. Cohort 2 3 4 5 January Look at the same data in cohorts 1 $5 $3 $2 $1 $0.5 February $6 $4 $2 $1 March $7 $6 $5 April $8 $7 May $9 Averages $7 $5 $3 $1 $0.5

34. The ability to do things in small batches is behind much of the lean movement.

35. Short cycle time triggers the immune system of big company budgeting.

36. (this is called foreshadowing.)

37. Lagging Historical. Shows you how you’re doing; reports the news. Example: sales. Explaining the past. Leading Forward-looking. Number today that predicts tomorrow; reports the news. Example: pipeline. Predicting the future.

38. Some examples A Facebook user reaching 7 friends within 10 days of signing up (Chamath Palihapitiya) If someone comes back to Zynga a day after signing up for a game, they’ll probably become an engaged, paying user (Nabeel Hyatt) A Dropbox user who puts at least one ﬁle in one folder on one device (ChenLi Wang) Twitter user following a certain number of people, and a certain percentage of those people following the user back (Josh Elman) A LinkedIn user getting to X connections in Y days (Elliot Schmukler) (From the 2012 Growth Hacking conference. http://growthhackersconference.com/)

39. Which means it’s time to talk about correlation.

40. 10000 1000 100 10 1 Jan Feb Mar Apr May Ice cream consumption Jun Jul Aug Sept Drownings Oct Nov Dec

41. Correlated Two variables that are related (but may be dependent on something else.) Ice cream & drowning. Causal An independent variable that directly impacts a dependent one. Summertime & drowning.

42. A leading, causal metric is a superpower. h"p://www.ﬂickr.com/photos/bloke_with_camera/401812833/sizes/o/in/photostream/

43. Growth hacking, demystiﬁed. Pick a metric to change Find correlation Test causality Optimize the causal factor

44. http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/ Is social action a leading indicator of donation?

45. http://blog.justgiving.com/nine-reasons-why-social-and-mobile-are-the-future-of-fundraising/ Is mobile use?

46. The Lean Analytics framework.

47. Eric’s three engines of growth Stickiness Virality Price Approach Keep people coming back. Make people invite friends. Spend money to get customers. Math that matters Get customers faster than you lose them. How many they tell, how fast they tell them. Customers are worth more than they cost.

48. Dave’s Pirate Metrics AARRR Acquisition Activation Retention Revenue Referral How do your users become aware of you? SEO, SEM, widgets, email, PR, campaigns, blogs ... Do drive-by visitors subscribe, use, etc? Features, design, tone, compensation, affirmation ... Does a one-time user become engaged? Notifications, alerts, reminders, emails, updates... Do you make money from user activity? Transactions, clicks, subscriptions, DLC, analytics... Do users promote your product? Email, widgets, campaigns, likes, RTs, affiliates...

49. The five stages Stage EMPATHY STICKINESS Gate I’ve found a real, poorly-met need that a reachable market faces. I’ve figured out how to solve the problem in a way they will keep using and pay for. VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives. REVENUE The users and features fuel growth organically and artificially. SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem.

50. Empathy stage: Localmind hacks Twitter Needed to ﬁnd out if a core assumption—strangers answering questions—was valid. Ran Twitter experiment instead of writing code Asked senders of geolocated Tweets from Times Square random questions; counted response rate Conclusion: high enough to proceed

51. Stickiness stage: qidiq streamlines invites Survey owner adds recipient to group Survey owner adds recipient to group Survey owner asks question Recipient gets invite Recipient installs mobile app Recipient creates account, profile Recipient can edit profile, etc. Recipient reads survey question Recipient responds to question Recipient sees survey results 70-90% RESPONSE RATE 10-25% RESPONSE RATE Survey owner asks question Recipient reads survey question Recipient responds to question Recipient sees survey results (Later, if needed…) Recipient visits site; no password! Recipient does password recovery One-time link sent to email Recipient creates password Recipient can edit profile, etc.

52. Six business model archetypes (Yours is probably a blend of these.)

53. E-commerce SaaS (freemium?) Mobile app (gaming) Two sided marketplace Media User generated content

54. Customer Acquisition Cost paid direct search wom (Which means eye charts like these.) Viral coefficient Viral rate inherent virality VISITOR Freemium/trial offer Invite Others Upselling rate Upselling Enrollment Capacity Limit User Disengaged User Free user disengagement Paid conversion Engaged User Tiering Paying Customer Support data Reactivation rate Freemium churn Reactivate Trial Over Disengaged Dissatisfied Trial abandonment rate Resolution Cancel Cancel Reactivate Account Cancelled Billing Info Exp. Paid Churn Rate FORMER USERS User Lifetime Value FORMER CUSTOMERS Customer Lifetime Value

55. Model + Stage = One Metric That Matters. The business you’re in The stage you’re at E-Com SaaS Mobile 2-Sided Media Empathy Stickiness Virality Revenue Scale One Metric That Matters. UCG

56. Really? Just one?

57. Yes, one.

58. In a startup, focus is hard to achieve.

59. Having only one metric addresses this problem.

60. www.theeastsiderla.com

61. Moz cuts down on metrics SaaS-based SEO toolkit in the scale stage. Focused on net adds. Net adds up: Net adds ﬂat: Net adds down: Was a marketing campaign successful? Were customer complaints lowered? Was a product upgrade valuable? Can we acquire more valuable customers? What product features can increase engagement? Can we improve customer support? Are the new customers not the right segment? Did a marketing campaign fail? Did a product upgrade fail somehow? Is customer support falling apart?

62. Metrics are like squeeze toys. http://www.ﬂickr.com/photos/connortarter/4791605202/

63. Ecommerce Empathy Stickiness Virality 2-sided market Scale Mobile app User-gen content Media Interviews; qualitative results; quantitative scoring; surveys Loyalty, conversion Inventory, listings CAC, shares, SEM, sharing reactivation (Money from transactions) Revenue SaaS Engagement, Downloads, churn churn, virality Content, spam Traffic, visits, returns Inherent virality, CAC Invites, sharing Content virality, SEM WoM, app ratings, CAC (Money from active users) Transaction, CLV Transactions, commission Upselling, CAC, CLV CLV, ARPDAU Affiliates, white-label Other verticals API, magic #, Spinoffs, mktplace publishers (Money from ad clicks) Ads, donations CPE, affiliate %, eyeballs Analytics, user data Syndication, licenses

64. Better: bit.ly/BigLeanTable

65. A company loses a quarter of its customers every year. Is this good or bad?

66. Not knowing what normal is makes you do stupid things.

67. Listening to what normal is makes you ignore disruptive things.

68. (more foreshadowing.)

69. Baseline: 5-7% growth a week • Are there enough people who really care enough to sustain a 5% growth rate? • Don’t strive for a 5% growth at the expense of really understanding your customers and building a meaningful solution • Once you’re a pre-revenue startup at or near product/market ﬁt, you should have 5% growth of active users each week • Once you’re generating revenues, they should grow at 5% a week “A good growth rate during YC is 5-7% a week,” he says. “If you can hit 10% a week you're doing exceptionally well. If you can only manage 1%, it's a sign you haven't yet ﬁgured out what you're doing.” At revenue stage, measure growth in revenue. Before that, measure growth in active users. Paul Graham, Y Combinator

70. Baseline: 10% visitor engagement/day 30% of users/month use web or mobile app 10% of users/day use web or mobile app 1% of users/day use it concurrently Fred Wilson’s social ratios

71. Baseline: 2-5% monthly churn • The best SaaS get 1.5% - 3% a month. They have multiple Ph.D’s on the job. • Get below a 5% monthly churn rate before you know you’ve got a business that’s ready to grow (Mark MacLeod) and around 2% before you really step on the gas (David Skok) • Last-ditch appeals and reactivation can have a big impact. Facebook’s “don’t leave” reduces attrition by 7%.

72. Baseline: CLV calculation 25% 5% 2% monthly churn monthly churn monthly churn 100/25=4 100/5=20 100/2=50 The average customer lasts 4 months The average customer lasts 20 months The average customer lasts 50 months

73. Baseline: CAC under 1/3 of CLV • CLV is wrong. CAC Is probably wrong, too. • Time kills all plans: It’ll take a long time to ﬁnd out whether your churn and revenue projections are right • Cashﬂow: You’re basically “loaning” the customer money between acquisition and CLV. • It keeps you honest: Limiting yourself to a CAC of only a third of your CLV will forces you to verify costs sooner. 20 mo. CL $30/month per customer $600 CLV 1/3 spend $200 CAC Now segment those users!

74. Putting this to work: The Lean Analytics Cycle

75. Pick a KPI Success! Pivot or give up Draw a new line Try again Did we move the needle? Measure the results Draw a line Find a potential improvement Without data: make a good guess With data: ﬁnd a commonality Design a test Hypothesis Make changes in production

76. Do AirBnB hosts get more business if their property is professionally photographed?

77. Gut instinct (hypothesis) Professional photography helps AirBnB’s business Candidate solution (MVP) 20 ﬁeld photographers posing as employees Measure the results Compare photographed listings to a control group Make a decision Launch photography as a new feature for all hosts

78. 5,000 shoots per month by February 2012

79. Hang on a second.

80. SRSLY? Gut instinct (hypothesis) Professional photography helps AirBnB’s business

81. Pick a KPI Success! Pivot or give up Draw a new line Try again Did we move the needle? Measure the results Draw a line Find a potential improvement Without data: make a good guess With data: ﬁnd a commonality Design a test Hypothesis Make changes in production

82. Without data: make a good guess With data: ﬁnd a commonality “Gee, those houses that do well look really nice.” “Computer: What do all the highly rented houses have in common?” Maybe it’s the camera. Camera model.

83. Landing page design A/B testing URL shortening Publisher analytics Cohort analysis General analytics Funnel analytics SaaS analytics User analytics Spying on users Inﬂuencer Marketing Gaming analytics User segmentation User interaction Customer satisfaction KPI dashboards

84. Agenda An introduction to Lean Analytics (30m) The challenges of being big (15m) When you have support (30m) The Lean Analytics framework (30m) A dose of pragmatism (15m) Metrics for innovation portfolios (15m) Break (15m) Some non-tech examples (10m) Tools of the trade (15m) When it’s you against the world (20m) Break (15m)

85. It’s diﬀerent when you’re big.

86. It is way too easy to mix these up. Target market B2B B2C Early stage Big/incumbent Company size/age Less WoM More formal decisions Intrapreneurs Business model vs. company stage Slower cycle time More legacy constraints

87. If big ﬁrms can’t innovate, it’s this guy’s fault.

88. When product and market are known, companies compete on how they do things.

89. To get the incremental cost to zero, companies competed on scale. (Literally, an economy of scale)

90. Scale comes from process, IP, org chart, capitalization. All of these assume the future will be like the past, only more so.

91. If a startup is an organization designed to search for a sustainable, repeatable business model, then an established company is an organization designed to perpetuate one.

92. Technology has radically changed the incremental cost of businesses.

93. Software is eating the world. http://www.ﬂickr.com/photos/ebolasmallpox/3733059220/

94. An economic order quantity of one. Crafted Massproduced Automated Digital Quantity Few Many Some One Cost High Low Medium Free Lead time Small Large Medium None Self-service Medium None Some Lots Customization High None Some Lots This is why software is eating the world. • Cloud computing • Social media • 3D printing • Per-customer analysis • Mobile tracking • Etc...

95. Sustainable competitive advantage allows for inertia and power to build up along the lines of an existing business model, which will soon die. Instead, seek transient competitive advantage. Rita Gunther McGrath, The End of Competitive Advantage

96. Scale is now a liability. Compete on cycle time.

97. Optimizing the probable means discounting the possible.

98. This isn’t about a lack of resources.

99. http://www.flickr.com/photos/maladjusted/5207565912

100. Blockbuster had a lot going for it.

101. Plenty of inventory, of course. But that matters less than...

102. ...market intelligence, customers, existing payment approval, and customer history.

103. The problem was framing: Blockbuster thought it was in the video store management business. Netﬂix realized it was in the entertainment delivery business.

104. YOU ARE HERE

105. OPTIMIZATION OF CURRENT METRICS LOCAL MAXIMUM YOU ARE HERE

106. INNOVATION WITH NEW RULES YOU ARE HERE GLOBAL MAXIMUM

107. YOU ARE HERE SHORT-TERM INVESTORS HATE GOING DOWNHILL

108. First mover advantage happens long before the market emerges. • $1B invested in Nook • $475M operating loss in April 2013 • CEO gone

109. Constraints slow things down vs.

110. Everything to lose: Why big companies need innovation.

111. F500 Life Expectancy (http://csinvesting.org/2012/01/06/fortune-500-extinction/) 75 15 years years 1950 Growth by entering a new business ... 2010 95 99 % fail % fail Corporate Strategy Board Clay Christensen

112. In other words, if your job is change you have your work cut out for you.

113. 2011 MIT study of 179 large publicly traded ﬁrms Companies that use data-driven analytics instead of intuition have 5%-6% higher productivity and proﬁts than competitors. Brynjolfsson, Erik, Lorin Hitt, and Heekyung Kim. "Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance?." Available at SSRN 1819486 (2011).

114. A dose of pragmatism

115. Many models for enterprise innovation Core Adjacent Transformative Do the same thing better. Nearby product, market, or method. Start something entirely new. Regional optimizations. Innovation, go-tomarket strategies. Reinvent the business model. • Customer development • Test similar cases • Parallel deployment • Analytics & cycle time • Fail fast • Skunkworks/R&D • Focus on the search • Ignore the current model & margins • Get there faster • Smaller batches • Solution, then testing • Increased accountability

116. Another way to look at it Core Adjacent Transformative Know the problem (customers tell you it) Know the solution (customers/regulations/ norms dictate it.) Know the problem (market analysis) Don’t know the solution (non-obvious innovation confers competitive advantage.) Don’t know the problem (just an emerging need/ change) Don’t know the solution. Waterfall: Execution matters Agile/scrum: Iteration matters Lean Startup: Discovery matters

117. The Three Horizons Core Adjacent Horizon 1 improves the Horizon 2 extends the current business operations business into new products, in the next 12 months. markets, or methods in the next 3 years. Those core businesses most readily identified with the company name and those that provide the greatest profits and cash flow. Maximize remaining value. Emerging opportunities, including rising entrepreneurial ventures likely to generate substantial profits in the future but that could require considerable investment. Transformative Horizon 3 changes the industry you’re in and your value network in the next 6 years. Ideas for profitable growth down the road—for instance, small ventures such as research projects, pilot programs, or minority stakes in new businesses. http://www.mckinsey.com/insights/strategy/enduring_ideas_the_three_horizons_of_growth

118. Lean applies. Startup may not. Core Adjacent Transformative Lean startup Lean methodologies.

119. Method (new “how”) 3 kinds of innovation Product (new “what”) Market (new “who”)

120. Another view Core Method (new “how”) Market diversiﬁcation Distribution innovation Pure startup Adjacent Disruptive Inventive Market (new “who”) Channel expansion Product (new “what”)

121. A three-maxima model of enterprise innovation Business optimization (ﬁve mores) Current state Business model innovation Product, market, method innovation You can convince executives of this because some of it is familiar. This terriﬁes them because it eats the current business.

122. Improvement Adjacency Remodeling Do the same, only better. Explore what’s nearby quickly Try out new business models Lean approaches apply, but the metrics vary widely. Sustain/ core Innovate/ adjacent Disrupt/ transformative

123. More things To more people Sustaining innovation is about more of the same. (says Sergio Zyman) Inventory increase Gifting, wish lists Highly viral offering Low incremental order costs For more money Maximum shopping cart Price skimming/tiering More often Loyal customer base that returns Demand prediction, notiﬁcation More eﬃciently Supply chain optimization Per-transaction cost reduction

124. Software, experimentation, and iterative cycles of learning help you get to the local maximum better and faster. That’s a good thing. But it’s not the only thing.

125. Adjacent innovation is about changing one part of the model in a way that alters the value network.

126. Adjacent product to the same market in the same way

127. Selling the same product to an adjacent market in the same way. Of P&G’s 38 brands, only 19 were sold in Asia as of 2011 Market expansion is seldom selling the same thing to new people. In Asia, P&G needed to Align pricing with novelty (prestige, mass-tige, over-the-counter) Change consumer expectations (moving from dilutes to concentrates) Adjust positioning and ingredients such as white fungus, ginseng, and the parasitic cordyceps

128. Selling the same product to the same in The biggest innovation logistics of the 20th century. market in a new way. http://www.flickr.com/photos/photohome_uk/1494590209

129. Selling the same product to the same market in a new way.

130. (At this point, observant Intrapreneurs should be asking, should P&G be in the house cleaning business? And that would be transformative.)

131. Transformative innovation is about taking a leap, changing more than one dimension simultaneously in search of a new business model.

132. If sustaining, incremental innovation produces linear growth, then disruptive, transformative innovation produces exponential growth.

133. Transformative isolation: Skunkworks

134. Some non-tech examples.

135. I lied. Everyone is a tech company.

136. Cost of experiments: down. http://www.ﬂickr.com/photos/puuikibeach/4789015423 Cost of attention: way up. http://www.ﬂickr.com/photos/elcapitanbsc/3936927326

137. Let’s pick on restaurants for a while.

138. Empathy: ﬁnd the need Before opening, the owner ﬁrst learns about the diners in her area, their desires, what foods aren’t available, and trends in eating. Key metrics: Popular items; frequent questions; before/after dining patterns. Reference: Emerging need.

139. Stickiness: conﬁrm the need is met. She develops a menu and tests it out with consumers, making frequent adjustments until tables are full and patrons return regularly. She’s giving things away, asking diners what they think. Variance and uncertain inventory make costs high. Key metrics: Customer loyalty; recommendations; referrals; endorsements; inventory turnover. Reference: Business idea.

140. Virality: will it spread? She starts loyalty programs to bring frequent diners back, or to encourage people to share with their friends. She engages on Yelp and Foursquare. Key metrics: Customer loyalty; recommendations; referrals; endorsements. Reference: Business positioning

141. Revenue: prove the business viability With virality kicked off, she works on margins—fewer free meals, tighter controls on costs, more standardization. She focuses on the price of acquiring new customers. Key metrics: Acquisition cost, revenue per cover, capacity, turnover. Reference: Business model.

142. Scale: prove it’s a market Knowing she can run a proﬁtable business, she funnels revenues into marketing and promotion. She reaches out to food reviewers, travel magazines, and radio stations. She launches a second restaurant, or a franchise. Key metrics: Franchise health; repeatability; problems escalated; variance; franchise revenues. Reference: Business plan.

143. A line in the sand 30% Labor costs Gross revenue = 24% 20% Too costly? Just right Understaffed?

144. A leading indicator (Varies by restaurant. McDonalds ≠ Fat Duck.) 50 reservations at 5PM http://www.ﬂickr.com/photos/mysticcountry/3567440970 250 covers that night http://www.ﬂickr.com/photos/avlxyz/4889656453

145. Restaurant MVP http://www.ﬂickr.com/photos/southbeachcars/6892880699

146. Is tip amount a leading indicator of longterm revenue?

147. Why does every table get the same menu?

148. Is purple ink better? http://tippingresearch.com/uploads/managing_tips.pdf

149. Stalking customers is pretty easy. http://targetmycustomers.appspot.com http://tippingresearch.com/uploads/managing_tips.pdf

150. When it’s you vs. the world. (A bagful of tricks from agitators in companies of all sizes.)

151. The skills you need make you a pariah. Successful innovators share certain attributes. Bad listener: Willfully ignore feedback from your best customers. Cannibal: If successful, destroying existing revenue streams. Job killer: Automation & lower margins are your favorite tools. Security risk: Advocate of transparency, open data, communities. Narcissist: Worry constantly about how you’ll get attention. Slum lord: Sell to those with less money, deviants, and weirdos.

152. New Know what kind of innovation you’re Market after. Current Based on H. Igor Ansoﬀ’s matrix Startup: New products for new !) markets. News rules, es cc units, business su at organizational re g nd structure. Innovation. (a Market development: Sell existing products to new markets, segments, uses. Export & license. lf a ll a Penetrate: itic ol Increase revenues, fp o market share, product sk ri ed quality, brand as re differentiation. nc I Marketing. Current ut o Product development: Invent new products for your market. R&D, enhancements. Acquisition. Product New

153. Frame it like a study Product creation is almost accidental. Unlike a VC or startup, when the initiative fails the organization still learns. http://www.flickr.com/photos/creative_tools/8544475139

154. When in doubt, collect data From tackling the FTA rate to visualizing the criminal justice supply chain.

155. Use data to create a taste for data Sitting on Billions of rows of transactional data David Boyle ran 1M online surveys Once the value was obvious to management, got license to dig.

156. Don’t just collect data, chase it.

157. Understand hidden constraints That pencil story is a myth. Graphite is conductive and explosive. The Minimum Viable Product is Viable for a reason.

158. Know what has to be built in-house SAP integration Employment law

159. Think subversively. http://www.ﬂickr.com/photos/bootbearwdc/1243690099/

160. Everything’s an excuse to experiment

161. Seriously. Everything.

162. Run it as a consulting business ﬁrst. (Just don’t get addicted to it. Your goal is to learn and overcome integration challenges and ﬁnd the 20% of features that 80% of the market will pay for.)

163. Convince your boss she asked for this Success! Pick a KPI Pivot or give up Draw a new line Try again Did we move the needle? Measure the results Draw a line in the sand Find a potential improvement Without data: make a good guess With data: ﬁnd a commonality Design a test Make changes in production Hypothesis

164. Slaughter a sacred cow: Prove a long-held assumption is wrong and you’ve got people’s attention. Know what you’ll do with it ahead of time.

165. Take baby steps.

166. Netﬂix

167. Tesla http://www.hdwallpapersinn.com/wp-content/uploads/2012/12/600-tesla.jpg

168. Twitter’s 140-character limit isn’t arbitrary. It’s constrained by the size of SMS (160 characters) and username (20 characters.) http://i.i.cbsi.com/cnwk.1d/i/tim/2011/11/18/ sms_screen_twitter_activity_stream_270x405.png

169. Figure out how to translate it back to a simple model that ﬁts the company’s existing value model. If your company dies, this is why.

170. Intrapreneurs often have to use proxies Stage Startup metrics Intrapreneur metrics Empathy Customers interviewed (needs & solutions), assumptions quantified, TAM, monetization possibility Non-customers interviewed; assumptions quantified, constraints identified, TAM, disruption potential Churn, engagement Support tickets, integration time, call center data, delays Viral coefficient, viral cycle time Net Promoter Score, referrals, case study willingness Revenue Attention, engagement Billable activity; signed LOIs; pilot programs; after-development profitability Scale Automation Contribution, training costs, licensing Stickiness Virality

171. When you have support. (What companies like P&G, Cognizant, GE, and Motorola do with a formal innovation program.)

172. Do you really have permission? What resources do you have? Staff, budget, unfettered access to customers? What scope of change can you make? Pricing, product, channel, branding?

173. Step one: Develop a portfolio approach.

174. Innovation portfolios at big companies Return Investment Core Adjacent 70% 20% 10% 20% Transformative 10% 70%

175. 1. Organizations’ structures emerge as a way to optimize the current business model. 2. Most innovations will come not from product or market, but from method— business model innovation. 3. Innovation groups must exercise organizational amnesia at the outset.

176. Step one: Frame your problems.

177. Maybe not so crazy. http://www.ﬂickr.com/photos/mizrak/4592706544

178. Step two: Deﬁne your gates and ﬁlters. These may lead to myopia. They are also your unfair advantages.

179. Step three: Secure funding, resources, and executive backing.

180. Step four: Generate new ideas.

181. Find non-obvious adjacencies POWER GRID WHICH FEEDS A NEEDS LIGHT AN BULB ELECTRICAL GENERATOR HAS A TURBINE LIKE A TURNED AROUND BECOMES A PLANE ENGINE TRAIN ENGINE REQUIRES SPINS & VIBRATES LIKE AN AND LOOKS LIKE A SOFTWARE TO CUT DOWN TREES BETTER MRI MACHINE WIND TURBINE

182. Build an ecosystem Canada’s largest directory publishing and local marketing services company 1.5M listings from 420K SMB & national customers Revenues >$1.2B 2,500 employees Created third-party listing API Took 8-10 mo (2009-10) to get approval API payoff happened 2y later KPI evolution Yahoo replaced Canadian digital properties search with the YellowAPI Soft: Signups, SDK, downloads Improved SEO, Comscore App usage, deals signed Functional prototype in hours, testing in days, and launching in weeks. Faster time to partnerships Budgets tripled in 2013 API calls generated API-generated revenue

183. Five common models for transformative innovation Acquisition Collaboration Isolation Buy promising startups Crowdsource, work with universities, suppliers, etc. Create a separate group with different conditions Incubation Internal startup ecosystem; LoB are “investors” Integration The LoB does innovation internally

184. Step ﬁve: Test by doing (experimentation beats projection.)

185. Focus on the model, not the plan Demand Amt Growth Wk 1 Wk 2 Wk 3 People per day on sidewalk 200 4% 204 Percent that buy a glass 10% 5% 15% 20% 25% Daily customers 20 31 Revenue Price per cup Proﬁt per cup Daily proﬁt 43 225 56 $156 $216 $281 $5 $5 Cost of Goods Cost per cup 216 $5 $5 30.6 41.5 52.8 $1 -2% .98 .96 .94 $4 4.02 4.04 4.06 $80 125 175 228

186. A business plan is just what happens when you drag the business model to the right.

187. Designing an experiment Problem, solution, and result hypothesis Test strategy (PoC, survey, interviews, kickstarter, prototype, A/B, etc.) Cohort & segment to be tested Metric or assumption being tested Timebox or total for test Action you’ll take if you pass or fail

188. Step six: Know what happens afterwards

189. Qualcomm’s innovation model: What was missing Hypothesis Unclear what happened to founders Needed a middle PoC decision Sustainability, not feasibility http://blogs.berkeley.edu/2013/01/28/ designing-a-corporateentrepreneurship-program-aqualcomm-case-study-part-1-of-2/ Experiment Boot camp Idea generation and selection Existing models Implement Idea advancement POC Biz sustainability Tech feasibility POC Ideas New models Open innovation Strategic value decision Boot camp decision End user/partner desirability Option value Implement decision Actions Exit value Company crowd storm Small team designs & Company extracts value conducts experiments

190. Step seven: Rinse, repeat.

191. The Lean Analytics lifecycle of an Intrapreneur Get buy-in Political fallout Find problems; don’t test demand. Skip the business case, do analytics Entitled, aggrieved customers Stickiness Know your real minimum based on expectations, regulations Hidden “must haves”, feature creep Virality Build inherent virality in from the start; attention is the new currency Luddites who don’t understand sharing Revenue Consider the ecosystem, channels, and established agreements Channel conﬂict, resistance, contracts Hand the baton to others gracefully Hating what happens to your baby Beforehand Empathy Scale

192. Metrics for innovation portfolios.

193. Core metrics Business plan. Metrics that matter Assume it will improve. • Return on investment • Total cost of ownership • Improvement in KPI • Total served market Product, market, and method will remain the same Examples: Redesigning packaging; pricing adjustment

194. Adjacent metrics Business model. Metrics that matter Assume it will fail. • Virality & word of mouth • Early adopter stickiness • Regulation • Total addressable market Your ultimate use case won’t be what you think it is today. Example: Mr. Clean Magic Eraser

195. Transformative metrics Business idea. Assume it will fail. You hope it will have the consequences you want but aren’t sure how. Example: Headcam recordings of all ofﬁcers Metrics that matter • People I’ve talked to • Prototype creation speed • Assumptions validated • Problems uncovered • Technical feasibility • Hidden constraints

196. Key points to clarify in an innovation program Hypothesis Experimentation • Articulate problems • Frame known advantages • Define the right filters • Many idea sources • Confirm funding (money, • Prioritize riskiest • • • people, customer access) Agree on analytical framework Balance market, product, & method adjacencies • • assumptions Time-box assessment stages Test technology, demand, and business feasibility MVP, prototype, pilot, or science as appropriate for type of innovation Implementation • Temporary incubator • Find a home or building • • • one Keep innovators involved Merge metrics with existing business KPIs Synchronize innovation cycles with enterprise cycles (budget, etc.) Portfolio metrics; Gates and KPIs for each stage; mix of core, adjacent, and disruptive innovation.

197. Hypothesis Goals, constraints, context Sourcing Top-down Bottom-up Sourcing Filtering Biases Strengths Focus Mandate Outside-in ost e l alm as th wil s w this . thi at re a o . e th ange. here a d t ot ch N y dt nee n inl eck a ions I erta P d c rat it. MV of alte e to ak nch m bu Boot camp POC Core Optimization Crosspollinate to current managers Adjacent De-risking Best solution Test/ validate w/current customers Disruptive Reframing R&D M&A Grow as a distinct business Evaluation of the innovation program itself Implementation Socializing Integration Adoption by existing line of business. Independence Creation of a new line of business.

198. Some tools and tricks

199. Traction graphs Your business model The stage you’re at ... change often if you’re doing it right. Your one metric So how do you track that over time?

200. Traction graphs Jan Feb Signups per day This axis changes for each metric Mar Conversion rate Apr Churn rate May Jun Viral coefﬁcient

201. Traction graphs 0% Jan Signups per day Feb Mar Conversion rate Apr Churn rate May Jun Viral coefﬁcient

202. Use vanity to get to meaningful metrics Your goal is to produce outcomes If the outcomes require action, and vanity motivates actors, use it But show how the vanity metric is a leading indicator of the real one Web traﬃc Activation Cart Size Conversion x rate Revenue

203. The three threes Three assumptions What big bets are you making? • “People will answer questions” • “Organizers are frustrated with how to run conferences” • “We'll make money from parents” • “Amazon is reliable enough for our users.” Three actions to take What are you doing to make these assumptions happen (or identify they’re wrong and change course?) • Product enhancements • Marketing strategies Three experiments to run • Feature tests • Continuous deployment • A/B testing • Customer survey

204. The three threes Three assumptions Monthly Board, investors, founders Strategy Three actions to take Weekly Executive team Tactics Three experiments to run Daily Employees Execution

205. The three threes Many people will answer questions Three assumptions Three actions to take Three experiments to run Get more people Change the UI Increase answer % Test timings Test better questions Questions from peers

206. The problem-solution canvas The Goal is to Learn CURRENT STATUS • List key metrics you’re LAST WEEK’S LESSONS LEARNED AND ACCOMPLISHMENTS) • What did you learn last week? tracking, where they’re at, and • What was accomplished? compare with last few weeks • On track: YES / NO? How are things trending? •

207. The problem-solution canvas Problem #1 (put name here) HYPOTHESIZED SOLUTIONS • List possible solutions that you’ll start working on next week. Rank them. • Why do you believe each solution will help you solve or complete solve the problem? METRICS / PROOF + GOALS • Metrics you’ll use to measure whether or not the solutions are doing what you hoped (solving the problem) • List proof (qualitative) you’ll use as well • Deﬁne goals for the metric Problem #2 (put name here) HYPOTHESIZED SOLUTIONS • List possible solutions that you’ll start working on next week. Rank them. METRICS / PROOF + GOALS • Metrics you’ll use to measure whether or not the solutions are doing what you hoped (solving the problem)

208. Use proxy data. Just be careful.

209. “A subjective degree of belief should rationally change to account for evidence.” (AKA Bayes’ Theorem.)

210. Conclusions

211. Key points Intrapreneurship is about adjacent or transformative innovation Sustaining innovation focuses on the Five Mores, within the current product, market, method, and business model. Adjacent innovation may come from a new product, market, or method, but the same business model Disruptive innovation has different customers, KPIs, and models The difference between a rogue agent and a special operative is permission Portfolios need framing, sourcing, ﬁlters, metrics, and socializing Balancing isolation and integration, R&D and M&A is contentious

212. “The most important ﬁgures that one needs for management are unknown or unknowable, but successful management must nevertheless take account of them.” Lloyd S. Nelson

213. Pic by Twodolla on Flickr. http://www.ﬂickr.com/photos/twodolla/3168857844

214. ARCHIMEDES HAD TAKEN BATHS BEFORE.

215. Once, a leader convinced others in the absence of data.

216. Now, a leader knows what questions to ask.

217. Ben Yoskovitz byosko@gmail.com @byosko Alistair Croll acroll@gmail.com @acroll