Optimizing website conversions through A/B split testing
•
2 recomendaciones•901 vistas
In this presentation Javid Jamae CTO/Co-founder of Camperoo (www.camperoo.com) will teach you how to define, run, and measure A/B tests and other experiments in your Lean Startup. This was presented at the Houston Lean Startup Circle on June 18, 2013.
Recomendados
Recomendados
Más contenido relacionado
Destacado
Destacado (17)
Similar a Optimizing website conversions through A/B split testing
Similar a Optimizing website conversions through A/B split testing (20)
Último
Último (20)
Optimizing website conversions through A/B split testing
- 1. OPTIMIZING CONVERSIONS THROUGH A/B SPLIT TESTING Javid Jamae CTO / Co-founder Camperoo Presented at the Houston Lean Startup Circle June 18, 2013 Special thanks to - EmilianoVillareal
- 2. ‘‘The goal of a Lean Startup is to inform our riskiest business model assumptions through empirical testing with customers – not rhetorical reasoning on a white board. ~ ASH MAURYA
- 6. Users coming in through a Zappos- style homepage will be 10% more likely to go from the homepage into a product search results page EXPERIMENTDESIGN FALSIFIABLE HYPOTHESIS EXPERIMENT SCOPE 40 man-hours to design and code 750 unique visitors per variant
- 7. UNIQUE VISITORS RANDOMIZER ANALYTICS CONTROL VARIANT EXPERIMENTIMPLEMENTATION 50% 50%
- 8. THEWINNERWAS? 5.86% ?? Percentage of people going from the homepage to a product search results page Control Variant
- 9. THEWINNERWAS? 5.86% 6.84% 16.89% increase! Percentage of people going from the homepage to a product search results page Control Variant
- 10. BUTWHY? Was it the: - Filters - Category browsing - Different slider - The combination ?
- 11. In Lean Startups, success is defined by learning, not by test results X A/B TOOL √ Validated Learning Loop √ Variant
- 12. +
- 13. Users coming into an AirBNB-style homepage with categories will be 5% more likely to go into a product search results page than without categories EXPERIMENTDESIGN FALSIFIABLE HYPOTHESIS EXPERIMENT SCOPE 1 man-hours to design and code 750 unique visitors per variant
- 14. THEWINNERWAS? 5.90% ?? Percentage of people going from the homepage to a product search results page Control Variant
- 15. THEWINNERWAS? 5.90% 5.98% Percentage of people going from the homepage to a product search results page Control Variant 1.35% increase
- 16. THELEANPROCESS IDEA HYPOTHESIS EXPERIMENT DESIGN EXPERIMENTATION MEASUREMENT Scientific = Process
- 19. ‘‘ If you measure something and it’s not attached to a goal, in turn changing your behavior, you’re wasting your time. ALISTAIR CROLL & BENJAMIN YOSKOVITZ BOOK: LEAN ANALYTICS
- 20. GOALORIENTEDPLANNING Landing Page Changes SEO Rankings Goal: Increase Acquisition Inbound Channels
- 23. Lean Analytics (Lean Series). O’Reilly Media.
- 24. Stage RISKSACROSSSTAGESProductCustomerMarket Problem/Solution Identify Competition/ Positioning Early / Inception MVP Narrow/Outbound Pricing Product Scale Inbound Cost Structure Scale Risks Qualitative, Activation / Retention Quantitative, Revenue / Growth
- 25. Favor experiments that make sense for your current stage & goals
- 26. Favor experiments that are innovative
- 27. Favor experiments that have high risk
- 28. Favor experiments that you can measure
- 29. Favor experiments that engaged customers seem to need or are asking for
- 30. Theory of Constraints A chain is no stronger than its weakest link. The Goal - Eliyahu M. Goldratt
- 32. Is Manchester United causing global warming?
- 34. ‘‘If you torture the data long enough, it will confess to anything. ~UNKNOWN
- 35. Don’t take Leaps of Faith
- 36. Leap of faith: A Zappos-style front page will increase browsability on our website Falsifiable hypothesis: Users coming in through a Zappos-style homepage will be 10% more likely to go from the homepage into a product search results page (vs. the AirBNB-style homepage)
- 37. Formula: [Repeatable Action] will [Expected Result] Writing a blog post about Houston activities will result in 25 new accounts Users entering Variation B of the store page will be 2% more likely to purchase Putting an item on sale will increase the chances that it will be added to a cart by 10%
- 38. Draw a line in the sand... ...before you start gathering data.
- 40. Are you measuring impact on a Goal or a Metric
- 42. Decide on a exit criteria up front
- 43. Cost It will take 5-10 hours to implement, if it takes longer, we will abandon. Scope We need 1,000 unique visitors per variant before the data has any meaning to us. Timebox If the test doesn’t end by July 2, then we will abandon. Follow up If the test fails, then try experiment C. If the test passes, then experiment B.
- 44. Pick the type of analysis
- 46. Pick the type of test
- 47. UNIQUE VISITORS RANDOMIZER CONTROL VARIANT Version A Version B A/B
- 48. UNIQUE VISITORS RANDOMIZER CONTROL VARIANT 1 A/B/N VARIANT 2 VARIANT 3
- 49. MULTIVARIATETESTING Enroll Sign Up Join Now Color: Text: Size: 27 Combinations
- 50. MULTIVARIATETESTING Enroll Sign Up Join Now Enroll Sign Up Join Now Enroll Sign Up Join Now Enroll Sign Up Join Now Enroll Sign Up Join Now Enroll Sign Up Join Now Enroll Enroll Enroll Sign Up Sign Up Sign Up Join Now Join Now Join Now
- 52. Amazon A/B Testing Service (beta) FREETOOLS Google Analytics - Experiments API Google Analytics - Content Experiments
- 53. WYSIWYGTOOLS
- 54. ENTERPRISETOOLS
- 56. THECAMPEROOWAY? github.com/andrew/split SPLIT + MIXPANELOpen Source, Ruby Freemium, SAAS mixpanel.com
- 58. Goal Build Measure Learn Increase Acquisition Zappos Redesign AirBNB with category navigation Search engine display of customer ratings Increase Activation Draconian modal dialog box KANBANBOARD
- 61. WEEKLYPLANNING Weekly meeting: Review experiment results Challenge interpretation Kill or restart overdue experiments Decide next experiments