Product Testing and Refinement
•Descargar como PPTX, PDF•
1 recomendación•2,188 vistas
This presentation was given to Techgirlz annual entrepreneurial summer camp for middle school girls. This presentation was part of a 2-hour workshop on product design, testing, and refinement.
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Destacado
Destacado (8)
Similar a Product Testing and Refinement
Similar a Product Testing and Refinement (20)
Último
Último (20)
Product Testing and Refinement
- 1. Product Testing and Refinement Chris Cera, CTO @ Arcweb July 9, 2014
- 2. What is the name of this cookie?
- 3. Samoas or Caramel deLites?
- 4. What should we name our product?
- 6. The Scientific Method 1. Ask a question 2. Do background research 3. Construct a hypothesis 4. Test your hypothesis 5. Analyze your data and draw a conclusion 6. Report the results
- 9. Talk to prospective customers
- 10. Currency
- 11. Currency
- 12. Currency
- 13. Currency
- 14. Currency
- 15. What should we name our product?
- 16. Hypotheses 1. “Samoas” 2. “Caramel deLites” 3. “Caramel DeLites” 4. … 5. … 6. ... 7. … 8. … 9. … 10. …
- 17. How to test our hypotheses?
- 18. What should we name our product?
- 19. Book Funnel Analysis • English speakers on earth – 100% • Read a business book – 45% • Discover the product ad – 30% • Click the product ad – 15% • Click the “Add to cart” button – 10% • Start the checkout process – 8% • Enter their credit card information – 7% • Enter their shipping information – 6% • Buy the book – 5%
- 20. Optimize
- 21. Hypotheses 1. “The 4-Hour work week” 2. “How to work less” 3. “How to get rich quick” 4. … 5. … 6. ... 7. … 8. … 9. … 10. …
- 23. Budget Challenge: $??? • Given – We have 10 names to test – We will test each one on 10 different people – It costs $1 for each test (i.e. to buy an ad) • How much money do we need?
- 25. The 4-Hour Workweek (of course)
- 26. How else could this have been tested?
- 29. How will users want to sign in?
- 30. Sign In Funnel Analysis • English speakers on earth – 100% • Discover the product – 50% • Download the product – 25% • Launch the product – 20% • Create a username – 10% • Sign into their account – 9% • Launch the product monthly – 5% • Launch the product weekly – 3% • Launch the product daily – 2% • Launch the product hourly – 1%
- 31. Sign In Page: Example
- 32. Sign In Page: Borrowing
- 33. Sign In Page: Borrowing
- 34. Sign In Page: Borrowing 3x
- 35. Sample Test Results • Twitter converts 2% of people • Facebook converts 10% of people
- 36. A/B Testing Execution • Given 3,000 people • A/B test for the first 300 (10% of them) – 6 different campaigns with 50 tests each • Then test final 2,700 people with best campaign. • Results – Worst campaign (Twitter) converted 2% – Best campaign (Facebook) converted 10% – Difference between 54 and 270 people. – For a $50 product, that is $10,800!
- 37. Budget Challenge: Implementation • Implement Twitter login: $10 • Implement Facebook login: $15 • Implement LinkedIn login: $20 • But we only have $10!?! What do we do?
- 38. Group Work
- 39. Group Work • Assignment [50 minutes] – List 1-2 of your funnels, then “guess and test” – Identify a step to optimize – List your hypothesis and test(s) – Test them on other group members, collect currency • Group Presentations [30 minutes] – Steps in the funnel(s) – Optimization step(s) – List your hypothesis and test(s) – Summarize the results
- 40. Thank you • Chris Cera, CTO • cera@arcweb.co • 215.435.2289
Notas del editor
- Question: Why does it have two names? Fake/exciting answer: Maybe they are trying to see what sells better? Real/boring answer: Girl scouts doesn’t impose naming standards on its multiple bakers.
- One of the most important components is the title! Review consequences of guessing wrong: Expensive. Life or death. Question: How would you figure out the right name? Question: How do scientists get answers to scientific questions?
- Sorry, but you still need science outside of science class! Question: Who remembers the scientific method?
- My attempt at creating a simple slogan
- Question: How to validate your hypothesis?
- All products need communication to understand prospective customers. Communication is necessary to collect currency. Questions, comments, and concerns should be addressed. How to get validation? Collect currency from a given sample of people. Question: What are the different forms of currency?
- Question: What is another way of receiving validation?
- Question: What is another way of receiving validation?
- Question: What is another way of receiving validation?
- Question: What is another way of receiving validation?
- Clicks
- Quick recap
- List out all of the hypotheses we want to test
- Verbal: Surveys asking if they will buy. Signature / LOI: Asking customers to sign a form certifying that they will buy once it’s available Pre-pay: Asking customers to buy them ahead of time Question: IMPORTANT!!! WHO IS TOTALLY CONFUSED FROM THIS EXAMPLE?
- Question: Would you do anything different for naming a book? Venue. Typically purchased online whereas girl scout cookies are not. Question: How can you test something like this online?
- Every product has a funnel like this. We are going to optimize the “Click the product advertisement step” Some might call this a WORKFLOW Advanced topics: Conversion Optimization, Conversion Funnel Analysis
- We are optimizing the “click the product ad” The heart of conversion analysis, and what marketers spend a lot of their time doing
- Question: Super simple 3rd grade math question for no really good reason
- Discuss being convinced. Gut feeling vs true data-driven. Sample size, confidence levels, etc
- Tim Ferris actually used search advertising to test the book title
- Surveys asking if they will buy. Asking customers to sign a form certifying that they will buy once it’s available Asking customers to buy them ahead of time
- Question: What is different? Question: Try to guess their hypothesis
- Question: What is different? Question: Try to guess their hypothesis
- Every product has a funnel like this. Some might call this a WORKFLOW Advanced topics: Conversion Optimization, Conversion Funnel Analysis
- Create a username and password
- “Borrow” the login capability from another product Explain how this is done
- Question: How can we test these? Question: Discuss the hypothesis for the “create a username” step in the funnel analysis slide.
- Describe the word “convert” Bring it back to the funnel.
- Question: Explain this in detail. Emphasize: This is just 1 campaign optimization! Really make sure people understand.
- Question: What do we do? How do we not build the functionality, and still learn from the experience. Scope Kills!!! -- Should mention the expense required for implementation. Solution: Have the buttons just collect the number of clicks, but redirect to the Twitter login with a message saying “Sorry, facebook isn’t implemented yet”