Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Triangulating Data to Drive Growth

307 visualizaciones

Publicado el

How to use both big and thick data, quantitative and qualitative user studies, to drive product development, design, and growth?

Case studies at KKTV and examples of leveraging Amplitude Analytics.

Publicado en: Datos y análisis
  • Sé el primero en comentar

Triangulating Data to Drive Growth

  1. 1. Leveraging Amplitude Case Studies @ KKTV Triangulating Data to Drive Growth Jason @ Growth Team | 20171110 1
  2. 2. 2 Jason Hou Growth Team Lead at KKTV Joined KKTV 2 months before service launch Built Growth Team from 0 to 4 Started using Mixpanel, Google Analytics by 2013 Started using Amplitude by 2014
  3. 3. 3 Actually.. This sharing is more than demonstrating how we use Amplitude
  4. 4. 4 Actually.. We hope to share our methodology and mindset of triangulating different types of data Of course, Amplitude is probably the easiest way for most people, at the time of this sharing
  5. 5. 5 Triangulating Data Triggers Actions Triangulate Macro Trends Quantitative Data Micro Streams Qualitative Data Human Judgement Industry Experience
  6. 6. ● My Highlights of Amplitude Analytics 我眼中的 Amplitde,有哪些亮點? ● Customer Support - From Macro to Micro 讓客服從「巨觀到微觀」 - 從資料點到個體紀錄 ● Marketing - High Definition Custom Audience 讓行銷有「高解析度的自建受眾」 - 優化 FB 廣告名單 ● Messaging Experiment - From Hypo to Actions 訊息實驗 - 從「假設到行動」 ● Cuz “You Never Know” - Triangulating Data 交叉比對 - 因為永遠有意外 6 Agenda
  7. 7. 7 KKTV’s Vision: Re-Invent TV Experience
  8. 8. My Highlights of Amplitude Analytics 我眼中的 Amplitde,有哪些亮點? 8 01
  9. 9. 9 Amplitude Enable Us to.. ● Get realtime data streams (no sampling), and fast reporting ● Jump between macro, aggregated trends to micro, individual streams => Compare two types of data: quantitative and qualitative ● Group users based on their behaviors (events) to create cohorts => No SQL needed to build custom audience or user segments ● Compare cohorts by applying multiple metrics and reports => Generate actionable insights, verify hypotheses quickly
  10. 10. 10 Powering The Entier Team 50%of the team play data every week Among the top 10 Amplitude users in KKTV: 4from Customer Support 3from Growth Team 2from Marketing, and 1from Content Licensing
  11. 11. 11 How We Leverage Amplitude….? Let’s see what the heavy users do ! 3 cases showing you how... ● Customer support ● Marketing ● Growth / Product use Amplitude
  12. 12. Customer Support - From Macro to Micro 讓客服從「巨觀到微觀」 - 從資料點到個體紀錄 12 02
  13. 13. 13 Whenever There’s a New Issue.. We enter the user’s info here
  14. 14. 14 This Complete User Activity History Shows Up (Fake Data)
  15. 15. 15 This Complete User Activity History Shows Up (Fake Data)
  16. 16. 16 This Complete User Activity History Shows Up (Fake Data)
  17. 17. 17 This Complete User Activity History Shows Up Events Event Properties User Properties (Fake Data)
  18. 18. 18 When There’s a Sudden Increase of Errors... Oops, bugs ??? (Testing Data)
  19. 19. 19 We Are Able to Dive In Quickly (Testing Data)
  20. 20. 20 Locate History Around the Error, and Share to Developers
  21. 21. 21 Recap: Jump from Macro Trends to Micro Streams (Testing Data)
  22. 22. 22 More Scenarios of Leveraging User Activity ● UX/UR Designers: Peek into user event history before interview ● Growth Team: QA trackings & AB testings, find growth targets ● Customer Support: Find issues, report bugs ● Developers: Trace events before & after bugs, find root cause
  23. 23. 23 Find Growth Targets: For Users Who Dropped Off ... Where did they go? What were they doing INSTEAD? (Demo Data)
  24. 24. Marketing - High Definition Custom Audience 讓行銷有「高解析度的自建受眾」 - 優化 FB 廣告名單 24 03
  25. 25. 25 What is Behavioral Cohort? Group users based on their actions (and/or attributes) See what they do, how they perform This sharing is also using this concept => Select top Amplitude users in KKTV => Show what they do
  26. 26. 26 FB Custom or Lookalike Audience Is Key to Boost Ad Return Create Custom Audience Create Lookalike Audience
  27. 27. 27 Build Cohorts Without SQLs (Demo Data)
  28. 28. 28 Build Cohorts Without SQLs Export, upload to FB, and then create Custom Audience (Demo Data)
  29. 29. 29 ● Marketing: Create Custom Audience ● Marketing: Send out targeted push notifications ● Content Operation: Discovery user persona from watch history More Scenarios of Leveraging Behavioral Cohort ● UX/UR Designers: Send out targeted surveys ● Growth Team: Compare cohorts by applying multiple metrics
  30. 30. 30 Compare Cohorts by Applying Multiple Metrics (Demo Data)
  31. 31. 31 Compare Cohorts by Applying Multiple Metrics (Demo Data)
  32. 32. Messaging Experiment - From Hypo to Actions 訊息實驗 - 從「假設到行動」 32 04
  33. 33. 33 Assumption vs Reality ● Assumption: ○ KKTV provides cross-platform experience ○ Users will know and jump between platforms Assumption is great, but... ..always remember to double check it with reality OK, HOW?
  34. 34. 34 Users Who Jumped Between Platforms
  35. 35. 35 Assumption vs Reality ● Assumption: ○ KKTV provides cross-platform experience ○ Users will know and jump between platforms ● Reality: (Right after KKTV was launched) ○ We acquired a lot of users on mobile ○ Very few of them used both mobile and web apps
  36. 36. 36 Questions Trigger Actions ● Assumption: ○ KKTV provides cross-platform experience ○ Users will know and jump between platforms ● Reality: (Right after KKTV was launched) ○ We acquired a lot of users on mobile ○ Very few of them used both mobile and web apps ● Follow-up questions: ○ For users who jumped between platforms, how are they different? ○ Is it important to encourage users to do so? How?
  37. 37. 37 More Observations & Quick Validations ● From user researchers: ○ Mobile users were surprised to know there’s KKTV Web App ○ They expressed satisfaction after using it ○ They described scenarios of when and why they would use it ● By using behavioral cohorts: ○ Cohort 1: Select users who jumped between mobile & web platforms ○ Cohort 2: Select users who didn’t ○ We compare two cohorts by applying retention & conversion metrics ○ => Cohort 1 performs far better then cohort 2
  38. 38. ● For users who signed up on mobile platforms... ○ What if we notify them there’s KKTV Web App? ○ How are we going to notify them? 38 Design Experiments ● Experiment examples: ○ Send out a push notification after sign-up, then compare w/ control group ○ Display a in-app welcome message, and show a picture of Web App
  39. 39. ● We inject experiment data into Amplitude ○ Separate users in different experiment groups into cohorts ○ Compare results by applying multiple metrics 39 Analyze Experiments
  40. 40. Cuz “You Never Know” - Triangulating Data 交叉比對 - 因為永遠有意外 40 05
  41. 41. Story of “You Never Know” - 5-Day-Long Session 41 2017-08-17 2017-08-13 Data Scientist Asked: “How is it even possible?”
  42. 42. Story of “You Never Know” - 5-Day-Long Session 42 For a session to end: { App is backgrounded } AND { Stops sending events for 5min }
  43. 43. 43 Story of “You Never Know” - 5-Day-Long Session
  44. 44. 44 Story of “You Never Know” - 5-Day-Long Session Looks strange..
  45. 45. 45 Story of “You Never Know” - 5-Day-Long Session
  46. 46. 46 Triangulating Data Triggers Actions Triangulate Macro Trends Quantitative Data Micro Streams Qualitative Data Human Judgement Industry Experience
  47. 47. THANK YOU 47

×