Leanplum_Controlled Experimentation_Panel_The Hive
•Descargar como PPTX, PDF•
0 recomendaciones•416 vistas
Recomendados
Recomendados
Más contenido relacionado
Destacado
Destacado (20)
Similar a Leanplum_Controlled Experimentation_Panel_The Hive
Similar a Leanplum_Controlled Experimentation_Panel_The Hive (20)
Más de The Hive
Más de The Hive (20)
Leanplum_Controlled Experimentation_Panel_The Hive
- 2. OPTIMIZING ON MOBILE • WHAT ARE THE CHALLENGES? • HOW DID WE SOLVE THEM? • HOW TO CHOOSE AN A/B TESTING SERVICE?
- 3. LEANPLUM IN A NUTSHELL
- 4. LEANPLUM IN A NUTSHELL 1. IMPLEMENT OUR MOBILE SDK 2. RUN TESTS FROM OUR DASHBOARD 3. VIEW RESULTS
- 6. NO CHANGE
- 7. CHALLENGES 1. FORM FACTOR 2. FRAGMENTATION 3. CONNECTIVITY 4. APP STORE HURDLES 5. METRICS 6. CONVERSIONS ARE DIFFERENT 7. COMPETITION
- 8. 1. FORM FACTOR LIMITED REAL ESTATE TESTING IS ESSENTIAL
- 9. 2. FRAGMENTATION TARGET DIFFERENT OS, CPU, SCREEN SIZES, HARDWARE
- 10. 3. CONNECTIVITY BATCHING, CACHING, DEFAULTS
- 11. 4. APP STORE HURDLES 2 WEEKS! TEST AND CHANGE ON THE FLY
- 12. 5. METRICS DAU/MAU AVG SESSION LENGTH RETENTION FLOWS VIRALITY
- 13. 6. CONVERSIONS ARE DIFFERENT CONVERT EASILY, FOLLOW UP OFTEN
- 14. 7. COMPETITION COST OF USER ACQUISITION IS HIGH A/B TESTING IS CRITICAL
- 15. WHAT ELSE? • EASE OF INTEGRATION • API • FLEXIBILITY • SECURITY
- 16. EARLY ADOPTER PROGRAM • FOUNDERS@LEANPLUM.COM • WWW.LEANPLUM.COM
Notas del editor
- Hi, I’m Andrew, one of the founders and CTO of Leanplum.Platform enablingcompanies tobetter understand and monetize their users on mobile with A/B testing.100s of A/B tests at Google.Applied best practices and methodologies learned from optimizing on the web at Google, but with a solution that addresses the specific challenges of optimizing on compiled client-side mobile platforms
- I want to share with you …
- So, let me describe our platform very briefly, and how it simplifies the process of A/B testing on mobile. It consists of 3 parts:
- 3. What happened linked directly to the tests. statistical significance for every metric,
- Here’s a couple of tests that were run by our platform.One app reminds users to do certain tasks.It’s also important to test downstream effects like how it affected engagement and retention.If there’s one thing I learned at Google, it’s that A/B testing is all about tradeoffs. While we tried to optimize one goal, we may have hurt something else.
- Another test was done on the onboarding slideshow that had captions next to each picture.Originally, the text didn’t have much contrast compared to the background, and was pretty hard to read.But improving the contrast had no significant change. One conclusion might be that there is a deeper issue at hand, that maybe people weren’t trying to read the captions in the first place. They concluded they had to do a deep dive on the metrics to determine what to test next.
- switch gears and talk about some of the mobile specific challenges that should be on your mind that you should be thinking about as you optimize your mobile applications.
- First of all, let’s start with the hardware itself.More things to optimizeWhat you show, how and whenUser flows take more stepsTedious to make every interface file or graphic testable, so we’ve provided a way to swap them out without having to write code for each one.
- So once you’ve designed your UI, are you done? Absolutely not!
- The nature of mobile means you can’t expect your users to be online all the time like on a website.Batching -> battery lifeRetrying, caching, default valuesWhat happens when in a test, then no connection next time the app starts?
- These apps are compiled and static, andit’s slow to push updates. Yet to the user, it’s important that they appear dynamic.2 weeks + 1 weekessential to put a process in place that allows you to iterate quickly and make changes on the fly
- Different from web (CTR + conversions)Complex apps-> complex funnels and flow diagrams.Engagement, retention, and virality are crucial.Compare all of these things across variants so you can get the entire story.
- Every mobile app wants to have some kind of conversion, but it’s hard to convert on mobile. For example, signup forms need to be simpler, perhaps using Facebook connect instead of a form. People don’t like spending time submitting personal information, perhaps utilize something like click-to-call. If you’re cross platform, it may be easier to collect credit card information on the web for usage in a mobile app.After you’ve gotten a conversion, you typically need to follow up because you haven’t gotten everything from the user you want. For example, you converted them to a user but haven’t gotten them to pay yet.Free to play games is a perfect example. They make it really easy to get into the game and start playing, but they don’t leave you alone until you pay. These kind of methodologies really help reduce the friction inherent on mobile.
- Finally, the mobile space is extremely competitive.You’ll be competing against other companies who are optimizing every day.It’s important to ship and iterate quickly. If you can take advantage of an existing service that provides you with a way to optimize, then you don’t need to spend time building and maintaining that yourself, or hiring data scientists to make sense of everything.
- Flexible – think about what you want to optimize, can you model it with their system, and can their analytics let you compare whichever metrics you want across your variants and whatever segments you may come up with.And finally, pay attention to how tamper-proof their system is. If you’re testing prices or gameplay, you could be giving a way for your players to cheat, if they have a way to intercept or modify game data. And what happens on jailbroken devices – do you exclude them from A/B tests?
- So hopefully, that gives you a good sense of the differences between optimizing for the web and for mobile, and how to choose a commercial A/B testing solution. We’re offering an early adopter program at Leanplum, where we’re accepting developers to join our beta. We can help out with integration and provide significant attention to help with whatever your use case is. If you’re interested, you can reach us at founders@leanplum.com or check out our website. Thanks.