Optimizely X Seminar Amsterdam Nov 10

8. Evolving from A/B Testing Tool to Experimentation Platform Web 2010 Mobile 2014 Personalization 2015 Full Stack 2016 Experimentation as a Service 2017

9. For Marketers Experimentation Platform For Developers In House

10. Optimizely X Web Experimentation Deliver great web experiences on every device Optimizely X Web Personalization Deliver targeted content and experiences in real time Optimizely X Web Recommendations Deliver recommended products and content Optimizely X Full Stack Democratizing the power of deep experimentation using Python, Java, Ruby, Node and more Optimizely X Mobile Deliver great mobile app experiences Optimizely X OTT Deliver great over-the-top TV app experiences

11. Optimizely X Web Experimentation Deliver great web experiences on every device Optimizely X Web Personalization Deliver targeted content and experiences in real time Optimizely X Web Recommendations Deliver recommended products and content Optimizely X Full Stack Democratizing the power of deep experimentation using Python, Java, Ruby, Node and more Optimizely X Mobile Deliver great mobile app experiences Optimizely X OTT Deliver great over-the-top TV app experiences

12. Powerful visual editing with Desktop App Built for the modern web Real-time results, powered by Stats Engine Deliver great web experiences on every device app.optimizely.com Optimizely X Experimentation

13. Behavioral targeting Incorporate 1st and 3rd party data Real-time experimentation Deliver targeted content and experiences in real time Optimizely X Personalization

14. Automated ROI Leverages machine learning Configurable Deliver recommended products and content app.optimizely.com Optimizely X Recommendations

15. Optimizely X Web Demo

16. Optimizely X Web Experimentation Deliver great web experiences on every device Optimizely X Web Personalization Deliver targeted content and experiences in real time Optimizely X Web Recommendations Deliver recommended products and content Optimizely X Full Stack Democratizing the power of deep experimentation using Python, Java, Ruby, Node and more Optimizely X Mobile Deliver great mobile app experiences Optimizely X OTT Deliver great over-the-top TV app experiences

17. Enables experimentation across stack Python, Java, Ruby, Node SDKs Easy for developers Democratizing the power of deep experimentation Optimizely X Full Stack

18. An experimentation platform you can trust Open platform It’s your data, where you want it. Open-source SDKs with pluggable components give you control over network activity and integrating with your own analytics pipeline. Fast decision engine Our SDKs bucket users in memory so experiments have no impact on latency. Scalable architecture Our real-time event collection servers used by thousands of customers across the globe allow you to get experiment results immediately.

19. Key Optimizely X Full Stack Use Cases

20. Optimizely X Full Stack Demo

21. Optimizely X Web Experimentation Deliver great web experiences on every device Optimizely X Web Personalization Deliver targeted content and experiences in real time Optimizely X Web Recommendations Deliver recommended products and content Optimizely X Full Stack Democratizing the power of deep experimentation using Python, Java, Ruby, Node and more Optimizely X Mobile Deliver great mobile app experiences Optimizely X OTT Deliver great over-the-top TV app experiences

22. iOS and Android SDKs Feature rollouts without App Store review Enables omnichannel experimentation Deliver great mobile app experiences app.optimizley.co mOptimizely X Mobile

23. Deliver great over-the-top TV app experiences Optimizely X OTT tvOS and Android TV SDKs Enables brave experimentation Extends the omnichannel experience

24.

25. Compliance roadmap

26. Customer Showcase: Experimentation @ TOMTOM

27. CASE STUDY – Map-shop Dave Powell ecom conversion manager

28. CASE STUDY – Map-shop • Traffic? • TomTom.com Map Product Page • Gut Feeling + Hypothesis • The Tests • Results • Questions

29. Traffic? MAP OF UNITED KINGDOM & REPUBLIC OF IRELAND MAP OF FRANCE MAP OF WESTERN EUROPE Testing on the map detail pages of our highest traffic volume/visitor numbers More traffic = quicker, more meaningful results

30. Control TomTom.co m Map product page

31. A Strong Hypothesis Dan Ariely - a subscription to Economist magazine: Offer A: Internet-only subscription for $59 Offer B: Print-only subscription for $125 Offer C: Print-and-Internet subscription for $125

32. If we test a seemingly irrelevant alternative (during a campaign) with an anchored layout hierarchy Then adds to basket of a higher price point will increase Due to the decoy/uglier sibling effect A Strong Hypothesis

33. Variation #1 TomTom.co m The Tests

34. Variation #2 TomTom.co m The Tests

35. Control TomTom.co m The Tests

36. Which was the winner?

37. Variation #1 TomTom.com Results

38. Variation #1 TomTom.co m Results

39. Variation #1

40. Variation #1

41.

42. Questions? magicslipper magicslipperdave.powell@tomtom.com

43. Be Brave and Experiment

44. Appendix