Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Optimizely Product Vision: The Future of Experimentation

Join Optimizely's product team to hear features and releases planned for 2019 and beyond.

  • Sé el primero en comentar

Optimizely Product Vision: The Future of Experimentation

  1. 1. 1 Optimizely’s Product Vision The Future of Experimentation Claire Vo SVP Product Jon Noronha Sr. Director, Product Byron Jones Director, Product Giannis Psaroudakis Director, Product
  2. 2. 2 Giannis Psaroudakis Director, Product Management Claire Vo Senior Vice President, Product Management Byron Jones Director, Product Management Jon Noronha Senior Director, Product Management Introductions
  3. 3. 3 Agenda 1. Introductions 2. Optimizely’s Vision & Priorities 3. Creating Universal Experiences 4. Embedding Trusted Results 5. Leveraging Enterprise Scale 6. Q&A
  4. 4. 4 Housekeeping ● We are recording this webinar ● You’ll receive the recording and slides ● We’ll answer questions at the end
  5. 5. 5 “Our success is a function of how many experiments we do per year, per month, per week, per day.” - Jeff Bezos “Our aim is to create the best product for our customers, and we do that through constant innovation and testing.” - Gillan Tans, CEO “Our company culture encourages experimentation and free flow of ideas.” - Larry Page “We use experimentation and testing to inform as much of the business as we possibly can.” - Gregory Peters, CPO Today’s Digital Leaders Win By Using Experimentation At-Scale
  6. 6. Spent with Digital Media 19hr week 2018 Digital-Centric Customer Spent with Digital Media 41 $396m Mobile commerce Mobile commerce $1 8t. 2008 Digital-Convenient Customer hr week Your customer has fundamentally changed how they engage
  7. 7. The digital-centric customer journey: end-to- end, multi-device engagement Marketing Product & Engineering Support Views ad at Checkout Signs up for 7 day trial Tracks packages Manages account Converts to subscriber Downloads App
  8. 8. 8 How does Optimizely build products?
  9. 9. 9 Device coverage Complete experience coverage Open platform Trusted results quickly Consumer-grade usability Security & compliance Program Management Strategic & tactical partnership Build Digital Products Market & Sell Service & Retain What you need OTT Voice/IOT Mobile Web / SPA Build Digital Products Market & Sell Service & Retain DEVICE COVERAGE
  10. 10. 10 Device coverage Complete experience coverage Open platform Trusted results quickly Consumer-grade usability Security & compliance Program Management Strategic & tactical partnership Build Digital Products Market & Sell Service & Retain What you need OTT Voice/IOT Mobile Web / SPA Build Digital Products Market & Sell Service & Retain Frontend and Server-side Experimentation & Personalization EXPERIENCE COVERAGEDEVICE COVERAGE
  11. 11. 11 Device coverage Complete experience coverage Open platform Trusted results quickly Consumer-grade usability Security & compliance Program Management Strategic & tactical partnership Build Digital Products Market & Sell Service & Retain What you need Build Digital Products Market & Sell Service & Retain TRUSTED RESULTS Open Platform Data & Integrations Security & Compliance OTT Voice/IOT Mobile Web / SPA Frontend and Server-side Experimentation & Personalization EXPERIENCE COVERAGEDEVICE COVERAGE
  12. 12. 12 Device coverage Complete experience coverage Open platform Trusted results quickly Consumer-grade usability Security & compliance Program Management Strategic & tactical partnership Build Digital Products Market & Sell Service & Retain What you need Build Digital Products Market & Sell Service & Retain ENTERPRISE SCALE Consumer grade usability for adoption & velocity Program Management Open Platform Data & Integrations Security & Compliance OTT Voice/IOT Mobile Web / SPA Frontend and Server-side Experimentation & Personalization TRUSTED RESULTSEXPERIENCE COVERAGEDEVICE COVERAGE
  13. 13. 13 Device coverage Complete experience coverage Open platform Trusted results quickly Consumer-grade usability Security & compliance Program Management Strategic & tactical partnership Build Digital Products Market & Sell Service & Retain What you need Build Digital Products Market & Sell Service & Retain Consumer grade usability for adoption & velocity Program Management Open Platform Data & Integrations Security & Compliance OTT Voice/IOT Mobile Web / SPA Frontend and Server-side Experimentation & Personalization ENTERPRISE SCALETRUSTED RESULTSEXPERIENCE COVERAGEDEVICE COVERAGE COLLECTDATA UNIVERSAL EXPERIENCES ANALYTICS ENTERPRISE SCALE
  14. 14. UNIVERSAL EXPERIENCES Test and personalize every digital moment across all channels. ANALYTICS ENTERPRISE SCALE
  15. 15. DEVICE & EXPERIENCE COVERAGE Optimizely Web & Full Stack FULL STACK. Test server-side logic & data to deploy confidently WEB. Experiment on & deliver a winning front-end experience Copy Images & Colors Layout i.e. booking flow, search bar location & visual treatment CTA Navigation Pricing or discounts Personalized content based on user Segment offers based on persona Search algorithms New feature rollouts
  16. 16. DEVICE & EXPERIENCE COVERAGE Universal Experiences Cross-Platform Metrics Share conversions across frontend and backend with universal attribution Web Performance Build experiments in a visual editor, deliver them with lower latency Consistent Features Use Environments in Web projects. Track Change History in Full Stack. Full Stack Usability Do more out-of-the-box with Full Stack. Easier SDKs running in more places.
  17. 17. WEB + FULL STACK Cross-Platform Metrics ● Reuse client-side event tracking with server-side tests ● Measure how web experiments impact downstream backend metrics ● Share event tracking across multiple Full Stack projects Share conversion events across Web and Full Stack experiments Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Coming Soon
  18. 18. CROSS-PLATFORM METRICS Native Mobile Experiment: Homes vs. Experiences
  19. 19. CROSS-PLATFORM METRICS Web Conversion: Booked Room Attribution Web conversion counts back Toward mobile experiment
  20. 20. OPTIMIZELY WEB Client-Side Experimentation Client Web Snippet Visual Editor Default Treatment Server Experimental Treatment Experimental Treatment Client Server Full Stack SDK Feature Flags in Code User User OPTIMIZELY FULL STACK Server-Side Experimentation Experimental Treatment
  21. 21. Experimental Treatment Client Server User WEB VELOCITY + FULL STACK PERFORMANCE Client + Server-side Experimentation Snippet Visual Editor SDK Feature Flags in Code Smooth handoff from Full Stack to Web
  22. 22. WEB Performance Innovation ● Optimizing key platform components for faster desktop & mobile execution ● Integrated new CDN delivery technology for lower latency connections, using Full Stack technology ● New implementation options to give you full control over your site’s performance We’re making Optimizely faster than ever for the modern web Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Coming Soon
  23. 23. ● Build an experiment once, and push through specialized environments ● QA experiments in dedicated environments before launching to production ● Streamline developer work by setting up environments in Optimizely that mirror your own development workflows Easily integrate experimentation into your existing development & QA workflows FULL STACK → WEB Web Environments Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. DEV TEST STAGE PROD Coming Soon
  24. 24. WEB → FULL STACK Change History ● Comprehensive records: see exactly who made a change, what they edited, and when ● Detailed audit log lets you investigate what changed at a specific time ● Improved search and filtering Track all changes made to experiments and feature flags in your account Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Coming Soon
  25. 25. 25 The right experience is not always the same for everyone The right experience depends on context The right experience requires continuous iteration Experimentation Personalization Sweet Spot Why You Need to Experiment to Personalize Effectively
  26. 26. 26 Different experiences resonate with different individuals, and personalization enables you to ensure you find an experience that resonates with your customers ✓x x Audience Students Page Homepage Everyone Banner B -2% Students Banner B +19% Students Banner C +19% Variation 1 Students Banner D +25% Variation 2 Students Banner E +10% Variation 3 Audience Everyone Page Homepage Everyone Banner A +5% Everyone Banner B -2% The right experience is not always the same for everyone
  27. 27. ● Match users attributes with new number, string, and boolean comparisons ● Easily adjust targeting conditions without re-deploying new code ● Compose multiple audiences into new segments Target experiments and feature rollouts using flexible rules and composable segments WEB → FULL STACK Flexible Targeting Released Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein.
  28. 28. ● Target users based on their predicted intent, using natural language keywords ● Adapt to changing user behavior and expectations automatically, without needing to manually tweak rules ● Experiment your way to the best personalization strategy - test larger or smaller audiences, and multiple variations Build predictive segments based on online behavior, powered by machine learning WEB Adaptive Audiences Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Beta
  29. 29. BUILD LEARN MEASURE WEB + FULL STACK Product Experimentation Think hypothetically ● Painted doors ● Validating MVPs ● Qualitative research ● Exploratory A/B testing Quantify product impact ● Feature validation ● Multivariate tests ● Iterative adjustments ● Measuring business impact Mitigate development risk ● Frequent, small releases ● Feature flagging ● Staged rollouts ● Safe rollbacks
  30. 30. FULL STACK Optimizely Rollouts Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Early Access ● Reduce risk with staged feature rollouts and safe rollbacks ● Powered by the same SDKs as Full Stack. Grow into full experimentation. ● Generally available next month. Contact your CSM if you’re interested in early access! www.optimizely.com/rollouts Free feature flagging for product development teams
  31. 31. FULL STACK Developer Efficiency Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Coming Soon ● New languages: Swift, Go ● Datafile Management: automatically sync configuration between Optimizely and your application ● Event Dispatching: simplify event tracking at enterprise scale ● User Profiles: manage user state and variation history across devices ● New Core: easy access to your configuration data, improved hooks for QA and testing, more detailed log messages Do more, faster. Our redesigned SDKs are more powerful and easier to use
  32. 32. API Gateway Go SDK Ruby SDK JS SDK React SDK Rest API Rest API Adapter 1 Rest API Rest API Rest API Rest APIJava SDK Adapter 2 Adapter 3
  33. 33. OUR VISION Universal Experiences DYNAMIC PRICING, ALGORITHMS ON OUR WAY Your order has just shipped! Track it now. NOTIFICATIONS, CONVERSATIONAL INTERFACES ADVERTISING Bringing experimentation and personalization to the full spectrum of digital experiences
  34. 34. UNIVERSAL EXPERIENCES ANALYTICS Make trusted decisions at scale with your experiment data ENTERPRISE SCALE
  35. 35. 35 MISSION Help the world’s innovators make trusted decisions at scale Experimentation Hero 10’s experiments / year Experimentation Program 100’s experiments / year Culture of Experimentation 1000’s experiments / year
  36. 36. ANALYTICS Optimizely Analytics Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Results & Stats Engine 1. Optimize your experiences and drive business value. 2. Learn how users respond to your experiment. 3. Control the risk of launched features. Machine Learning 1. Accelerate learnings and impact with ML-powered Stats Accelerator. 2. Optimize rewards for a period of time. 3. Identify statistically significant variants faster. Integrations 1. Integrate with best-of-breed technology platforms. 2. Discover powerful ways to target content. 3. Measure performance in the tools you already use. TODAY Standard Experiment Experiment with Stats Accelerator Reach statistical significance up to 300% faster
  37. 37. ANALYTICS Easy Event Tracking Track and optimize every element of the customer journey with Optimizely. Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Released E E E Website Sign ups Conversions …. Payments Revenue and Refunds BI Lifetime Value ● Attribute visitor actions from any system to your experiments. ● Optimize metrics across the customer journey. ● Simplify data integrations with other applications in your technology stack.
  38. 38. ANALYTICS Easy Event Tracking | Customer Use Cases Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Released E E E E cross-channel actions offline conversions computed metric data external events Easy Event Attribution (purchased in-store) (e-mail opened) (lifetime value) (notification clicked)
  39. 39. ANALYTICS Results Export Access, analyze and visualize experiment performance with enhanced flexibility. ● Analyze experiment results with any SQL or Data Science tool. ● Enrich your analysis with data from your Data Warehouse. ● Monitor experiment performance using your own dashboards. Beta Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Reporting Data Science Data Warehouse
  40. 40. ANALYTICS Results Export | Customer Use Cases Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Beta Custom Analysis (SQL, Data Science/Mining...) Reporting Dashboards
  41. 41. ANALYTICS Event Debugger Inspect and debug your experiment events faster. ● Verify in real-time that Optimizely is receiving your events. ● Debug issues with your event setup to accelerate your experimentation velocity. Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Coming Soon
  42. 42. ANALYTICS Extensible Analytics Platform Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein. Query Data Compute Metrics Statistical Analysis Display Results Metrics Object Results Object Data Object Experiment Analysis Model External Data Custom Models Certified Reports An Experiment Analysis platform designed to be extended and certified by Data Scientists... … so that it’s used by everyone in the organization to make trusted decisions at scale Future EVENTS METRICS BUILDER STATS ENGINE RESULTS EXTENSIBLE INTERFACES
  43. 43. UNIVERSAL EXPERIENCES ANALYTICS ENTERPRISE SCALE Govern, orchestrate, and report on your experience optimization program.
  44. 44. 10X YOUR PROGRAM Enterprise Scale AUTOMATED ORCHESTRATION Use automation to accelerate teams into building high-velocity experimentation programs. CUSTOMIZED REPORTS Easier socialization of an experimentation through custom reporting & dashboards. ENTERPRISE GOVERNANCE Roles, controls, and workflows that give large teams the ability to scale with confidence. Govern, orchestrate, and report on your experience optimization program.
  45. 45. ENTERPRISE SCALE Jira Integration ● Track development in Jira and experiments from Optimizely with a unified view across systems ● See the status of Optimizely rollouts in Jira issues to reduce context switching and friction in development ● See key information about experiments and ideas right in Jira Streamline your workflow by linking Optimizely ideas, experiments and feature flags to Jira issues Released Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein.
  46. 46. ENTERPRISE SCALE Slack App ● Integrate with Slack to ensure real- time updates on progress of ideas and experiments ● Users stay informed and can engaged with experiments from outside of Optimizely ● Improve experimentation communication and collaboration and increase testing velocity Integrate Program Management with Slack, enabling more effective and efficient collaboration Released Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein.
  47. 47. Coming Soon ENTERPRISE SCALE Experiment Reports ● Easily and quickly report on experiment results ● Acts as a system of record and cohesive view of an experiment ● Helps improve prioritization and decision making over time Easily create a report summarizing key details, results, and assets for your experiments Materials presented are for demonstration purposes only and may not be developed and / or delivered. Optimizely provides no guarantees and has no liability for information contained herein.
  48. 48. 49 Program Reporting Core operational metrics like win rate and velocity across your teams. Experiment Reports Shareable summary of your experiments ROI Measure your team’s revenue impact on your business. TODAY FUTURE Make it easier to share results and derive your return on investment in experimentation ENTERPRISE SCALE Reporting
  49. 49. 50 2019 Optimizely ROADMAP WEB & ✔ Support for Dynamic Websites Adaptive Audiences PERSONALIZATION ✔ Enhanced Multivariate Testing Faster Performance Environments FULL STACK ✔ Feature Management ✔ Flexible Targeting ✔ Environments Change History ✔ Multi-language Experiments ✔ Fast Config Sync ✔ Custom Analytics Integrations PLATFORM ✔ Stats Accelerator Enhanced Export ✔ GDPR ✔ Cross Product Events ✔ Easy Event Tracking Event Debugger ✔ Flexible Segmentation PROGRAM ✔ Teams Experiment Reports MANAGEMENT ✔ Program Reporting ✔ Slack App ✔ Jira Integration ROI Reporting BETA BETA Talk to your CSM to learn more Coming Soon 2019 Delivered 2018
  50. 50. 51 Q&A
  51. 51. 52 Thank you!
  52. 52. 53 Archive
  53. 53. 58 Easy Event Tracking (Data-In) 1. Track events from any system into Optimizely. 2. Optimize metrics across the customer journey. 3. Simplify event integrations with other applications in your stack. Results Export (Data-Out) 1. Analyze results with any SQL or data science tool. 2. Enrich your analysis with data from your DW. 3. Build dashboards using tools you are familiar with. THIS YEAR This Year | Analytics Released Beta
  54. 54. 59 Easy Event Tracking (Data-In) 1. Track events from any source. 2. Optimize metrics across the customer journey. 3. Simplify event integrations with other applications in your stack. Results Export (Data-Out) 1. Analyze results with any SQL tool. 2. Enrich your analysis with external data from your data warehouse. 3. Build dashboards using tools you are familiar with. Event Debugger 1. Verify that Optimizely is receiving your events in real- time. 2. Debug issues with your events faster to accelerate your experimentation velocity. THIS YEAR Coming SoonBetaLaunched This Year | Analytics

×