Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.

Metrics, metrics everywhere (but where the heck do you start?)

3.162 visualizaciones

Publicado el

There’s no one-size-fits-all approach to metrics. In this session, Cliff Crocker and I walk through various metrics that answer performance questions from multiple perspectives — from designer and DevOps to CRO and CEO. You’ll walk away with a better understanding of your options, as well as a clear understanding of how to choose the right metric for the right audience.

Publicado en: Internet
  • Sé el primero en comentar

Metrics, metrics everywhere (but where the heck do you start?)

  1. 1. Metrics, metrics everywhere (but where the heck do you start?)
  2. 2. @tameverts @cliffcrocker #velocityconf
  3. 3. Who cares about performance today? How do I measure performance? How fast am I? How fast should I be? How do I get there?
  4. 4. The myth of a single metric
  5. 5. Start render DNS TCP TTFB DOM loading DOM ready Page load Fully loaded User timing Resource timing Requests Bytes in Speed Index Pagespeed score 1s = $$ DOM elements DOM size Visually complete Redirect SSL negotiation
  6. 6. Who cares about performance?
  7. 7. “47% of consumers expect a web page to load in 2 seconds or less.” Akamai, 2009
  8. 8. 1s = $27M DNS 144ms Start render 1.59s Hero image render 2.01s
  9. 9. How do I measure performance?
  10. 10. Androiddevicefragmentation OpenSignal,August2015
  11. 11. How do we measure UX for mobile?
  12. 12. https://gist.github.com/larahogan/b681da601e3c94fdd3a6
  13. 13. RUM versus plus synthetic
  14. 14. Real User Monitoring 101
  15. 15. Technology for collecting performance metrics directly from the end user’s browser Involves instrumenting your site via JavaScript Measurements are fired across the network to a collection point through a small request object (beacon)
  16. 16. What makes RUM great  Always on  Every user, every browser, everywhere  Able to capture human behavior/events  Only getting better
  17. 17. Questions your RUM data can answer
  18. 18. What are my users’ environments?
  19. 19. How do visitors move through my site?
  20. 20. How are my third-party scripts performing in real time?
  21. 21. What impact does performance have on my business?
  22. 22. Synthetic Monitoring 101
  23. 23. Uses automated agents (bots) to measure your website from different physical locations A set “path” or URL is defined Tests are run either ad hoc or scheduled, and data is collected
  24. 24. What makes synthetic monitoring great  Rich data collected (waterfall, video, filmstrip, HTTP headers)  Consistent “clean room” baseline  Nothing to install  Doesn’t require users / ability to measure pre-production and competition
  25. 25. Questions your synthetic data can answer
  26. 26. How do I compare to the competition?
  27. 27. How does the design of my pages affect performance?
  28. 28. How does the newest version of my site compare to previous versions?
  29. 29. How well am I sticking to my performance budget?
  30. 30. What if my third parties fail? Original: 3.5s SPOF: 22.7s
  31. 31. 38© 2014 SOASTA CONFIDENTIAL - All rights reserved. Why are these numbers so different? I don’t trust your data. Your numbers are wrong. How are you calculating page load time? I can’t share two sets of numbers with the business.
  32. 32. “But it loads so much faster for me!?!” 2015 Macbook Pro Warm browser cache FIOS X86 – Windows 7 VM Completely cold cache/DNS Throttled bandwidth
  33. 33. boomerang.js Episodes
  34. 34. W3C Performance Working Group
  35. 35. How fast am I?
  36. 36. http://soasta.io/perftimings
  37. 37. NavigationTiming API
  38. 38. Browser support for NavigationTiming http://caniuse.com/#feat=nav-timing
  39. 39. 48© 2014 SOASTA CONFIDENTIAL - All rights reserved. Network operations Front-end developer Marketing and site operations Designer C-level
  40. 40. Use case: Measure network performance
  41. 41. I need visibility into…  issues with authoritative DNS servers  impact of denial of service attacks on end users  efficiency of connection re-use  tier 1 connectivity issues (load balancer, web server)
  42. 42. Start render DNS TCP TTFB DOM loading DOM ready Page load Fully loaded User timing Resource timing Requests Bytes in Speed Index Pagespeed score 1s = $$ DOM elements DOM size Visually complete Redirect SSL negotiation
  43. 43. Measuring DNS and TCP function getPerf() { var timing = window.performance.timing; return { dns: timing.domainLookupEnd - timing.domainLookupStart}; connect: timing.connectEnd - timing.connectStart}; }
  44. 44. What’s with all those zeros!  Connection reuse  DNS caching  Prefetching
  45. 45. Focus on higher percentiles 85th percentile Median (50th)
  46. 46. Use case: Measure front-end browser events
  47. 47. How do I…  understand the impact of back-end versus front-end on page speed?  investigate how DOM complexity affects performance?  measure a “fully loaded” page?
  48. 48. Start render DNS TCP TTFB DOM load event DOM ready Page load Fully loaded User timing Resource timing Requests Bytes in Speed Index Pagespeed score 1s = $$ DOM elements DOM size Visually complete Redirect SSL negotiation
  49. 49. Isolate front-end vs. back-end
  50. 50. Isolate front-end vs. back-end function getPerf() { var timing = window.performance.timing; return { ttfb: timing.responseStart - timing.connectEnd}; basePage: timing.responseEnd - timing.responseStart}; frontEnd: timing.loadEventStart - timing.responseEnd}; }
  51. 51. Front-end Back-end
  52. 52. Measuring the critical rendering path
  53. 53. Investigate DOM events function getPerf() { var timing = window.performance.timing; return { DLoading: timing.domLoading – timing.navigationStart}; DInt: timing.domInteractive – timing.navigationStart}; DContLoaded: timing.domContentLoadedEventEnd – timing.navigationStart; DContLoadTime: timing.domContentLoadedEventEnd – timing.domContentLoadedEventStart}; DComplete: timing.domComplete - timing.navigationStart}; PLoad: timing.loadEventStart -
  54. 54. Understanding critical rendering path  DOM Loading – browser begins parsing initial bytes of the document  DOM Interactive – doc parsed, DOM has been constructed  DOM Content Loaded – No blocking style sheets  DOM Complete – All processing complete, all assets downloaded https://developers.google.com/web/fundamentals/performance/critical-rendering-path
  55. 55. 2618 DOM nodes 86 DOM nodes Visualizing DOM complexity
  56. 56. Use case: Measure object-level performance
  57. 57. I need to understand…  how third-party content affects my performance  how long a group of assets takes to download  how assets served by a CDN are performing
  58. 58. Start render DNS TCP TTFB DOM loading DOM ready Page load Fully loaded User timing Resource timing Requests Bytes in Speed Index Pagespeed score 1s = $$ DOM elements DOM size Visually complete Redirect SSL negotiation
  59. 59. ResourceTiming interface
  60. 60. Browser support for Resource Timing http://caniuse.com/#feat=resource-timing
  61. 61. Cross-Origin Resource Sharing (CORS) Start/End time only unless Timing-Allow-Origin HTTP Response Header defined Timing-Allow-Origin = "Timing-Allow-Origin" ":" origin-list- or-null | "*"
  62. 62. ResourceTiming var rUrl = ‘http://www.akamai.com/images/img/cliff-crocker- dualtone-150x150.png’; var me = performance.getEntriesByName(rUrl)[0]; var timings = { loadTime: me.duration, dns: me.domainLookupEnd - me.domainLookupStart, tcp: me.connectEnd - me.connectStart, waiting: me.responseStart - me.requestStart, fetch: me.responseEnd - me.responseStart } Measuring a single resource
  63. 63. Other uses for ResourceTiming  Slowest resources  Time to first image (download)  Response time by domain  Time a group of assets  Response time by initiator type (element type)  Cache-hit ratio for resources For examples see: https://github.com/lognormal/beyond-page-metrics
  64. 64. Using Resource Groups PLT impact not due resource groups PLT impact correlates with improvement from image domains
  65. 65. Use case: Measure the user experience
  66. 66. I just need to understand…  when users perceive the page to be ready  how long until my page begins to render  when content above the fold is visible
  67. 67. Start render DNS TCP TTFB DOM loading DOM ready Page load Fully loaded User timing Resource timing Requests Bytes in Speed Index Pagespeed score 1s = $$ DOM elements DOM size Visually complete Redirect SSL negotiation
  68. 68. The fallacy of “First Paint” in the wild  Support for First Paint is not standardized between browsers  Metric can be misleading (i.e. painting a white screen)
  69. 69. First Paint is not equal to Start Render! Chrome – “First Paint” True Start Render
  70. 70. Start Render and filmstrips
  71. 71. UserTiming Interface  Allows developers to measure performance of their applications through high-precision timestamps  Consists of “marks” and “measures”  PerformanceMark: Timestamp  PerformanceMeasure: Duration between two given marks
  72. 72. Browser support for UserTiming http://caniuse.com/#feat=user-timing
  73. 73. Measure duration between two marks performance.mark(“startTask”); //Some stuff you want to measure happens here performance.mark(“stopTask”); //Measure the duration between the two marks performance.measure(“taskDuration”,“startTask”,“stopTask”);
  74. 74. How long does it take to display the main product image on my site?
  75. 75. Record when an img loads <img src=“image1.gif” onload=“performance.mark(‘image1’)”> For more interesting examples, see: Measure hero image delay http://www.stevesouders.com/blog/2015/05/12/hero-image-custom-metrics/ Measure aggregate times to get an “above fold time” http://blog.patrickmeenan.com/2013/07/measuring-performance-of-user- experience.html
  76. 76. Adoption of UserTiming for top 25 shopping sites (Global) Nope User Timing Sales
  77. 77. http://soasta.io/perftimings
  78. 78. How do I measure performance when the onload event loses relevance? Use case: Measure performance of SPAs
  79. 79. http://www.ryan-williams.net/hacker-news-hiring-trends/2015/october.html
  80. 80. What is a SPA? • Single Page Application • HTML loads once • Fluid user experience • Can be great for performance • Hard to measure
  81. 81. onload event visible resources
  82. 82. Measuring SPAs • Accept the fact that onload no longer matters • Tie into routing events of SPA framework • Measure downloads during soft refreshes • (support in boomerang.js for Angular and other SPA frameworks)
  83. 83. How fast should I be?
  84. 84. Use case: Measure business impact
  85. 85. I need to understand…  how performance affects business KPIs  how our site compares to our competitors
  86. 86. Start render DNS TCP TTFB DOM loading DOM ready Page load Fully loaded User timing Resource timing Requests Bytes in Speed Index Pagespeed score 1s = $$ DOM elements DOM size Visually complete Redirect SSL negotiation
  87. 87. http://www.soasta.com/blog/mobile-web-performance-monitoring-conversion-rate/
  88. 88. 2% increase in conversions for every 1 second of improvement
  89. 89. Cut load time by 80% 83% traffic increase 32% increase in time on site 108% increase in interaction rate with ads
  90. 90. So what? You must look at your own data.
  91. 91. Not all pages are created equal For a typical ecommerce site, conversion rate drops by up to 50% when “browse” pages increase from 1 to 6 seconds
  92. 92. Not all pages are created equal However, there is much less impact to conversion when “checkout” pages degrade
  93. 93. What is the Conversion Impact Score? The Conversion Impact Score (CIS) is a relative score that ranks page groups by their propensity to negatively impact conversions due to high load times. For each page group, the Conversion Impact Score is calculated using the proportion of overall requests that are associated with that group, along with the Spearman Ranked Correlation between its load times and number of conversions. The Conversion Impact Score will always be a number between -1 and 1, though scores much greater than zero should be very rare. The more negative the score, the more detrimental to conversions that high load times for that page group are, relative to the other page groups.
  94. 94. TL;DR The Conversion Impact Score answers this question: How much impact does the performance of this page have on conversions?
  95. 95. Conversion Impact Score
  96. 96. http://www.soasta.com/blog/website-monitoring-conversion-impact-score/
  97. 97. How do I get there?
  98. 98. Create a performance budget Setting a Performance Budget http://timkadlec.com/2013/01/setting-a-performance-budget/ Performance Budget Metrics http://timkadlec.com/2014/11/performance-budget-metrics/
  99. 99. Set meaningful, page-specific SLAs
  100. 100. Chapter 8: Changing Culture at Your Organization
  101. 101. Free download (until November 4) http://soasta.io/time ismoneybook
  102. 102. performancebacon.com performancebeacon.com
  103. 103. Thanks!
  104. 104. Meet us at booth #51

×