Presentation given on Thursday, March 23, 2017 at MobileTea meetup in Rome, Italy.

1. Effectively Monitoring
Client-Side Performance
MobileTea, Rome | @AndrewRota

2. 1. Client-Side Performance
2. Collect + Monitor
3. Effectively

3. Andrew Rota
Software Engineer, .
@AndrewRota .
Boston, MA

4. Web Performance

16. Client-Side Web
Performance

17. Measuring
Performance

18. Navigation
Render
Additional
Resources
Interaction

19. Navigation
Render
Additional
Resources
Interaction

20. Navigation
Render
Additional
Resources
Interaction

21. Navigation
Render
Additional
Resources
Interaction

22. There’s no single metric
for measuring web
performance

23. Performance Metric Sources
1. Observational data
2. Browser devtools
3. Browser reported metrics

24. Observational Data

25. Performance Metric Sources
1. Observational data
2. Browser devtools
3. Browser reported metrics

26. Browser Profiling Metrics

27. Performance Metric Sources
1. Observational data
2. Browser devtools
3. Browser reported metrics

28. Performance Metric Sources
1. Observational data
2. Browser devtools
3. Browser reported metrics
• Navigation Timing API
• Paint metrics
• Custom Metrics

29. Navigation Timing API

30. navigationStart
responseStart
responseEnd
domInteractive
domContentLoaded
domComplete

31. Paint Metrics
window.performance.timing.msFirstPaint
window.chrome.loadTimes().firstPaintTime
Internet Explorer
Chrome

32. firstPaint

33. Custom Metrics
window.performance.now()

34. Custom Metrics
window.performance.mark(‘mainImageLoaded’)
window.performance
.getEntriesByName(‘mainImageLoaded’)[0]
.startTime

35. mainImageLoaded
tableSorted

36. Making Metrics
Meaningful

38. Traditional Site Visit
Lifecycle

39. Page
1
Page
2
Page
3
Page
4

40. But what about single
page applications (SPAs)?

41. Page
1
Page
2
Page
3
Page
4

42. SPAs can improve
performance, but complicate
performance metrics

43. With SPAs,
1) instrument router actions, and
2) know that metrics won’t be 1:1.

44. Determine meaningful abstract metrics
on top of low-level browser/app metrics

45. Can we automate, collect,
and monitor these metrics?

46. 1. Client-Side Performance
2. Collect + Monitor
3. Effectively

47. 1. Real User Monitoring (RUM)
2. Synthetic
Collecting Performance Metrics

48. RUM Performance Metrics
Users Collect Metrics

49. Sources for RUM Metrics
1. Observational data
2. Browser devtools
3. Browser reported metrics
• Navigation Timing API
• Paint metrics
• Custom Metrics

50. Tools for RUM Metrics

51. Browser reported
metrics
Collect Metrics
Send Metrics
(XHR)

52. Tools for RUM Metrics
• Custom JavaScript
• Open source JS libraries
• github.com/lognormal/boomerang
• Third party monitoring services

53. Synthetic Performance
Metrics
Automated Test Collect Metrics

54. Sources for Synthetic Metrics
1. Observational data
2. Browser devtools
3. Browser reported metrics
• Navigation Timing API
• Paint metrics
• Custom Metrics

55. Tools for Synthetic
Performance Metrics
Scheduler Browser + Test Runner

56. Schedulers
• Cron
> 0 * * * * /path/to/script.sh

57. Schedulers
• Cron
• Continuous integration server
• Jenkins
> 0 * * * * /path/to/script.sh

58. Synthetic test runners
• Phantomas
• Browser-Perf
• WebPagetest
• Lighthouse

59. Phantomas
Headless Browsers

60. Phantomas
> phantomas http://reddit.com
{
"metrics": {
"requests": 73,
"bodySize": 580491,
"contentLength": 629382,
"timeToFirstByte": 94,
"timeToLastByte": 105,
"DOMqueries": 247,
"firstPaint": 0,
"domInteractive": 215,
"domContentLoaded": 213,
"domContentLoadedEnd": 355,
"domComplete": 792,
"httpsRequests": [
"https://www.reddit.com/",
"https://www.redditstatic.com/reddit-init.en.qq9nT0inXvw.js",
"https://www.redditstatic.com/reddit.en.xlHlxcJziiU.js",
"https://www.redditstatic.com/reddit.ooBJpEoO9vY.css",
"https://www.google-analytics.com/analytics.js"
]
// … and more!
}
}

61. Phantomas
> phantomas http://reddit.com
{
"metrics": {
"requests": 73,
"bodySize": 580491,
"contentLength": 629382,
"timeToFirstByte": 94,
"timeToLastByte": 105,
"DOMqueries": 247,
"firstPaint": 0,
"domInteractive": 215,
"domContentLoaded": 213,
"domContentLoadedEnd": 355,
"domComplete": 792,
"httpsRequests": [
"https://www.reddit.com/",
"https://www.redditstatic.com/reddit-init.en.qq9nT0inXvw.js",
"https://www.redditstatic.com/reddit.en.xlHlxcJziiU.js",
"https://www.redditstatic.com/reddit.ooBJpEoO9vY.css",
"https://www.google-analytics.com/analytics.js"
]
// … and more!
}
}

62. Phantomas
> phantomas http://reddit.com
{
"metrics": {
"requests": 73,
"bodySize": 580491,
"contentLength": 629382,
"timeToFirstByte": 94,
"timeToLastByte": 105,
"DOMqueries": 247,
"firstPaint": 0,
"domInteractive": 215,
"domContentLoaded": 213,
"domContentLoadedEnd": 355,
"domComplete": 792,
"httpsRequests": [
"https://www.reddit.com/",
"https://www.redditstatic.com/reddit-init.en.qq9nT0inXvw.js",
"https://www.redditstatic.com/reddit.en.xlHlxcJziiU.js",
"https://www.redditstatic.com/reddit.ooBJpEoO9vY.css",
"https://www.google-analytics.com/analytics.js"
]
// … and more!
}
}

63. Phantomas
> phantomas http://reddit.com
{
"metrics": {
"requests": 73,
"bodySize": 580491,
"contentLength": 629382,
"timeToFirstByte": 94,
"timeToLastByte": 105,
"DOMqueries": 247,
"firstPaint": 0,
"domInteractive": 215,
"domContentLoaded": 213,
"domContentLoadedEnd": 355,
"domComplete": 792,
"httpsRequests": [
"https://www.reddit.com/",
"https://www.redditstatic.com/reddit-init.en.qq9nT0inXvw.js",
"https://www.redditstatic.com/reddit.en.xlHlxcJziiU.js",
"https://www.redditstatic.com/reddit.ooBJpEoO9vY.css",
"https://www.google-analytics.com/analytics.js"
]
// … and more!
}
}

64. Browser-Perf

65. Browser-Perf
> browser-perf http://reddit.com
--selenium=selenium.example.com

66. Browser-Perf
> browser-perf http://reddit.com
--selenium=selenium.example.com

67. Browser-Perf
> browser-perf http://reddit.com
--selenium=selenium.example.com

68. Browser-Perf
> browser-perf http://reddit.com
--selenium=selenium.example.com

69. WebPagetest
http://www.webpagetest.org/

70. WebPagetest
http://www.webpagetest.org/

71. WebPagetest
Can run on mobile devices

72. WPT: Speed Index
Paint Time of “Above the Fold” Content
( )
Source: WebPagetest

73. Calculating Speed Index
1. Take the duration until the page is visually
complete, above the fold
2. Separate it into 100ms intervals
3. For each interval, assign it a “percent
visually complete”
4. Invert that percentage so it’s “percent
incomplete” or “percent remaining”
5. Multiply that by the interval length (100ms)
6. Sum all of the intervals. Speed Index!
Mobile viewport size
will affect this metric!

74. Lighthouse

76. Lighthouse

77. Lighthouse + WPT?!
https://github.com/WPO-Foundation/webpagetest/pull/825

78. Storage and Monitoring

79. • Aggregation
• Data storage + queries
• Visualization
• Alerts + Notification
Storing and Monitoring

80. Aggregation via StatsD

81. Storage + Queries with Graphite
mySite.homepage.phantomas.domComplete

82. Visualization with Grafana

83. Visualization with Grafana
Source: play.grafana.org/

84. 1. Client-Side Performance
2. Collect + Monitor
3. Effectively

85. Effective Monitoring
• Reduce noise
• Be aware of environment differences
• Beware skewed data
• Make data actionable
• Utilize data

86. Reduce Noise
• Isolate third party scripts and
external services
• Control conditional code paths (A/B
tests, etc.)

87. Be aware of environment
differences
• Especially in mobile
• Bandwidth differences
• CPU/memory differences

88. Pyramid of Environment Parity
Mock user agent
Throttle network connection, CPU
Device simulator
Real device

89. Pyramid of Environment Parity
Mock user agent
Throttle network connection, CPU
Device simulator
Real device
Accuracy
Ease/Speed

90. Beware Skewed Data
• Observer effect
• Noisy neighbors

91. Make Data Actionable
Know what changes will
affect each of your metrics

92. Do something with this data!

93. Utilize Data
• Agree on performance metrics
• Agree on performance goals
• Allocate time and resources
to achieving these goals

94. Performance is a means
to an end:
Remember

95. Performance is a means
to an end:
user experience
Remember

96. Andrew Rota, .
@AndrewRota .
Thanks!
Recommend Resources
• bit.ly/google_rails
• bit.ly/webpagetest_docs
• bit.ly/browser_perf
• bit.ly/statsdcc
• bit.ly/phantomas_js
• graphite.readthedocs.io