This document discusses the importance of website performance and user experience. Some key points: - Studies show that users perceive website speed to be 15-20% slower than it actually is, and slowdowns have a bigger impact than speed increases. - A study of mobile shopping found that speed was the number one priority for users, with functional issues on sites like Zalando impacting perceptions. - To measure performance, companies should establish performance SLAs by defining key metrics and targets based on user tasks. They also need to collect both synthetic and real user monitoring data. - Performance analytics can show how sites compare to competitors and identify pages for optimization. Metrics like revenue risked from poor performance are also

2. @jeroentjepkema
Founder & CEO

3. Slides: bit.ly/MW-OOnfs13

4. Content
vs.
Performance

5. How we perceive performance....

6. How we perceive performance....
15%-20% worse than in reality

7. How we perceive performance....
15%-20% worse than in reality
Slow downs have more impact

8. How we perceive performance....
15%-20% worse than in reality
Slow downs have more impact
Task completion has positive impact

9. How we perceive performance....
15%-20% worse than in reality
Slow downs have more impact
Task completion has positive impact
Always compared to past experiences

10. A simple online business model:
Marketing
Conversion Optimization
New
visitors
Search
Tweets
Mentions
ADs seen
Conversion
rate
Growth
Number of
visits
Loss
Bounce
rate
Revenue
Pages per
visit
Time on
site
X
Order
value

11. Dutch consumers visit 18-23 websites
before they finally convert
Source: TNS/Nipo

12. Performance influence cycle
Search/Orientation phase
1
Perception of performance
http://twinkle100.measureworks.nl

13. Performance influence cycle
Stimulate content/
conversion
4
Search/Orientation phase
5
1
Perception of performance
http://twinkle100.measureworks.nl
Delivered
performance
3
2
Your website

14. Need
for
Speed
@jeroentjepkema, MeasureWorks
@jeroentjepkema, MeasureWorks

15. 0
Source: Jakob Nielsen

16. 1
0,3
0
Interaction: Let’s conversate...
Instantaneous: I like it!
Source: Jakob Nielsen

17. 3
1
0,3
0
Mmm, shall I click away?
Interaction: Let’s conversate...
Instantaneous: I like it!
Source: Jakob Nielsen

18. 10
Only if the task/content is relevant
3
1
0,3
0
Mmm, shall I click away?
Interaction: Let’s conversate...
Instantaneous: I like it!
Source: Jakob Nielsen

21. 0:00 sec.
Conversation
starts

22. 0:00 sec.
Conversation
starts
2.5 sec.
First reply

23. 4.5 sec.
Finish
conversation
0:00 sec.
Conversation
starts
2.5 sec.
First reply

24. 4.5 sec.
Finish
conversation
0:00 sec.
Conversation
starts
Comfort zone
Tolerated
2.5 sec.
First reply
Frustrated

25. The holy grail in experience is
“flow state”

26. “”
Flow is an
optimal experience
that is
“intrinsically enjoyable”

27. Provide relevant
content to support
task completion

28. Deliver it fast,
focus on perception

29. http://www.nytimes.com/2012/08/19/opinion/sunday/why-waiting-in-line-is-torture.html?pagewanted=all&_r=0

30. http://answers.yahoo.com/question/index?qid=1005081200005

31. That’s not my
website?

32. Over 60% of people that experienced a slow website ranked
design 2 points lower compared to a fast experience
Slow
Average
Fast
% of respondents that rated the actual website speed
100
75
50
25
0
1
2
3
Design score (1=bad - 5=beautiful)
4
5

33. Experience
the
difference
Tested with webpagetest.org (3G traffic shaping, 2000Kbps Down, 1000 Kbps Up, 150ms latency)

35. Perception
vs.
Technology

36. Desktop shopping experience setup....
Buy a summer
holiday:
100 users
‣ From 5 selected websites
‣ Rank from fastest to
slowest after task execution
No measurements tools allowed ;-)
Desktop only,
either via LAN or WiFi

37. Perceived
Technology
(User Ranking)
(Full page download)
1
Sunweb
2
D-reizen
3
Neckermann
4
Arke
5
Globe
Technology vs. Perceived
Study performed by MeasureWorks for Emerce eTravel, 2013

38. Perceived
Technology
(User Ranking)
(Full page download)
1
Sunweb
Sunweb
2
D-reizen
Globe
3
Neckermann
Neckermann
4
Arke
D-reizen
Globe
Arke
5
Technology vs. Perceived
Study performed by MeasureWorks for Emerce eTravel, 2013

39. Perceived
Technology
(User Ranking)
(Full page download)
1
Sunweb
Sunweb
2
D-reizen
Globe
3
Neckermann
Neckermann
4
Arke
D-reizen
Globe
Arke
5
Technology vs. Perceived
Study performed by MeasureWorks for Emerce eTravel, 2013

40. Globe
Neckermann
Sundio
D-reizen
Arke

41. M-commerce?

42. Mobile shopping experience setup....
100 mobile users
Perform 2 tasks:
‣Buy a book
‣Buy a T-shirt
Mobile browsers only
via 3G, no WiFi

43. #1:
Speed

45. Functional issues
reported with
Zalando
50
37,5
Round 1
25
45
42,1
12,5
15,4
0
Zalando
HM
20
7,7
V&D
24
3,9
Tom Tailor
4
Design
1. Buy a T-shirt
Speed
Mobile Readiness
Other
2. Feedback
40
30
Round 2
20
10
12
0
54
34
Zalando
Mobile Only research:
- Task completion: Only use smartphone to buy a book
- N = 100, users range from 20-65
15,4
HM
8
V&D
Tom Tailor
21
Design
18
Speed
6
Mobile Readiness
Other

46. Nexus - Android - 3G
Tested with webpagetest.org (3G traffic shaping, 2000Kbps Down, 1000 Kbps Up, 150ms latency)

47. Nexus - Android - 3G
Tested with webpagetest.org (3G traffic shaping, 2000Kbps Down, 1000 Kbps Up, 150ms latency)

48. #2:
Usability

49. Bol.com
Selexyz
De Boekerij
ECI
Bruna
Cosmox
Mobile
Readiness

50. 100
75
50
46
44
26
24
25
16
0
Bol.com Selexyz
Bruna
Other
1. Buy a book
Mobile Only research:
- Task completion: Only use smartphone to buy a book
- N = 100, users range from 20-65
Design
Speed
16
Mobile Readiness
2. Feedback bol.com
16
Other
Design
12
Speed
Mobile Readiness
Other
3. Feedback from different stores

51. Mobile
Readiness

52. -
One touch?
+

53. Login or
not?

54. Mobile
Search

55. Speed matters

56. Complete Web Monitoring

57. Web Analytics
Usability
Performance
(what did they do on
the site?)
(how did they
interact with it?)
(could they do what
they wanted to?)
Complete Web Monitoring
VoC
Social Media
Competition
(what were their
motivations?)
(what were they
saying?)
(what are they up
to?)

58. “Hard” data
Web Analytics
Usability
Performance
(what did they do on
the site?)
(how did they
interact with it?)
(could they do what
they wanted to?)
Complete Web Monitoring
VoC
Social Media
Competition
(what were their
motivations?)
(what were they
saying?)
(what are they up
to?)
“Soft” data

59. “Hard” data
Web Analytics
Usability
Performance
(what did they do on
the site?)
(how did they
interact with it?)
(could they do what
they wanted to?)
Complete Web Monitoring
VoC
Social Media
Competition
(what were their
motivations?)
(what were they
saying?)
(what are they up
to?)
“Soft” data

60. Data driven

61. #1:
Performance
SLA

62. Establish a baseline:

63. Establish a baseline:
A pre-defined set of metrics

64. Establish a baseline:
A pre-defined set of metrics
that describes normal behavior

65. Establish a baseline:
A pre-defined set of metrics
that describes normal behavior
in order to detect variancies

66. Establish a baseline:
A pre-defined set of metrics
that describes normal behavior
in order to detect variancies
and to be comparable within historic context

67. A practical (and easy DIY) example:

68. Purchasing a book,
must be completed (speed),
where every page loads under 3 sec.,
Customer journey
Metric: Speed
Target: Sec
using IE9 and higher,
User scenario
from any location in the Netherlands,
User locations
for 90% of all users,
every day between 6am and 12pm,
measured with Real User Monitoring.
Percentile
Window
Collection type
Read more: Metrics 101, Velocityconf 2010

69. Performance SLA

70. 100%
150%
125%
100%
75%
50%
50%
25%
25%
0%
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Sessions
Bounce rate
75%

71. LD50: 5.2 sec.
100%
150%
100%
125%
100%
75%
50%
50%
25%
25%
0%
0%
1
2
3
4
5
6
7
8
9
10
11
12
Baseline: 2,7 sec.
Poverty Line: 10,6 sec.
13
14
Sessions
Bounce rate
75%

72. #2:
Collecting
Data

73. (CLOUD) DATA CENTER
INTERNAL USERS
CUSTOMERS
INTERNET
Third-party/
Cloud Services
Storage
Major
ISP
DB Servers Web Servers
Network
Mainframe
Middleware
Servers
App
Servers
Load
Balancers
Local
ISP
Content
Mobile
Delivery
Networks Carriers

74. (CLOUD) DATA CENTER
INTERNAL USERS
CUSTOMERS
INTERNET
Third-party/
Cloud Services
Storage
Major
ISP
DB Servers Web Servers
Network
Mainframe
Middleware
Servers
App
Servers
Load
Balancers
This is what you control...
Local
ISP
Content
Mobile
Delivery
Networks Carriers

75. (CLOUD) DATA CENTER
INTERNAL USERS
CUSTOMERS
INTERNET
Third-party/
Cloud Services
Storage
Major
ISP
DB Servers Web Servers
Network
Mainframe
Middleware
Servers
App
Servers
Load
Balancers
This is what you control...
Local
ISP
Content
Mobile
Delivery
Networks Carriers
What you’re blamed for..

76. Measuring performance?

77. Measuring performance? Outside-in, from
the browser perspective...

78. Synthetic
Monitoring

79. Simulate business transactions

80. Used for error detection & Root Cause Analysis

81. Ultimately, synthetic monitoring shows you if your
site’s working or not...

82. Real User
Monitoring

83. But, synthetic isn’t enough...

84. Synthetic heartbeat
Real Users
But, synthetic isn’t enough...

85. Relies on navigation timing API, custom variables can be added
Turn on
“Site Speed”
in your GA
account!

86. Or try
Soasta “Light”

87. Ultimately, Real User Monitoring shows you how many
users are affected by bad performance...

88. #3:
Performance
analytics

89. Service levels
Service Level Report
KPI’s
082
084
Performance (avg)
Page 1
www.homepage.nl
Loadtime.
Loadtime
Availability (Document.
(.fully.loaded.)
complete)
Competition
You
5
97,0%
2,20
3,50
10%
85%
Page 2
ge.nl/autoverzekeren
99,0%
2,50
3,30
20%
90%
Page 3
overzekeren/afsluiten
95,0%
2,30
3,60
15%
70%
95,0%
2,43
3,57
20%
67%
1
Page 5
homepage.nl/zoeken
94,0%
2,48
3,62
23%
59%
Branche
2
Page 4
homepage.nl/contact
Competitors
0
Page 6
e.nl/productoverzicht
93,0%
2,53
3,67
25%
92,0%
2,58
3,72
28%
44%
3
33
34
35
ge.nl/ab8test/variant2
Page 8
91,0%
2,63
3,77
30%
37%
ge.nl/facebookplugins
Page 9
90,0%
2,68
3,82
33%
89,0%
2,73
3,87
35%
22%
ww.homepage.nl/faq
Page 11
88,0%
2,78
3,92
38%
14%
37
38
39
40
# 6
3,4 sec
Is your ranking
against
competitors
is the average
performance of your
competitors
40%
40% of your
compeitors are
faster than you
29%
page.nl/muziekpagina
Page 10
36
Week
52%
Page 7
ge.nl/ab8test/variant1
4
Source:
Availability: Gomez
Loadtime: Soasta
Competition: Webpagetest.org

90. Online Satisfaction Report
100
Availability (%)
80
400
0
-400
-800
-1200
60
40
20
Page 1
Page 11
0
Mentions (#)
1200
1200
800
Page 2
Page 3
Page 4
Positive Messages
Source:
Availability: Gomez
Sentiment: Twitter, Facebook
Negative Messages
Data collected from ING.nl
Availability
Messages from Twitter & Facebook, collected with Coosto

91. Speed vs. Engagement per month (Non-Bounce)
90
78,75
56,25
Pageviews (#)
Non-Bounce rate (%)
67,5
45
33,75
22,5
11,25
0
0-0,5
0,5-1
1-1,5
1,5-2
2-2,5
2,5-3
3-3,5
3,5-4
4-5
5-6
Non-Bounce CategorienaamPageviews
#
6-7
7-8
8-9
9-10
>10

92. Revenue risked
100.000 visits
(Performance/Time Period/Percentiles)
Optimal Flow (P_25)
Bouncerate (%)
Potential Conversion
2.1
100
49%
51.000
Page 2
4.0
100
51%
24.990
Page 3
2.0
100
60%
9996
Page 4
3.5
85
25%
6372
Actual Flow (P_85)
Speed (sec.)
Availability (%)
Bouncerate (%)
Actual Conversion
Page 1
4.1
100
59%
41.000
Page 2
6.3
100
54%
18.860
Page 3
Speed: Soasta
Availability: Webpagetest.org
Availability (%)
Page 1
Source:
Speed (sec.)
2.0
100
60%
7544
Page 4
4.7
85
53%
3013
0-2,7 sec.
2,7-3,6 sec.
>3,6 sec.

93. If anything else...

94. Deliver
fast(er)

95. Measure
Real Usage

96. Find out
where to
improve?

97. Focus on a fast
experience
…without it,
NO time for
content

98. Go check your site @
Webpagetest.org

99. Slides: bit.ly/MW-OOnfs13

100. Thanks! More questions?
M: jtjepkema@measureworks.nl
T: @jeroentjepkema
W: www.measureworks.nl

101. Outrun your
competitors

102. Optimize all
your touch
points

103. Regular website
Mobile Web
Deliver the same experience to all your (mobile) users
Native Application

104. Mobile first
www.lukew.com

105. Deliver content accordingly
Prioritize your business goals
Mobile first
www.lukew.com

106. Design for
(mobile)
use cases

107. Focus on
task
completion

108. With
mobile
conditions
in mind

109. Optimized for the right screen

111. Native app
Mobile Web - Portrait
Mobile Web - Landscape

112. (Touch)
Screen

113. 49%
One device,
One hand
http://www.uxmatters.com/mt/archives/2013/02/how-do-users-really-hold-mobiledevices.php?goback=.gmp_72842.gde_72842_member_215909354

114. Golden
touch
http://www.uxmatters.com/mt/archives/2013/02/how-do-users-really-hold-mobiledevices.php?goback=.gmp_72842.gde_72842_member_215909354

115. Make it
(feel) fast!

116. Abandonment"Rate"Across"200+"Sites"/"177+"Million"Page"Views"
Over"2"weeks"/"All#Browsers#
30"
Abandonment"Rate"(%)"
25"
20"
Abandonment*Rate*,*
All*Browsers*
15"
10"
5"
0"
0"
1"
2"
3"
Source: Gomez real user monitoring
4"
5"
6"
7"
8"
9"
Page"Load"Time"Band"(sec.)"
10"
11"
12"
13"
14"
15"
It only
takes a
second...

117. Abandonment"Rate"Across"200+"Sites"/"177+"Million"Page"Views"
Abandonment"Rate"Across"200+"Sites"/"177+"Million"Page"Views"
Over"2"weeks"/"All#Browsers#vs.#iPhone#Safari#
Over"2"weeks"/"All#Browsers#
25"
Abandonment"Rate"(%)"
30"
25"
Abandonment"Rate"(%)"
30"
20"
20"
15"
15"
10"
10"
5"
5"
0"
Abandonment*Rate*,*
All*Browsers*
Abandonment*Rate*,*
All*Browsers*
Abandonment*Rate*,*
iPhone*Safari*
0"
0"
1"
1"
2"
2"
3"
4"
5"
6"
7"
8"
9" 10" 11" 12" 13" 14" 15"
0"
3"
4"
5"
6"
7"
8"
9" 10" 11" 12" 13" 14" 15"
Page"Load"Time"Band"(sec.)"
Source: Gomez real user monitoring
Page"Load"Time"Band"(sec.)"
Source: Gomez real user monitoring
..To click
away