Breaking the Kubernetes Kill Chain: Host Path Mount
Using digital performance analytic to optimize digital user experience id av5
1. 1 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
Using Digital Performance
Analytic to optimize Digital
User Experience
2. 2 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
The New Digital Customer
Empowered
60%
rate performance/response time as
#1 mobile app expectation ahead of
features and functionality
(Forrester, Nov 2014)
DemandingMulti-
device 2+ BILLION
People have some form of a social
media account.
44%
will air their frustration on social
media when poor experience
(Dynatrace Survey, Dec 2014)
95%
digital shoppers who started on a
smartphone continued on a PC
(Forrester, Q4 2014)
3. 3 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
No Business is Immune to Digital Disruption
Do you think your
business will be
disrupted by digital
technologies in the
next 12 months?
Forrester, March 2014
7. 7 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
3 seconds of tolerance
• 46% will shop elsewhere if a site takes longer than this
• 60% rate performance/response time as #1 mobile app
expectation ahead of features and functionality
8. 8 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
New Reality, New Questions
+ What is a typical customer
journey across touch-
points?
+ What is driving
abandonment at given
pages/steps?
+ Why is my app getting
2 stars ratings?
+ How do visitors
perceive my site vs.
my competitors?
+ Why is my conversion rate low with this
customer segment?
+ How is conversion
affected by
performance?
+ Why is A/B Testing
not improving
conversions?
+ What is the impact of
adding another ad
block to my site?
9. 9 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
What’s Preventing Better User Experience Insights?
+ What is a typical customer
journey across touch-
points?
+ What is driving
abandonment at given
pages/steps?
+ Why is my app getting
2 stars ratings?
+ How do visitors
perceive my site vs.
my competitors?
+ Why is my conversion rate low with this
customer segment?
+ How is conversion
affected by
performance?
+ Why is A/B Testing
not improving
conversions?
+ What is the impact of
adding another ad
block to my site?
• Siloed analytics per channel
• Rising complexity
• 50 external services behind a
site on average (social, ads,
analytics, etc.)
• Device type proliferation –
600+ devices to cover 80% of
the market
11. 11 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
Getting Answer from Data
- In WWII, Allied
bombers were key to
strategic attacks
- constantly shot down
over enemy territory.
The planes needed
more armour
- Abraham Wald
started with a simple
diagram—the outline
of a plane
12. 12 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
Optimised Digital User Experience
User Behavior
User EnvironmentUser Action
Response Time
Errors
13. 13 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
5 Key Performance Metrics to optimise digital user
experience
Responsiveness
Latency
Third party
impact
Load Testing
Metric
Benchmarks
15. 15 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
Responsiveness
• Is your application accessible in
different country or region?
• Are the users of your application
Satisfied, Tolerating or Frustrated
• Is your application optimized for the
platform it’s being viewed on?
• Are your users able to easily view
and navigate through the
application?
19. 19 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
Responsiveness: The whole visit matters
20. 20 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
Responsiveness: The whole visit matters
50% transactions abandoned
Visit 2
6% steps with issues
Visit 3
Visit 4
Visit 5
Visit 6
Visit 1
visit also needed to compute conversion
and bounce rates
21. 21 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
• Service Recovery/
Personalised call to
• Failed Check in for Gold
Frequent Flyer members
• Failed ecommerce transaction
of more than $200
There is an opportunity in this
24. 24 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
Latency: Optimising every component
• The average attention
span of today’s consumer
is 3 - 8 seconds.
• Lag times between
requests and page loads
run the risk of losing
customers.
• 1 Sec delay = 7%
reduction in conversion
25. 25 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
Latency: Optimise your images
According to the HTTP Archive, the average top 1,000 web page is
1491 KB in size
26. 26 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
Latency: Improving your Latency
• Allow more requests to happen concurrently.
• Shorten the server round trips by bringing content closer to
users.
• Reduce the number of round trips.
• Improve the browser cache, so that it can:
• store files and serve them where relevant on subsequent pages in a visit
and
• store and serve files for repeat visits.
31. 31 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
Load Testing Metric
• China Railway
Corporation
• 2.97 Billion Page
Visits per day
• 280M Million
tickets per day
33. 33 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
Load Testing: Why you Load Test
• Be Prepare before you launch your app or site
• Plan for your intended load and concurrent usersLaunch Readiness
• Understand your user behaviour
• Plan ahead for peak (Marketing campaign/ year end
events)
Prepare for Peak
• Start small but prepare to scale
• You do not plan for your application to stay smallDesign to Scale
34. 34 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
Load Testing Metric
• Web Server (Busy and Idle Threads/ Throughput/ Bandwidth)
• App Server (CPU Hotspot/ Worker Threads/ Memory issues)
• Host Health (CPU, Memory, Disk I/O/ Key Processes)
• Application (Time Spent in logical Tier or Layer / No. of calls into a logical tier)
• Database Activities
• Average # SQL per user/ Total # SQL statements/ slowest SQL)
• Connection Pool (Utilisation / Acquisition time)
• Error Detection (# of exceptions/ # of log entries)
Download the “Key Performance Metrics For Load Tests Beyond Response Time” whitepaper
35. 35 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
Benchmarking: Context is key
4.59 sec 2.04 sec
36. 36 COMPANY CONFIDENTIAL – DO NOT DISTRIBUTE #Dynatrace
Benchmarking: Features vs Performance
What's the thing you touch the first thing you wake up?
- Your handphone?
What is the last thing you do at night?
- Maybe a tablet, streaming video or shopping
What happens when the steaming hang of the shopping site hang?
- You get frustrated and you post your frustration on the social media
And businesses are adapting to attract these new digital customers
With the current digital trends, all almost all industries thinks that their industry will be impacted by the digital disruption
In Insurance – 22 day to 22 mins
Egov initiative, Paying of Tax, Checking of CPF and even registering of new companies are done online
In Banking, Citibank CEO Micheal Corbat even say that Citibank is a Technology company with a banking licence.
(In the case of Visa, during a failed transaction over credit card. The cashier will ask you “Do you have another card. She will not ask if you have another Visa card.
You are lucky if you tries to swipe the card more than 2 times
Last but not least, Even in F&B, you see your waitress, keying in your order using ipads
What’s even more interesting, Product differentiation may not even be important any more. The new fledging companies are all platform companies that do not have physical products.
And what happens when the basic product and services get commoditizes. Access and performance becomes critical.
Let me show you a video from a Forester Analyst on the impact of ignoring this Digital Disruption.
So what does the digital disruption got to do for business
So we now know or agree that digital is becoming more important. But just in 2014… There are
15.8 outage per hour. What are organisation doing to address this?
And So what if your website is up? Is it enough?
Forrester 2014
In our study we found 46% of shoppers will shop elsewhere if it takes longer than 3 seconds for a retail site to load.
And yet only 40% of the fortune 500 load within 3 seconds.
Which is a problem in considering average time to just load the home page of the Singapore Financial Industry is already 4.017 seconds.
SO in order to be Digital Predator, we need to ask ourselves new questions And this shift in the digital user’s expectation prompts business to ask new questions
Questions like
And the data to digital performance is key to answering these question. However, it is not simple
Some of these data can be fragmented and Siloes For example,
The traditional IT way of monitoring application will just tell you that the Network and Infra is up and available
The Google Analytic data give insights to web performance, but cannot co-relates to the IT architectures
Coupled with rising complexity when a average website can host up to 50 external service (including, social, ads and analytics on top of the rising no of devices and OS out there
Digesting this data is not easy. After all what we want is answer, not data
Let me give you 1 example where data is used to derive Answers
He marked bullet holes corresponding to where each returning bomber had been shot.
The wings, nose, and tail were blackened with bullet holes, so these were the spots that needed more Armor. Or so we thought.
But a more analytical way is to understand what happen to the planes that did not come back?
That are actually hit in the AREA where there are no bullet holes in Abraham’s diagram.
So similarly in Digital Performance, we need to interprete the data from various sources to deliver a Optimise digital user experience
Metrics like User Action Response Time, Errors, User Behaviour and User environment are all like the bullet holes in the previous example where data are analysed together to get an answers – the Optimise Digital User Experience
And At Dynatrace we recommend 5 performance metric that business should look out for to optimise the user experience
fundamental that for all you website or application is responsive and available in your intended geographies across the different devices and platforms
As you collect the user data, all users, all transaction, you can map out the no. of Satisfied, Tolerating or Frustrated user across region and geographies
You should also understand the impact of different devices and OS on your applications.
Is your application accessible in different country or region?
Is your application optimized for the platform it’s being viewed on?
Why is all this important?
Eg. The recent launch of iphone 6S and iOS or window 10 updates. It is important to note if the majority of your users has done the upgrades and what impact is that having on your applications? If a big percentage of user is impacted, and conversion is dropping, you may need to update your application quickly
The ability to drill down on the frustrated or affected users helps you to prioritise and drill down to the root cause faster.
The entire user experience of the user is crucial, it is no good that your application is performing working 80% of the time only for it to fail during the last steps.
You may have encountered experiences like travel booking or booking of movie tickets but only to have the transaction failed in the last step. In fact, it could even be more frustrating.
We at Dynatrace have note that in the case where 6% of the step that have issues; there could be more then 50% abandonment.
However, with digital performance analytics if the booking site can call you back and say that, Hi Mr Jerry, we noticed your transactions have failed. How can I help you complete your transactions?
What business impact it can have, if this is done for:
Failed Check in to airlines for gold or platinum members
Failed ecommerce transaction worth more than $200
You may have encountered experiences like travel booking or booking of movie tickets but only to have the transaction failed in the last step. In fact, it could even be more frustrating.
We at Dynatrace have note that in the case where 6% of the step that have issues; there could be more then 50% abandonment.
However, with digital performance analytics if the booking site can call you back and say that, Hi Mr Jerry, we noticed your transactions have failed. How can I help you complete your transactions?
What business impact it can have, if this is done for:
Failed Check in to airlines for gold or platinum members
Failed ecommerce transaction worth more than $200
“latency” is the amount of time it takes for the host server to receive and process a request for a page object.
And as mentioned earlier in the slides, attention span of the consumers today is 3 to 8 secs and with the average website having more than 50 external content, it is crucial to optimise every component.
So a waterfall chart like this will help identify when components are causing the bottleneck
Website are getting more and more heavy and one of the most common culprit is images.
This is an example of a very poorly design website, with too many pages to load and too confusing.
DoubleClick is a subsidiary of Google which develops and provides Internet ad serving services.
DoubleClick is a subsidiary of Google which develops and provides Internet ad serving services.
Measuring third-party content can accurately determine whose domain is responsible for poor performance.
While we trust that most of the 3rd party provider does adhere to their SLA most of the time, but we need to manage them.
After all it is your website or business at stake, not the 3rd party provider’s
The proactive monitoring of these 3rd party providers will “save” your site as most outages started with performance degration over a period of time
Have a big launch coming up? Make sure you’re ready. Ensure that your website or application can handle the spike in traffic that is likely to happen on launch day
That might have been a pretty extreme case. But without data on Web load,
Your business may be impacted when it matters most to you.
IRAS – last weeks of tax filing
Education – accessing results and applying for new course
And when you just spend 30k on a advertising campaign to promote your services
Launch Readiness. Many businesses always test their feature and function but ignore performance. But note that Performance is usu the factor that given your app the “ratings” no point having a poor rated app and having to redesign the app again
Over time, new features gets added into you website and application.
Need to balance the features vs performance. Do house keeping and manage your most critical feature that give you 80% of your business