Whether you are just starting to think about adopting a cloud model or have already implemented this in your organization, by moving from on-premise applications to the cloud you are giving up lots of control. How do you make sure you meet end-users’ performance expectations when someone else takes the reigns? End-users expect sub 1s response times as your on-premise apps did. How can you ensure such performance with your cloud service provider? Web performance is paramount for your users and not having the right metrics in place to remain in control can lead to costly performance issues.

1. How to Ensure
High-Performing, Highly
Available Cloud Applications
Ben Rushlo
Director, Web Performance Consulting, Keynote Systems
April 21, 2009
©2008 Keynote Systems, Inc. Keynote & Client Confidential.

2. Cloud Computing
“A form of standardized IT-based capability
– such as Intranet-based services, software, IT infrastructure –
offered by a service provider that is accessible via Internet
protocols from any computer,
is always available
and scales automatically to adjust to demand,
is either per-per-use or advertising-based,
has Web- or programmatic-based control interfaces,
and enables full customer self-service.”
Frank Gillett, Forrester Research
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 2

3. 2008 – Year of Web Glitches
Aug 12, Oct 17
Feb 15 Jul 27
Dec 1
April, May, June. July
Jul 10 Jun 17
Jul 8
Nov 28
Jun 17
Jun 17
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 3

4. Cloud Computing’s Reliability & Availability Gap
Enterprises can’t completely rely on cloud infrastructures to
run their business
Almost no SLAs provided by cloud providers
Using the Cloud lessens control and increases complexity of
managing technology
Who provides you “Cloud Insurance”?
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 4

5. Ensure Reliability and Availability in ALL Layers
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 5

6. Cloud Computing Is Changing
Performance Management
April 21, 2009
©2008 Keynote Systems, Inc. Keynote & Client Confidential.

7. The Application Lives Everywhere
The “Cloud” – Web services, distributed data
centers/infrastructure, content delivery network etc
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 7

8. The Browser as a “Platform”
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 8

9. Complexity is Increasing
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 9

10. Complexity is Increasing
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 10

11. Complexity is Increasing
Domains
1 -2 5-10
Third Party Content
1-2 requests per page 10 30 requests per page
Page Sizes
100-300K 300K – 2MB
Languages
HTML, limited JS and CSS Multiple JS and CSS per page (XML,
Flash, Flex etc)
Hosting
Single location Hybrid (Cloud)
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 11

12. The Application Life Cycle On “Red Bull”
Quality
Analysis
IT
Operations
Web
Development
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 12

13. Increased User Expectations
The Internet has become a utility
Always on, always responsive
The competition is raising the bar
Even if it is not your direct competition
8s 4s 2s ?
The Cloud model changes expectations
Users are not simply “viewing content”, instead they are interacting,
using applications
The user doesn’t care that it is hosted in the cloud and will not lower
expecations
1s or less might become the new rule
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 13

14. Performance Management Best
Practices – More Important Now
Than Ever
April 21, 2009
©2008 Keynote Systems, Inc. Keynote & Client Confidential.

15. Performance Management Best Practices
Test at every stage in
Application Lifecycle –
build with the user in
mind
Test from the Cloud –
where your users are
Test with a Browser –
what your users use
Capture the Right
Metrics – what impacts
your user
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 15

16. Key Metrics – Pre Launch
Average download times
Scalability
Per element performance Quality
Analysis
Client side processing
IT
Variability – over geography Operations
Web
Web service performance Development
Pre render vs. render time
Network vs. Application vs. Content
Page Construction
Test out the cloud before your users do
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 16

17. Key Metrics – In Production
Performance
-average versus target Quality
Analysis
Availability
– errors and outages IT
Operations
Variability Web
Development
– throughout day and
overall
Third-Party Quality
Trends and Events
Don’t let your users monitor the cloud for you
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 17

18. Total Site Quality Framework
Tip: Avoid Pet Metrics
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 18

19. Key Performance Challenges
April 21, 2009
©2008 Keynote Systems, Inc. Keynote & Client Confidential.

20. Asset Count More Important Than Size
Size matters less and less
Asset count primary driver for high speed user’s
performance experience
Roundtrips! Roundtrips! Roundtrips!
Most sites have highest asset counts in three areas
External CSS
External JS
Static/designer images
Combination of these files is key
Sprites, packing JS etc.
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 20

21. JS and CSS Strategy
External JS and CSS was designed to allow caching across
the site
However, many sites are loading unique JS and CSS per page
Some of this related to “silos” within organizations
Developers need to think about the “user flow” not per page
Coordination is key
Similar functionality should be in “Global” JS and CSS and loaded on
key entry points in site
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 21

22. File Size
Size does matter to poorer connected users
The “grey area” users – wireless, mobile devices, dial-up, satellite,
coffee shops, hotels etc.
JS and CSS
Need to be compressed
Need to be minified (comments removed, whitespace removed)
Old code pruned
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 22

23. Download Time
0
1
2
3
4
5
6
7
8
9
UPS
Live
Travelocity
Wikipedia
Sprint
HotJobs
Career Builder
Disney
Fidelity
Yellow Pages
©2009 Keynote Systems, Inc. Keynote & Client Confidential.
Google
AT&T
Orbitz
Merrill Lynch
MSN
eBay
Ask
40 Typical Home Pages
CNN
Expedia
Time On Netw ork
Client Side Processing
AOL
Bank Of America
Symantic
Facebook
Ticketmaster
NY Times
Apple
Hewlett-Packard
Client Side Processing
April 21, 2009
Amazon
CBS Sportsline
Verizon
Yahoo
USA Today
Dell
Walmart
Priceline.com
MSNBC
Weather.com
Charles Schwab
FedEx
Monster
23

24. Flash Initialization
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 24

25. JS and CSS Rendering
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 25

26. JavaScript and The Browser
External JS is not only a roundtrip but is also serialized the
browser
Each file much be loaded in serial
Exponentially extends the download
Worse in the top of the load (pre render)
Pre render JS files should be limited (ie. few of them)
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 26

27. JS Threading Example
Javascript
files load
one file at a
time
None of these
images were
downloaded to the
browser until 2.4
seconds into a 2.8
second page load
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 27

28. Geographic Distribution of Content
Performance varies significantly based on geography
Key factors that drive amount of variation are
Element counts (Roundtrips!)
User distance from data center
Use of Content Delivery Network
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 28

29. Online Tax Application- Canada
25.0
20.0
15.0
Seconds
10.0
5.0
0.0
Home Page Online Online Edition Get Started Validation
Vancouver Telus 4.61 1.18 1.37 6.70 0.51
Calgary Telus 6.78 1.64 1.58 8.49 0.57
Toronto Bell 9.27 2.71 2.71 17.90 0.97
Montreal Verizon 11.68 3.48 3.14 21.46 1.09
Vancouver Telus Calgary Telus Toronto Bell Montreal Verizon
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 29

30. Third Party Usage and Quality
It is quite common to find pages that have 10+ domains that
are third party
Floodlight tags (which dynamically insert new domains)
Tracking (Omniture, Google Analytics etc)
Media buy/tracking tags
A/B testing
Survey/opinion tags
Etc
Third party quality needs to be monitored
Third party tags must be POST render where possible
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 30

31. Third Party Call Blocking
Visual Content
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 31

32. Cache Management
For many sites, visitors return quite frequently
Sites need to take into account the impact of caching
Cache needs to be done appropriately
Far future expire headers
Etag configuration
The performance impact of “validating” cache with the
server is AS SIGNIFICANT as downloading that content
fresh (for high speed user)
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 32

33. Automotive Site Example
– Return Versus New
3.00
2.50
400K
2.00
Seconds
1.50 40K
1.00
0.50
0.00
Home Page - New Visitor Home Page - Return Visitor
Geometric Mean
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 33

34. Don’t Forget The Basics
Cloud vendors must provide
performance metrics
Measure your cloud application
yourself as well
Ensure SLAs in place
Measure…Measure…Measure
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 34

35. Keynote’s Cloud: More than 240 locations
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 35

36. Test your apps for FREE from Keynote’s Cloud
Download KITE 2.0
@
kite.keynote.com
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 36

37. How to reach me
Ben Rushlo
ben.rushlo@keynote.com (623) 547-7068
http://www.linkedin.com/in/benrushlo
©2009 Keynote Systems, Inc. Keynote & Client Confidential. April 21, 2009 37