Calling all DevOps teams! With back-to-school, holidays, and elections right around the corner it’s important to ensure your organization’s applications are ready for peak load performance. Millions of customers will be demanding the most from your website and mobile applications, so how can you be sure they will deliver? Can your applications’ life cycles withstand the volume? Make sure your Application Development and Management teams are ahead of the curve this season. Join this webinar with Tom Chavez, CloudTest Product Manager to learn his tips and suggestions from years of helping hundreds of organizations prepare for peak load performance.

1. ©2016 SOASTA, All rights reserved.
Tom Chavez
@tomchavez
Are Your applications Ready for
Peak Load Performance?

2. ©2016 SOASTA, All rights reserved.
• How fast is your site now?
• Measure everything
• Too much data? Science!
• Prioritize what to fix
• Optimize your site
• What should I test?
• Test early and often – shift
left and right

3. ©2016 SOASTA, All rights reserved. 3
Measure, Optimize, Test – Repeat

4. How fast is your site
now?

5. ©2016 SOASTA, All rights reserved. 5
How fast is your site now?
• Google says 1000ms!
• No single right answer
• Not all pages are equal
o Some pages should be faster than others
o The slowest pages might not be the ones to fix first

6. ©2016 SOASTA, All rights reserved. 6
How is your site now?
• mCommerce is global
o US is not the benchmark
• 39% of mobile users
are unhappy with their online experiences
o Top reasons: slow pages and site freezes/crashes
• 57% have experienced problems when using mobile
• 3 out of 4 consider mobile slower than desktop
• 46% will not return to a site that performs poorly
• 27% of smartphone users exceed data cap every month

7. ©2016 SOASTA, All rights reserved. 7
When is your Peak Season?
• Peak is year-round
o Super Bowl Sunday – 110 million viewers
o Bi-weekly or monthly payroll processing
o Valentine’s Day, President’s Day, Mother’s Day, Father’s Day
o Quarterly processing and filing
o US Tax filing
o Spring, Summer, Fall & Winter season sales
o July 4th, Christmas in July
o Back to School
o Halloween
o Singles Day – world’s biggest shopping day, $9B in 12 hours
o Black Friday, Cyber Monday and Holidays

8. Measure Everything

9. ©2016 SOASTA, All rights reserved. 9
Measure Real, Synthetic, and Virtual Users
• Real Users (RUM) provide deep data about your visitors
o No other way to know everything about your users on your site
• Synthetic Users tell you how parts of your site are doing
o When no real users are visiting, synthetic users are still coming
• Virtual Users tell you how your site will do at scale
o Can’t get a million friends to visit your site at the same time?
• Collect all the data and save it in the cloud!
o Storage is cheap in the cloud, plus easier to analyze there
• Add App Performance Management (APM) for a complete picture
The Three Musketeers of Performance

10. ©2016 SOASTA, All rights reserved. 10
Other Things to Measure
• Session paths
o How are your users going through your site
• Mobile App Performance
o How are your mobile apps performing?
o What are users doing in your mobile apps?
• Revenue
o How much money are you making from each user and where?
• How fast are your competitors’ pages?
o You can’t insert code on their sites but you can measure
Your company may be measuring these already

11. Too much data?
Science!

12. ©2016 SOASTA, All rights reserved. 12
Analyze
• Benchmark and baseline your site
o How fast is your site now? How about the competition?
• Correlate performance and outcome
o Is poor performance affecting the business?
• Find root causes for performance issues
o Use RUM and APM data
• Fix pages that impact most
o Make a prioritized punch list of the pages to fix
Use the data you’ve collected to gain insights

13. ©2016 SOASTA, All rights reserved. 13
Baseline Your Performance
How is your site doing? How does it compare to the competition?

14. ©2016 SOASTA, All rights reserved. 14
Correlate Performance and Outcome
How does Page Load affect Conversion Rate?

15. ©2016 SOASTA, All rights reserved. 15
Find Root Causes for Poor Performance
Not all problems are from your app or back end systems

16. ©2016 SOASTA, All rights reserved. 16
Page Visits vs. Performance

17. ©2016 SOASTA, All rights reserved. 17
Page Visits vs. Performance by Device

18. ©2016 SOASTA, All rights reserved. 18
Prioritize What to Fix – Greatest Impact
The poorest performing page might not be the most important page

19. Fix and Optimize

20. ©2016 SOASTA, All rights reserved. 20
Page Bloat
The average web page is more than 2MB in size

21. ©2016 SOASTA, All rights reserved. 21
Page Bloat
Where are all those kilobytes going

22. ©2016 SOASTA, All rights reserved. 22
Doesn’t Bandwidth Solve the Problem?

23. ©2016 SOASTA, All rights reserved. 23
Comparing Network Performance

24. ©2016 SOASTA, All rights reserved. 24
Comparing Network Performance

25. ©2016 SOASTA, All rights reserved. 25
Doesn’t Bandwidth Solve the Problem?

26. ©2016 SOASTA, All rights reserved. 26
Doesn’t Bandwidth Solve the Problem?
25x Faster Bandwidth != 25x Faster Load Time

27. ©2016 SOASTA, All rights reserved. 27
Page Bloat
Image Optimization
• Compress images
o Reduces time to send, increases number stored in cache
• Reformat images
o Photos: JPEG, PNG-24; Low complexity: GIF, PNG-8; Line: SVG
• Spriting
o Multiple images in a single image file: just one request
• Load feature images early and quickly
o Defer non-essential images “below the fold”

28. ©2016 SOASTA, All rights reserved. 28
Front-End vs. Back-End Load Times
What’s slowing down page loads the most?

29. ©2016 SOASTA, All rights reserved. 29
Prepare for Peak Events
Optimize content for peak load events
• Simplify website content
o Keep only the most important content on the page
• Reduce infrastructure requests
o Block unnecessary requests, out-of-region visitors
• Leverage historical comparisons
o Review past traffic levels to predict future levels

30. ©2016 SOASTA, All rights reserved. 30
Measure Before and After
Is the new release faster than before?

31. What Should I Test?

32. ©2016 SOASTA, All rights reserved. 32
Creating a Test Plan – Session Analysis
Which paths are the most important to users?

33. ©2016 SOASTA, All rights reserved. 33
Creating a Test Plan – Session Analysis
Which paths are the most important to users?

34. ©2016 SOASTA, All rights reserved. 34
Creating a Test Plan – Peak Load
What was the busiest Day, Hour, Minute, Second?

35. ©2016 SOASTA, All rights reserved. 35
Creating a Test Plan – Page Flows
What were most popular page flows with think times?

36. Network Emulation

37. ©2016 SOASTA, All rights reserved. 37
Network Emulation
• Testing to match realistic situations
o Users are not all from one location on a fast network
o Packets get delayed, lost, re-ordered
• Apps perform differently as network conditions worsen
o Reduced throughput
o Timeouts occur, connections are lost
o Transactions take longer, longer sessions
Adding real network modeling to load and performance testing

38. ©2016 SOASTA, All rights reserved. 38
Network Emulation Terms
• Bandwidth: the bit-rate of available capacity of the network
• Upload rate: the maximum rate packets are sent from the queue
• Download rate: the maximum rate packets can be received from the queue
• Packet: a unit of data carried over a network
• Latency: the time for the packet to get from one point to another
• Jitter: a variation in the delay of received packets
• Packet loss: the percent of packets dropped from the network queue
• Corruption: the percent of packets arriving with incorrect data
• Duplicate: the percent of packets duplicated during transfer
A short dictionary of terms

39. ©2016 SOASTA, All rights reserved. 39
Sample Network Emulation Profile

40. Test Early
and Often

41. ©2016 SOASTA, All rights reserved. 41
Start Testing Yesterday
• How long will it take?
o 300 test cases x 2 browsers = 600 cases
o 600 cases x 30-minutes/case = 300 hours
o 300 hours / 6 hours/day = 50 days
o 50 days / 5 people = 10 days
• And for mobile:
o 300 text cases x 26 devices = 7800 cases
o 7800 cases x 30 minutes/case = 3900 hours
o 3900 hours / 6 hours/day = 650 days
o 650- days/5 people = 130 days of test – 26 weeks!
And even earlier for Mobile

42. ©2016 SOASTA, All rights reserved. 42
Don’t Wait Til the Last Minute to Test!

43. ©2016 SOASTA, All rights reserved. 43
Test Right and Test Left, even into Production
• Performance Testing in development
o Set performance SLAs for new code and test in development,
same as unit and functional tests which code must pass
• Performance Testing in staging
o Higher loads in near-production systems
• Performance Testing in Production
o Full scale load tests to expected load and beyond
o Find weak and stress points and prioritize fixes into project
Performance Testing is Continuous

44. ©2016 SOASTA, All rights reserved. 44
Test Continuously
Test from Development to Integration to Staging to Production

45. ©2016 SOASTA, All rights reserved. 45
Test in Production
Some issues will only be found at scale load on production systems
Don’t forget your third party providers: CDNs, external scripts

46. ©2016 SOASTA, All rights reserved. 46
Measure Mobile App Performance Under Load
All mobile apps perform well when there is no load

47. ©2016 SOASTA, All rights reserved. 47
Automate Your Testing
Use Continuous Integration tools to automate testing

48. ©2016 SOASTA, All rights reserved. 48
Now That the Test is Over
• Analyze results for future areas to address
o Weak or stress areas that aren’t likely this time but …
• Identify third parties that were affected by load
o Coordinate with suppliers to address issues
o Align schedules or at least be aware of their changes
Even if the test passed, you’re not done

49. ©2016 SOASTA, All rights reserved. 49
Issues Will Happen
• “I am 100% confident that everything we tested will work
just fine.” – Nordstrom Performance Leader
o When loads are different than what you modeled, may have
problems
o E.g. At Nordstrom, during a recent anniversary sale, mobile sign-
ins were 9x greater than anticipated
o Used RUM to get to the root cause of the issue
Realize that Issues Will Happen Where You Can’t Predict Them

50. Q & A
Resources
Next Steps

51. ©2016 SOASTA, All rights reserved. 51
Questions, Resources & Next Steps
• The Performance Beacon
o soasta.com/blog
o https://www.soasta.com/blog/2015-popular-posts-web-performance/
• Upcoming Webinars
o soasta.com/webinars:
Holiday Readiness: What to test, how to test
o Recordings of past webinars
• CloudTest on-Demand
o soasta.com/cloudtest: SOASTA will build & run tests for you
Resources

52. ©2016 SOASTA, All rights reserved. 52
“The leader in performance analytics.”
www.soasta.com
SOASTA

53. ©2016 SOASTA, All rights reserved. 53
CloudTest
• Load testing from development to production
o Continuous testing at any scale, on-prem or in the cloud
• Real-time analytics with customizable dashboards
o Safely test in production with full visibility
• Run tests live or automatically
o Full hands-on control or hands-off automation
• Full end-to-end view
o Front-end performance to back-end systems
Fast, scalable, affordable load testing for performance optimization