Performance: See the Whole Picture, ingnite session at WebPerfDays NY 2013.

1. WebPerfDays New York 2013
Performance: See the Whole
Picture
Alexander Podelko
alex.podelko@oracle.com
alexanderpodelko.com/blog
@apodelko
October 17, 2013
1

2. About Me

I have specialized in performance for the last
16 years

Currently performance testing and
optimization of Hyperion products at Oracle

Board director at CMG http://cmg.org, organization
of performance and capacity professionals
– Conference November 4-8, 2013 in La Jolla, CA
Disclaimer: The views expressed here are my personal views only and do not necessarily represent those of my
current or previous employers. All brands and trademarks mentioned are the property of their owners.
2

3. Agenda

Performance basics to make sure that we
are on the same page

Are we taking care of all parts of
performance puzzle?
– Where do we place load testing in web
operations?
3

4. Performance

End-to-end, every factor impacts

Front-end
– Network and client-side time

Back-end
– Server time

Ignore subtleties for the 5 min talk
4

5. Front-End Time

Takes 80-90% of response time for modern
web sites
– So indeed deserves the attention it is getting
now

Is constant, doesn’t change with load
– Single user may investigate
– Again ignore subtleties for the 5 min talk

Doesn’t matter for scalability
5

6. Back-End Time

Server processing

Change with load
– You need apply load to investigate

Closely correlated with scalability
6

7. Front- vs. Back-End

Back-end response times depend on load
Thanks to Andy Hawkes for the graph
7

8. Performance Risk Mitigation I

Single-user performance engineering
– Profiling, WPO, single-user performance

Software Performance Engineering
– Modeling, Performance Patterns
8

9. Performance Risk Mitigation II

Instrumentation / APM / Monitoring
– Production system insights

[Auto] Scalable Architecture

Continuous Integration / Deployment
– Ability to deploy and remove changes quickly
9

10. But all of them
don’t replace
load testing
Load testing
complements
them an several
important ways
10

11. Load Testing: Terminology
Any testing applying multi-user synthetic load
to the system
–
–
–
–
–
–
Load testing
Performance testing
Stress testing
Scalability testing
Volume testing
Reliability testing
–
–
–
–
–
Concurrency testing
Endurance testing
Longevity testing
Soak testing
Stability testing
11

12. The Stereotype

Load / Performance Testing is:
– Last moment before deployment
– Protocol Level Record-and-Playback
– Large corporations
– Expensive tools requiring special
skills
– Lab environment
No ! It is just
– Scale-down environment
one case from
–…
numerous
options !
12

13. Load Testing

Any way to apply multi-user synthetic load:
– Recording / playback on the protocol level
– Recording / playback on the GUI level
– API calls / requests scripts
– System and components levels
13

14. Web Operations and Load
Testing

“Our experiences were universal:
Our software crashed or couldn’t
scale”
– Jesse Robbins, Foreword

Found 1 mention:
– “In our opinion, the only way to make
sure our production systems could
handle the load was to test them, live,
during the day when traffic was
flowing” Brian Moon, p.134
14

15. What Load Testing Adds

Verification that the system handles the
load

Verification of multi-user performance

Performance optimization
– Exactly the same load

Debugging/verification of multi-user issues
15

16. Can You Handle Peak Load?

You can’t know without testing:
16

17. Single-User Performance
Improvement

May lead to multi-user performance
degradation
17

18. Missing Parts

Some important performance engineering
areas are not covered much at Velocity
– Modeling, load testing, etc.

Come to the Performance and Capacity
conference by CMG http://www.cmg.org
– November 4-8, 2013 in La Jolla, CA
– Perfect complement to Velocity / WebPerfDays
18

19. Takeaways

See performance end-to-end, the whole picture

Time spent is not the only variable
– Load is another, it may be different under load

Still no way to verify that the system will work
under the load but load testing
– Let’s talk during open space
19

20. Questions?
Alexander Podelko
alex.podelko@oracle.com
alexanderpodelko.com/blog
@apodelko
20