3rd in the "Rewriting the Rules of Performance Testing" series. Slides for this webinar cover ways to start scaling up performance testing with the leading agile testing expert Matt Heusser and SOASTA's Dan Bartow.

1. SOASTA Webinar Series
CLOUD TESTING RULE 3:
Rewriting the Rules of Performance
Testing Across the
Performance Testing
Lifecycle

2. Performance Has Been a Lab Exercise
Dev & Test Lab DEPLOY
Production Environment
Status Quo: Performance Testing is often
small test window prior considered hard & non-repeatable
to release due to real constraints:
• Linear, multi-step approaches
Analyze • Non-correlated, offline analysis
• Tune Code
• Build Test
• Adjust • Disconnected teams
• Setup • Collect
Environment Metrics
Settings
• Costly test hardware
• Find Issues
Run Fix • Time-intensive test setup
• Inability to hit realistic scale
Time-consuming testing limits
deployment options

3. Do We Spend Time Where It Counts?
Many approaches are process and pre-test heavy
Traditional Approaches
TEST BUILDING HW Setup TEST ANALYZE FIX RETEST REPEAT?
TIME
BUILD H
W
TEST H
W
TEST H
W
TEST H
W
TEST H
W
TEST
Extra
ANALYZE ANALYZE ANALYZE ANALYZE ANALYZE
S FIX S S S S
time!
FIX FIX FIX FIX
e e e e e
t t t t t
u u u u u
p p p p p
Eliminate constraints and TEST MORE, at SCALE, in LESS TIME
Agile approaches deliver results fast

4. Continuous Performance Testing is Progressive
Dev & Test Lab Production Environment
Start Include Hit expected
Recheck
small, early to performance scale in
Test in regularly for Plan for
root out code- focus in staging and
Production regression growth
level issues iterations maintenance
and capacity
windows
Keys to agile performance testing?
SPEED – SCALE – COST

5. Performance Bugs Know No Borders
Issues span teams, time and scale
Latency between systems
Network Network configuration
Capacity limits
& Ops
Network bandwidth
CDN file placement
Conflict with other apps
Stage / Team
DNS routing
Load Balancer configuration Firewall max capacity
Auto-scaling failures Unbalanced web servers
Max sockets exceeded
Release
& Deploy Slow third-party plug-ins s Global latency variance
Security bottlenecks
Caching issues and benefits
Search technology limits
Default configuration settings
Inadequate server resources
Garbage collection
Memory leaks
Memory leaks
Database thread counts
Dev Slow pages
& Test Inefficient database queries
Code bugs
Method-level tuning
Test Lab Staging Production (100% +++)
Scale and Time

6. Complete Performance Testing
Proactive and Predictive
Track performance regression
(and progression) over time!
Performance Coverage Status
Begin to track performance
coverage across a range of
critical objectives!

7. Let’s Look at the NEW RULES
Performance Testing: Early, Often, Always
Matt Heusser
matt@xndev.com

16. Q&A
Questions & Answers will post on CloudLink:
http://cloudlink.soasta.com
Previous & Upcoming SOASTA Webinars:
Stop Cheating & Start Running Realistic Tests (w/ Scott Barber)
Run More Tests & Find More Issues (w/ Neil Gunther)
http://www.soasta.com/info-center/webinars/
Contact SOASTA: Contact Matt
www.soasta.com/cloudtest/ matt@xndev.com
info@soasta.com Follow Matt:
866.344.8766
Follow us:
twitter.com/mheusser
twitter.com/cloudtest
facebook.com/cloudtest