In response to increasing market demand for well-performing applications, many organizations implement performance testing programs, often at great expense. Sadly, these solutions alone are often insufficient to keep pace with emerging expectations and competitive pressures. Scott Barber shares the fundamentals of implementing T4APM™ including specific examples from recent client implementations. T4APM™ is a simple and universal approach that is valuable independently or as an extension of existing performance testing programs. The approach hinges on applying a simple and unobtrusive "Target, Test, Trend, Tune” cycle to tasks in your application lifecycle—from a single unit test through entire system production monitoring. Leveraging T4APM™ on a particular task may require knowledge specific to the task, but learning how to leverage the approach does not. Scott provides everything you need to become the T4APM™ coach and champion, and to help your team keep up with increasing demand for better performance, regardless of your current title or role.

1. MD
AM Tutorial
4/29/13 8:30AM
Managing Application Performance: A
Simplified Universal Approach
Presented by:
Scott Barber
PerfTestPlus, Inc.
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

2. Scott Barber
CTO of PerfTestPlus and recognized worldwide as a thought-leader in software system performance and
testing, Scott Barber describes himself as a “tester, author, speaker, disrupter, and dad.†Scott is
the author of Web Load Testing for Dummies and co-/contributing author of Performance Testing
Guidance for Web Applications, Beautiful Testing, and How to Reduce the Cost of Testing. He has written
more than 100 articles and delivered keynotes on five continents. Currently, Scott’s focus is on applying,
enhancing, and teaching Rapid Performance Testing and T4APM. When he’s not being a geek, Scott
enjoys spending time with his partner Dawn and his sons Nick and Taylor. Learn more at
about.me/scott.barber.

3. 4/11/2013
Managing Application Performance:
A Simplified Universal Approach
Created for:
By:
Scott Barber
Chief Technologist
PerfTestPlus, Inc.
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
Chief Technologist, PerfTestPlus, Inc.
sbarber@perftestplus.com
www.perftestplus.com
@sbarber
Co-Founder: Workshop On Performance and Reliability
www.performance-workshop.org
Author:
Co-Author:
Books: www.perftestplus.com/pubs
Managing Application Performance
Contributing Author:
About me: about.me/scott.barber
© 2013 PerfTestPlus, Inc. All rights reserved.
1

4. 4/11/2013
What is Performance?
System or application characteristics related to:
Speed:
- responsiveness
- user experience
Scalability:
- capacity
- load
- volume
Stability:
- consistency
- reliability
- stress
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
What is a Performant System?
A system optimized to balance:
• Cost
• Time to Market
• Capacity
while remaining focused on:
• Quality of Service to System Users
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
2

5. 4/11/2013
Performance vs. Load Testing?
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
The Performance Lifecycle is:
Conception to Headstone
Not
Cradle to Grave
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
3

6. 4/11/2013
Prevent Poor Performance…
… don’t just react when it happens.
Who is Responsible?
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
4

7. 4/11/2013
… But where does it fit in the SDLC?
5

8. 4/11/2013
…Or maybe like this?
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
…Or this?
•
•
•
•
Current Capacity
Projections
Scalability Plan
Reliability
DevOps &
Architects
Prod
Stage
• Prepare for Prod
• Tune/Optimize
• Predict
• Early Warning
Identification
Managing Application Performance
Load
Team &
DevOps
Architects
Analysts
& Scrum
• Single User
Responsiveness
• Resource
Management
• Component-Level
Concurrency
Dev
Test
Scrum &
Load
Team
• Multi-User
Responsiveness
• Resource Trending
• Limit Identification
© 2013 PerfTestPlus, Inc. All rights reserved.
6

9. 4/11/2013
…Or maybe even?
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
Performant Application
Delivery
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
7

10. 4/11/2013
Delivering Performant Systems;
Commonly, Y2K -> Today
SPE (Software Performance Engineering)
Math-centric Architect/Design method
+ Profiling
Context-free code optimization
+ Load Testing
Demonstrate (too late) poor performance characteristics
+ Capacity Planning
Math-centric guessing to determine hardware needs
+ APM (Application Performance Management)
Proactively react to production issues
=> Inefficient & Reactionary Delivery & Maintenance
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
Delivering Performant Systems;
Moving Forward
Minimize Need for Complex Mathematics
Few teams have mathematicians w/ advanced degrees
Disconnect Accuracy from Empirical Production Data
Because we can’t depend on what we don’t have yet
Remove Socio-Political Barriers
Collaboration improves information sharing & efficiency
Keep/Adapt most Valuable Practices
Many good concepts have been poorly implemented
Distribute Valuable Practices Throughout Life Cycle
Many valuable practices have suffered from stove-piping
I’m testing a model for the entire ‘Performance Journey’
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
8

11. 4/11/2013
Performant Application
Delivery
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
Key Points
• Entire Lifecycle
• Entire Team Focused
• Individual, Role, & Team Accountability & Responsibility
• Continuous & Collaborative
“DevOps”
•
•
•
•
•
•
•
Code Profiling
Requirements Revision
Design Validation
Budgets
Units & Components
Monitoring
Tuning
Managing Application Performance
Perf Team
•
•
•
•
•
Load/Stress Testing
Environment Config
Capacity Planning
Concurrency Checks
Situation Analysis
Individual
•
•
•
•
Snapshots
Comparisons
Niche Concerns
Mini-Field Studies
(UAT)
• “What-if ”
© 2013 PerfTestPlus, Inc. All rights reserved.
9

12. 4/11/2013
Preventing Poor Performance with
a little work…
every day…
from every one.
Managing Application Performance
Proactive
Micro & Macro
Establish Goals
Update Targets
Dev & Prod
Times
Resources
Sizes
Frequencies
Dashboard!
© 2013 PerfTestPlus, Inc. All rights reserved.
Units
Stories
Tiers
Resources
Goals
Assess
Compare
Investigate
Accept
Answer
10

13. 4/11/2013
Unit-Level Testing Tools
(‘cause folks always ask)
FireBenchmarks; Performance testing addin for NUnit
JUnitPerf; a collection of JUnit test decorators for performance
Firefox Performance Tester's Pack
HTTPerf
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
11

14. 4/11/2013
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
An Ounce of Prevention…
12

15. 4/11/2013
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
13

16. 4/11/2013
What is it?
• Load Tests
• Stress Tests
• Endurance Tests
• Infrastructure/Configuration Verification
• Load Variability Analysis
• Background Load for Acceptance Tests
Or any other deliberate approximation of
Production Conditions
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
14

17. 4/11/2013
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
15

18. 4/11/2013
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
16

19. 4/11/2013
Managing Application Performance
•
•
Clarify the question
Understand the driver(s) behind the question
•
•
Simplest path to (partial) answer(s)
Comprehensive path to (partial) answer(s)
•
•
Only tasks that fit in time box
Stick to tasks requiring available resources
•
•
Snapshots are your friends
Anecdotal is sufficient
•
•
Identify patterns
Confirm patterns (time permitting)
•
•
•
© 2013 PerfTestPlus, Inc. All rights reserved.
Answer(s)
Time/Effort to answer(s)
Follow-on questions of interest
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
17

20. 4/11/2013
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
18

21. 4/11/2013
The Bottom Line
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
Addendum
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
19

22. 4/11/2013
Questions?
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
Contact Info
about.me/scott.barber
Chief Technologist
PerfTestPlus, Inc.
E-mail:
sbarber@perftestplus.com
Web Site:
www.PerfTestPlus.com
Blog:
scott-barber.blogspot.com
Twitter:
@sbarber
Managing Application Performance
© 2013 PerfTestPlus, Inc. All rights reserved.
20