Presented at Jasig 2010 conference March, 2010 in San Diego.

1. Adding Performance
Testing to a Software
Development Project
Cris J. Holdorph
Lennard Fuller
© Copyright Unicon, Inc., 2008. Some rights reserved. This work is licensed under a
Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.
To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/us/

2. Agenda
1. What is performance testing?
2. Performance Test Environment
3. Setting Goals
4. Performance Test Tools
5. Performance Test Data
6. Analyzing the Results
7. Optimizing the System
2

3. What is performance
testing?
3

4. Performance Testing Overview
Purpose of performance testing is
NOT to find bugs.
●
Intended to eliminate performance
bottlenecks
●
Establish a baseline for future
regression testing
●
Results are compared to a clearly
defined set of Goals
4

5. Typical Performance Testing Cycle
Run Test
Optimize Analyze *
* Repeat until Goals are reached.
5

6. Performance Test
Environment
6

7. Performance Test Environment
●
Separate Hardware
– Production duplicate if possible
●
External Systems
– Will your performance test system hit your
production LDAP? SSO? SIS?
7

8. Goals
●
Expected Users
●
Expected Response times
●
Making sense of the misunderstood
– User Concurrency
●
Time to return HTML for a page is not equal
to the time to load a page
– Testing Full Page loads versus HTML only page
loads
8

9. Tools
●
Grinder
●
JMeter
●
Commercial
– Loadrunner
– Silk Performer
●
Recording
●
Modifying / Tokenizing
9

10. Prepping the Test
●
Set up Test Environment
●
Load Database with Realistic Data
– Type of data
– Amount of data
●
Record the Test Script
– Record (including sleep time)
– Tokenize / Modify / Edit
●
Set up Test Scenario
– Test script mix
– Sleep time randomization
10
– Ramp up time

11. Running the Test
●
Gather information during the test
– CPU, IO, memory, GC
●
Zero out logs before test begins
●
Reset the database before test begins
– Only necessary if performance test changes data in
certain ways, or if test adds data
●
Restart the application
●
Prime the application
– Login as a test user(s) and do actions similar to what
the performance test does
●
Pressing run (start) on the test tool 11

12. Analysis
●
Gather statistics from different sources
– Test tool, sar, custom shell script, gc logs
●
Examine application logs
– Are there any errors or stack traces that show a
performance related failure?
●
uPortal only
– uPortal built in stats database not recommended
for performance testing, but can help debug
production performance problems
●
Look for Trends
●
Look for Spikes 12

13. Optimization
Can be difficult, often more of an art than a
science due to complexity.
Cardinal Rule of Performance Optimization:
“Modify ONE variable at a
time and remeasure.”
13

14. Some Areas for Optimization
●
JVM args
●
Application Content
– Portlets, Layout, etc.
●
Application Configuration
– Logging Level, Logging Location
– Database Connection Pooling
– Tune Cache Configuration
●
System Architecture
– Use Apache HTTPD in front of Tomcat to serve static
resources
– Use hardware load balancing / SSL
14
– Increase available network bandwidth

15. Part of Normal Development
●
How to make Performance Testing part of the
normal Software Development lifecycle
●
Benefits
– Regression Testing (find problems before they
occur in production)
– Anticipate Hardware needs before code rolls to
production
●
Need
– Test Environment
– Test Scripts
– People to run/analyze the test 15

16. Limitations: Spikes and Valleys
●
Performance testing does NOT simulate the
real world
– Both Queueing Theory and Experience show that
usage is not uniform!
– Caution: Outside factors could influence your
system
16

17. Real world looks more like...
17

18. Limitations
●
Users don't all do the same things in the
same order
●
Real usage can differ wildly in terms of
associations, and actual data
– i.e. chinese class with 400 students
●
AJAX / client side activities are hard to
performance test, using traditional
performance testing tools
18

19. Questions & Answers
Cris Holdorph
Lennard Fuller
Unicon, Inc.
holdorph@unicon.net
lfuller@unicon.net
www.unicon.net
19