This presentation focuses on the basics of Performance Testing. It talks about the processes, challenges and activities involved with Performance Testing.

1. Fundamentals of Application Performance Testing
Practical Performance Analyst – 28th July 2012
http://www.practicalperformanceanalyst.com

2. Agenda
Performance Engineering Life Cycle
What is Proactive Performance Management
Holistic View of Performance
What is Performance Testing
Why is Performance Testing Important
Aspects to consider while Performance Testing
Pitfalls to avoid during Performance Testing
Performance Testing Process
Challenges involved in Performance Testing
Deliverables for the Performance Testing process
Resources & tools for Performance Testing

3. Performance Engineering Life Cycle
Software Development Life Cycle
Functional Requirements Gathering
Architecture & Design
Build Application
System Test,
System Integrated Test & UAT
Deploy Into Production
Performance Engineering Life Cycle
Non Functional Requirements Gathering
Design for Performance &
Performance Modelling
Unit Performance Test & Code Optimization
Performance Test
Monitoring & Capacity Management

4. Proactive Performance Management
Performance Requirements Analysis
Performance Modelling & Capacity Planning
Build & Optimization
Performance Testing
Performance Monitoring
Capacity Management
Proactive Performance Management helps by –
Identifying performance issues early in the design
Validate suitability of your application architecture for your documented NFR’s
Validate your application capacity requirements for the documented NFR’s
Advocates early SVT to identify potential bottlenecks early in the build and development cycle
Reduces potential for major rework later in the program
Advocates using modelling techniques to predict performance impact for growing workload

5. Txn Performance
- Response Times, etc.
Application Performance – Operations/Sec, Messages/Sec, Transactions/Sec, etc.
Infrastructure Performance – CPU Utilization, Memory Utilization, Disk IOPS, etc.
Network Performance – Packet Loss, Jitter, Packet Re- ordering, Delay, etc.
Holistic View of Performance

6. What Is Performance Testing
Performance Testing is quite often confused with Performance Engineering. Although Performance Testing is an integral and important part of Performance Engineering its usefulness is confined to validating Application Performance later in the SDLC (Software Development Life Cycle).
Performance Engineering as we’ve defined it in the past consists of a set of roles, responsibilities, processes and deliverables across the SDLC whose objective is to build applications that meet their NFR’s.
Performance Testing can be defined as the process of validating your application performance against the set of defined Non Functional Requirements.
The objectives of Performance Testing are generally a combination of the following:
Validate Application Scalability – Performance Testing
Validate Application Performance at Peak Workload – Stress Testing
Validate Application Performance over a extended duration of time – Soak Testing
Establish Application Breaking Point – Break Point Testing
Performance Testing has traditionally been and continues to be a proactive process. Programs and project generally lack both the capability and willingness to consider Performance early in the SDLC.
While Performance Testing allows for validation of Application Performance later in the SDLC, a Performance Engineer can use a combination of different modelling techniques to validate application designs, infrastructure capacity and application performance early in the SDLC and mitigate risk accordingly.

7. Why Is Performance Testing Important
Performance Testing is important for the following reasons –
Identify bottlenecks across the application and highlight opportunities for tuning
Validate your design & infrastructure capacity assumptions made early at the design stage
To validate if your applications meet the program Non Functional Requirements
Validate that the application is able to deliver the expected End User Experience while consuming resources optimally across the infrastructure and network tier
To determine the breaking point of your applications. Knowing the breaking point of your applications helps you understand the maximum Business Workload the application can process before performance starts to degrade.

8. Aspects to consider while Performance Testing
Following are the aspect you should keep in mind when planning Performance Testing –
Performance Testing should ideally start through setting up of the program Performance Engineering team at the Requirements Gathering & Design Stage. At this stage you should be working on closing of the following documents:
Program Performance Engineering Strategy
Performance Testing Workload
Non Functional Requirements
Performance Testing Strategy
Start early and don’t wait until the code base completely stabilizes. I encourage you to start Performance Testing early, identify opportunities for tuning and go back with feedback to the development teams.
Work closely with the development teams to meet your documented NFR’s. Make sure you have the buy in of your program leads, environment leads and development team leads. With the appropriate buy in issues get addressed a lot faster.
Discuss and document your environment requirements. Anything lesser than 100% capacity of production will require you to use some black magic (aka. Performance Modelling Techniques) to extrapolate performance characteristics
Discuss and document your data requirements (Script data requirements, application data requirements, etc.). Data preparation can be one of the most challenging tasks on a program.
Discuss & document your Performance Testing and Performance Monitoring (Application + Infrastructure) tool requirements. You will also need to discuss and work out the cost implications due to Virtual Users (VU) licensing.
Make sure everyone the program understands the NFR’s and overall SVT Business Workload very clearly. Ambiguity in any of these areas will land you in serious trouble later in the program.
Track performance issues closely. Use your program risks & issues register to track performance bottlenecks that are preventing you from progressing with SVT.

9. Aspects to consider while Performance Testing
Consider having a good balance of Performance Engineers, Performance Testers, Diagnostics resources & Application Experts. To be successful you should plan to build a Performance Testing team that doesn’t just execute but also knows how to identify relevant defects, tune the application/database and work with the development teams in coming up with fixes to relevant Performance issues.
Plan your resourcing model early on. To obtain a good balance of Performance Engineers & Performance Testers you will need a good head start. Obtaining Performance Engineering skills is generally tough and you might want to look at sourcing them from a niche consulting outfit.
Don’t wait till for SIT & UAT to complete for you to start planning for Performance Test. Performance Testing just before go live is the best way to ensure that your application fails to meet the documented NFR’s (if they even exist i.e.).

10. Pitfalls to avoid during Performance Testing
Ensure that you’ve got well defined and clearly articulated Non Functional Requirements. Your Non Functional Requirements should be signed of by business and program management.
Ensure that you’ve got well defined and clearly articulated Performance Testing Workload & Performance Testing Strategy. Your Workload & Strategy should be signed of by business and program management.
Ensure that you’ve got the appropriate tools for Performance Testing, Diagnostics & Performance Monitoring. Avoid recommending or using Open Source tool as far as Performance Testing is concerned. There are quite a few affordable tools (including cloud based solutions) for Performance Testing.
Push for an Performance Testing environment that’s equivalent to production. This will ensure that you don’t have to extrapolate performance of the application from your testing environment into the production environment.
Model application performance early on in the SDLC using modelling techniques described earlier. You can use a combination of analytical and simulation modelling techniques to validate application performance for the given combination of application architecture and infrastructure capacity.
Update your Business Workload for Performance Testing as your business grows and the environment business works in changes. Change in Business Workload would require updating of your NFR’s and rebase lining of your Workload for Performance Test. It changes your requirements and can have considerable impact on tool requirements, infrastructure requirements, resource requirements, etc. but is an important that that should be tracked through the life of the program.

11. Performance Testing Process
Document Non Functional Requirements
Document Performance Testing Workload
Document Performance Testing Strategy
Document tooling, licensing, data, environment requirements
Document resource requirements
Setup Performance Testing Team
Setup Environment for Performance Testing
Create Performance Testing Scripts
Create Data for Performance Testing
Setup Performance Testing Workload Scenarios within Tesing Tool
Setup Monitoring & Diagnostics tools
Execute Performance Tests
( A )
Collect test data from testing, monitoring & diagnostics tool
Identify performance bottlenecks & tune the application
Exit if we’ve met our NFRs else loop back to Step ( A )

12. Challenges involved in Performance Testing
Challenges convincing the delivery teams to focus on Performance Engineering & Performance Testing activities early in the SDLC
Challenges obtaining funding for the appropriate performance testing, monitoring & diagnostics tools
Challenges obtaining the right combination of resources to address your Performance Testing needs
Challenges obtaining clean, transformed and scaled data for your Performance Testing needs
Challenges getting access to Business SME for duration of Performance Test
Challenges getting access to the application SME’s during the course of the program
Challenges getting buy in from all the development and support teams to support Performance Testing activities
Challenges convincing the client to invest in Performance Testing infrastructure equivalent to production
Challenges due to lack of Non Functional Requirements
Challenges due to lack of understanding of Business workload

13. Deliverables – Performance Testing
Following would be the deliverables for Performance Testing -
Non Functional Requirements document (If not created during the initial PE phase)
Overall Business Workload to be processed by the application (If not created during the initial PE phase)
Overall Performance Testing approach
Tooling requirements
Data requirements
Resource requirements
Infrastructure requirements
Network requirements

14. Resources & Tools
HP Load Runner - http://www8.hp.com/us/en/software-solutions/software.html?compURI=1175451
IBM Rational Performance Tester - http://www-01.ibm.com/software/awdtools/tester/performance/
Microsoft Visual Studio Ultimate Edition - http://www.microsoft.com/visualstudio/en-gb/products
Webload - http://radview.com/
Neoload - http://www.neotys.com/
Load Storm - http://loadstorm.com/performance-testing-tool
Micro Focus Silk Performer - http://www.microfocus.com/products/silk/silkperformer/index.aspx
SOASTA Cloud Performance Test - http://www.soasta.com/

15. Thank You
Please support us by taking a moment and sharing this content using the Social Media Links at Practical Performance Analyst
trevor@practicalperformanceanalyst.com