Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Berlin QA - Performance Testing
1. Performance testing & beers
Berlin QA // Because blame doesn´t fix bugs
José Aracil // Operations Manager // Globe Testing
2. About us
We are fun!
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
3. 16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
About us Focussed on Software Testing and ALM
Knowledgeable and experienced staff
Innovation: in the forefront of our services
Obsessed with excellence and attention to
detail
4. 16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
Our services
Functional Testing Performance Testing Performance Management
Test Execution
(manual or automated)
Test management
Performance,
load,stress,
soak…
System
optimization
Availability
and
performance
Understand end
user experience
Proactive
problem
resolution
Foundation
Traceability & Control Vision & ReportingProcess Standardization
(i.e. Waterfall, Agile)
DevOps
Application Lifecycle Management
Training & Coaching Outsourcing & turnkey projects
Diagnostics
Consulting ServicesQA Tools Deployment
6. Application evolution
• Valuable, but growth is capped
• Costly, brittle, monolithic and proprietary
• Must change structure to evolve
• Next generations of SAP and Oracle based
on SOA
• SOA driving new custom applications and
legacy integrations
• New Web 2.0 models
• “Mashups” in the enterprise
New and Agile Business Capabilities
Shared Services / Composite Apps
Yesterday’s applications: Today’s applications:
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
7. The challenges of our modern world
Composite Applications
Service oriented applications
Modern, mobile applications
Agile & Modern business require
MODERN and FAST applications
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
8. 16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
The challenges of our modern world
Agile & Modern business require
MODERN and FAST applications
Application Unavailable
10x more performance risks
Functional defects
10. Performance
• Optimal application performance is in the
experience of the end user.
Why is my application not performing?
• Looks simple, if we optimize all layers in isolation,
the resulting system should perform.
Performance is in the eyes of the beholder
11. Performance
• Optimal application performance is in the
experience of the end user.
Why is my application not performing?
• Looks simple, if we optimize all layers in isolation,
the resulting system should perform.
Performance is in the eyes of the beholder
12. Performance
• Optimal application performance is in the
experience of the end user.
Why is my application not performing?
• Looks simple, if we optimize all layers in isolation,
the resulting system should perform.
Performance is in the eyes of the beholder
13. Probably a MEMORY leak…
or may be a CODE issue…
I guess it is NETWORK!
Does your application have
performance issues?
14. It is probably a memory
leak issue!
No, it may be a CODE
issue!
I guess is Network
related!
We are not ready
for the 100, 1000,
100000 users!
We SHOULD have
TESTED
Performance!
15. How HP LoadRunner Works
5 Analysis, diagnostics and reporting
DatabaseInternet/WAN
Middle Tier
(HTTP, WebSphere, Server, Network)ApplicationEnd user
Thousands of users
are simulated
Transaction recording
“Transfer
Funds”
“Check Balance”
“Pay Bills”
Scripting1
Test
Scenario
Test planning2
Load generation3 Monitorization4
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
16. Planning
Recipe for success
Test Strategy
•Set expectations
•Project planning
•Environment description
•Responsible people
•Test scenario definition
•Test case definition
•Monitors to be used
•“Live” document
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
17. Planning
Preparation
Test Env. + Data
Recipe for success
Test Strategy
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
•Env. Setup (pre-prod)
•Data load to
resemble prod.
18. Planning
Preparation
Tool setup
Recipe for success
Test Strategy
Test Env. + Data
•Performance testing
tools
•Monitorization tools
•Other tools
(reporting,…)
16/08/2013
20. Planning
Preparation
Monitor config.
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
•Select any
“interesting” metrics
that should be
monitored
•Identify advanced
metrics
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
21. Planning
Preparation
Scripting
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
Monitor config.
•Turn test cases into
automated scripts
•Complexity changes
depending on the tool
used
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
22. Planning
Preparation
Smoke Test
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
Monitor config. Scripting
•Check that previous
phases have been
done properly
•It doesn´t add value
to carry on if there are
pending items
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
23. Planning
Preparation
Execution
Baseline
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
Monitor config. Scripting
Smoke Test
•Obtain a standard for
comparison
•If performance is not
good, it doesn´t add
value to carry on with
other test
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
25. Planning
Preparation
Execution
SOAK Test
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
Monitor config. Scripting
Smoke Test
Baseline
Performance
Testing
Análisis+Changes+
Report
Other tests
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
Load Testing
26. Planning
Preparation
Execution
Stress
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
Monitor config. Scripting
Smoke Test
Baseline
Soak Test Load Testing Other tests
Performance
Testing
Análisis+Changes+
Report
•Optional test
•Only if previous test
are successful
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
27. Planning
Preparation
Execution
Reporting
Final report
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
Monitor config. Scripting
Smoke Test
Baseline
Soak Test Load Testing Other tests
Stress
Performance
Testing
Análisis+Changes+
Report
•Summary of the test
executed, problems
found and solutions
applied
•Too much detail
doesn´t add value to
the project
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
28. Planning
Preparation
Execution
Reporting
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
Monitor config. Scripting
Smoke Test
Baseline
Soak Test Load Testing Other tests
Stress
Final report
Performance
Testing
Análisis+Changes+
Report
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
29. Planning
Preparation
Execution
Reporting
Final reportStressSOAK TestPerformance TestingBaselineSmoke TestScriptingMonitor config.Test case designTool setupTest Env. + Data
•Transactions
•Image and text
checks
•Test data
•Iterative process
•Minimum 2 iterations
Recipe for success
Test Strategy
Tool setupTest Env. + Data Test case design
Test Strategy
Monitor config. Scripting
Smoke Test
Baseline
Soak Test Load Testing Other tests
Stress
Final report
Performance
Testing
Análisis+Changes+
Report
•Set expectations
•Project planning
•Environment description
•Responsible people
•Test scenario definition
•Test case definition
•Monitors to be used
•“Live” document
•Performance testing
tools
•Monitorization tools
•Other tools
(reporting,…)
•Select any
“interesting” metrics
that should be
monitored
•Identify advanced
metrics
•Turn test cases into
automated scripts
•Complexity changes
depending on the tool
used
•Check that previous
phases have been
done properly
•It doesn´t add value
to carry on if there are
pending items
•Obtain a standard for
comparison
•If performance is not
good, it doesn´t add
value to carry on with
other test
Other tests
•Optional test
•Only if previous test
are successful
•Summary of the test
executed, problems
found and solutions
applied
•Too much detail
doesn´t add value to
the project
16/08/2013Confidential and Propietary // All rights reserved // Globe Testing
Load Testing•Env. Setup (pre-prod)
•Data load to
resemble prod.
30. Thank you for your time
José Aracil // jose.aracil@globetesting.com
Confidential and Propietary // All rights reserved // Globe Testing
Notas del editor
SDLC – Systems Development Life Cycle
Organizations have spent a decade or more automating their business through software. Most of the architecture was very siloed, monolithic and proprietary making it hard to maintain, change, or integrate which meant there was limited innovation. Not all have the supporting technology, systems and management processes to enable success throughout the lifecycle.Today we’re on the threshold of a massive refresh of Enterprise business applications. The next generation of SAP and Oracle applications will be based on SOA, which fundamentally requires a new architectural model. SOA adoption is also driving new custom applications and legacy integrations. And new Web 2.0 models like Ajax are leading customers to not only refresh web application and their web presence, but to build completely new models.All of these dynamics allow organizations to integrate and innovate business processes in a way they’ve never been able to do before. However, they need the right processes, the right products and the right management disciplines to realize the promise of agility and innovation while increasing quality and reducing cost.
Purpose of Slide:The world is changing. Mobile, Cloud, Composite Applications trends are all combining to increase speed of delivery and also the complexity of the applications.We expect applications to be available wherever we are and when ever we want themWe DEPEND on applications to complete our daily work
Purpose of Slide:When performance issues emerge – it’s not good for the end user. In many cases, the end user cannot access the application at all. They may have to re-enter data and re-do the last step and will inevitably be frustrated and unhappy
Purpose of Slide:This slide and the next slide are the elevator pitch… How many times have applications gone into production to only experience unexpected performance issues?Often performance testing is short cut, and to make matters worse, the production environment is almost always different from the non-production environment. These questions should resonate with IT leaders who will certainly remember a project that went live..only to go straight into problem management to figure out what is going wrong.The key is that performance issues often cause pain and expensive rework for IT organizations – often with expensive business impacts.
The key is that performance issues often cause pain and expensive rework for IT organizations – often with expensive business impacts.