Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Performance engineering10042915
1.
2. Enable Performance Testing through the entire development cycle to ensure adequate
performance of the code by the end of project
Define the acceptance criteria and definition of done or pass/fail of a story.
Build the performance testing into smaller cycles and call them unit performance testing
that will measure component level performance .
Build performance test cases in the tools like Jira or any other tool to enable visibility.
Ensure that the Agile team are confirming the release against the agreed set of
performance requirements.
Ensure continuous feedback among all stakeholders .
3. Groups Tasks
Define how work will be broken down into small
chunks.
Performance Team lead identifies performance
testing stories and make sure the acceptance
criteria is defined.
Resource Planning for sprint work.
Need to train the performance testers to adapt
to agile approach.
Tasks
Create performance test cases.
Create scripts run tests measure performance
and report performance
Create front end performance measurement
using web page test or equivalent .
Scrum
Level
Progra
m
Level
4.
5. Performance Activities
Create the performance test cases in shared repository for each sprint in parallel with
coding.
Work on unit level performance test of each module being developed .
Report the individual module performance to scrum team
Automate the tests explore the tools that will work better to include as part of CI
Work on getting the test data created for regression test
Environments
Can be performed in QA environment after the code has been QA certified
Tools
We can explore the tools like JunitPerf .
We can also include the load testing scripts as part of the CI build.
Resources
Agile revolves around principle of cross training and working more in collabaration so this can
be achieved by developers also to begin with once the performance engineer get ramped up.
6. Performance Activities
Create the performance test cases in shared repository for the hardening sprint.
Work on performance testing integrated components like services along with the UI
Report the performance of the integrated components.
Automate the tests and run the integrated tests(Single users tests) as part of CI To
track performance.
Continue to work on getting the test data created for regression test
We may have this sprint or might create integration user stories as part of next
sprint.
Environments
Can be performed in QA env after the code has been QA certified
Tools
Use load runner or Jeter be flexible with tool choices
Resources
Performance test engineer
7. Performance Activities
Fully integrate yourself into the Sprint team, immediately
Performance awareness, practices, and results organically increased because it was a well represented function within the team as
opposed to an
after thought farmed out to a remote organization.
Build multiple performance and stress test scenarios with distinct goals and execution schedules. Plan for longer soak and stress tests as
part of the release process, but have one or more per-sprint, and even nightly, performance tests that can be continually executed to
proactively measure performance, and identify defects as they are introduced. Consider it your mission to quantify the performance
impact of a code change.
Extend your Continuous Integration (CI) pipelines to include performance testing. At PayPal, I built custom integrations between Jenkins
and JMeter to automate test execution and report generation. Our pipelines triggered automated nightly regressions on development
branches and within a well-understood platform where QA and development could parameterize workload, kick-off a performance test
and interpret a test report. Unless you like working 18-hour days, I can’t overstate the importance of building integrations into tools
that are already or easily adopted by the broader team. If you’re using Jenkins, you might take a look at the Jenkins Performance Plugin.
Define Key Performance Indicators (KPIs). In an Agile model you should expect smaller scoped tests, executed at a higher frequency. It’s
critical to have a set of KPIs the group understands, and buys into, so you can quickly look at a test and interpret if a) things look good,
or b) something funky happened and additional investigation is needed. Some organizations have clearly defined non-functional criteria,
or SLAs, and many don’t. Be Agile with your KPIs, and refine them over time. Here are some of the KPIs we commonly evaluated:
Environments
Needs to be performed in pre prod after the code has been QA certified
Tools
Use load runner
Resources:
Performance Test engineer
8. Change Agile from a development practice to a delivery practice
Build a change-friendly framework
Bake performance testing into every iteration Performance awareness, practices, and
results will organically increase if it is a well represented function within the team as
opposed to an after thought
Automate as soon as possible
Extend your Continuous Integration (CI) pipelines to include performance testing
In an Agile model you should expect smaller scoped tests, executed at a higher
frequency. It’s critical to have a set of KPIs the group understands, and buys into, so
you can quickly look at a test and interpret if a) things look good, or b) something funky
happened and additional investigation is needed
Monitor performance trends per iteration
Monitor SLA performance per iteration