STAMP (Software Testing Amplification) is a suite of tools to increase testing capacity and detect bugs. It includes Descartes, which tests test cases against mutants to evaluate effectiveness, and DSpot, which aims to improve code coverage and mutation score by generating new test cases. CAMP (Configuration Amplification) executes test cases against multiple configurations, and Evocrash generates test cases to reproduce bugs found in production logs. The tools can be used standalone, with IDEs like Eclipse, or integrated into build tools like Maven and Jenkins to amplify testing practices and help improve quality.

1. 1
STAMP
Test Amplification for DevOps
Pascal Urso, PhD (Activeeon)

2. 2
Scenario – how to have good tests?
Bugs detected by unit
tests
Bugs detected by IT
Etc
Etc
Vision: test pyramid & bug filter
How can I
improve it?
How can I
reproduce it?
How can I
automatise it?

3. 3
Idea – Software Testing AMPlification
l What about a software that produces test variants from your test cases?
l What about a software that tests your tests?
l What about a software that produces variants from your test configurations and execute
your tests against them?
l What about a software that reproduces a production bug simply analysing your production
logs?
l What about a test automation solution that let you emphasize test and quality in your
DevOps processes?

4. 4
Idea – Software Testing AMPlification
l A suite to increase your capacity to detect bugs
- Descartes (“test your tests”)
- DSpot (“amplify your tests”)
- CAMP (Configuration AMPlification)
- Evocrash (generate test case to reproduce bugs)
l Use it by the means of standalone shell scripts
l Available as Maven/Gradle plugins
l Use it within Eclipse/IntelliJ IDE
l Amplify your test practice

5. 5
Idea – Mutation
l A (extreme) mutation is a (extreme) change in your code
class A {
int field = 3;
public void Method(int inc) {
field += 3;
}}
•
• J.H. Andrews et al. 2006. Using mutation analysis for assessing and comparing testing coverage
criteria. IEEE Trans on Software Engineering 32, 8 (2006), 608–624.
• G. Petrovic et al. 2018. State of Mutation Testing at Google. In Proceedings of the International
Conference on Software Engineering - Software Engineering in Practice (ICSE SEIP).
Mutation
class A {
int field = 3;
public void Method(int inc) {
field -= 3;
}}
Extreme mutation
class A {
int field = 3;
public void Method(int inc) {
}}

6. 6
Idea – Descartes
Idea – DSpot
l Descartes
l Fitness : does the test kills a mutant ?
l I.e. does the test find bugs ?
l Result : your test is “good”
l (mutation score)
l DSpot
l Fitness : improved code coverage ?
l Or improved mutation score!
l Result : new tests !
Mutation
Fitness
evaluation
stop
?
mutants
source code
result
tests
Mutation
Fitness
evaluation
stop
?
mutants
test
result
source
code

7. 7
Idea – CAMP
Configuration AMPlification

8. 8
Scenario – You need an infra

9. 9
9
• DSpot Wizard
• Execute it and check results in
Eclipse Console
• Manage multiple configurations and
executions
How to – use DSpot within Eclipse

10. 10
10
• Create a freestyle job to run PIT
Descartes
• Add a post build action to
generate STAMP reports
• Execute it
• Check trends
How to – use Descartes within Jenkins

11. 11
11
• Execute your system test cases against multiple configurations
• Use it as standalone or into your continuous delivery pipeline
How to – execute CAMP

12. 12
12
• Feed Evocrash with your production logs
• … Evocrash generates a test case able to reproduce the bug : just fix it
• (saved 2 days to manually reproduce a crash : have a nice weekend :-))
How to – use Evocrash in
Gradle/Maven

13. 13
How to – now you can have this

14. 14
14
How to – try it with STAMP virtual lab

15. 15
Conclusions
l It’s not about “computers write software”: at least now, computers need
people as much as we need them
l Want to try it? Getting started for Dummies (see references)
l Want to contribute? Github repo (see references)
l Next steps:
- Optimizations and enhancements
- Microservices
- More Jenkins, Github and Gitlab integrations
- Metrics enhancements
- More use cases and experiments
- A lot of other stuff: stay tuned!

16. 16
References
l STAMP Official site: https://www.stamp-project.eu/
l STAMP code: https://github.com/STAMP-project/
l STAMP for dummies:
- https://github.com/STAMP-project/dspot/blob/master/docs/dspot-for-dummies.md
- https://github.com/STAMP-project/pitest-descartes/blob/master/docs/descartes-for-dummies-
mvn.md
- More tutorials are coming….
l The Practical Test Pyramid -
https://martinfowler.com/articles/practical-test-pyramid.html

17. 17
Thank you!
Questions?
Answers
Tweets?