We run a k€ 21,752 EU project to push the test automation research front. This talk motivates why this is (tax) money well spent and presents some research highlights: 1) test result visualization, 2) mutation testing, and 3) AI-assisted bug assignment.
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Test Automation Research... Is That Really Needed in 2018?
1. Research Institutes of Sweden
TEST AUTOMATION
RESEARCH… IS
THAT REALLY
NEEDED IN 2018?
@mrksbrg
mrksbrg.com
Markus Borg
Swedish Institute of Computer Science
3. Development engineer, ABB, Malmö, Sweden 2007-2010
▪ Editor and compiler development
▪ Safety-critical systems
PhD student, Lund University, Sweden 2010-2015
▪ Machine learning for software engineering
▪ Bug reports and traceability
Senior researcher, RISE AB, Lund, Sweden 2015-
▪ Software engineering for machine learning
▪ Software testing and V&V
3
Who is Markus Borg?
6. Problem Statement
▪ Contemporary dilemma. Modern software
teams must optimize for both:
▪ bug free software
▪ ease of change
▪ Ever-faster release cycles => more automation
6
Project Goal
▪ Help software teams to increase the
development speed without sacrificing quality
▪ Advance the state-of-the-art in test
automation
13. ▪ Why game engine?
▪ Interaction out-of-the-box
▪ Why Unity?
▪ Fairly simple
▪ Scales well
▪ Very popular
▪ Unity???
▪ Cross-platform game engine and IDE
▪ Drag-and-drop 2D and 3D scenes
▪ Scripting in C#
13
25. 25
Solution Approach
Toward mutation testing in the cloud
Sten Vercammen
▪ Goal
▪ Make mutation testing fast enough to fit during nightly build
▪ Approach
▪ Distribute the work
▪ Investigate bottlenecks and recommend optimizations
26. 26
Lessons learned from proof-of-concept tool
▪ Most steps are independent
▪ Speed-ups of 12x-13x with 16
workers
▪ Good chance to finish during
nightly builds
35. ▪ In line with human activity
– But instantaneous!
35
Results
36. ▪ Productification of solution in internal tool
▪ Simplified solution without ensemble
▪ Deployed in bug tracker for large project
▪ Presents instantaneous recommendation of responsible team
▪ Accuracy 8% lower than manual work
36
Prototype deployed at Ericsson