This webinar was co-hosted by Xray and Curiosity Software on 18th May 2021. Watch the on demand recording here: https://opentestingplatform.curiositysoftware.ie/xray-in-sprint-testing-webinar In-sprint testing must tackle three pressing problems: 1. You must know exactly what needs testing before each release. There’s not time to test everything. 2. You need up-to-date and aligned test assets, including test cases, data, scripts and CI/CD artefacts. 3. Test teams must know what needs testing, when, and have on demand access to environments, tests and data. These problems are near-impossible to crack at organisations who struggle with application complexity, rapid system change, and overly-manual testing processes. Challenges include: 1. Test creation time. Manually creating test cases, data and scripts is slow and unsystematic, resulting in low coverage tests. 2. Slow test maintenance. Changes break tests, with little time in sprints to check test cases, scripts, and data. 3. Knowing when testing is “done”. There is little measurability or peace of mind when systems “go live”. This webinar will set out how maintaining a “digital twin” of the system under test prioritises testing time AND maintains rigorous tests in-sprint. You will see how: 1. Intuitive flowcharts generate optimised test cases, scripts, and data. 2. Feeding changes into the models maintains up-to-date tests. 3. Pushing the tests to agile test management tooling then makes sure that teams know which tests to run, when, with full traceability and a measurable definition of ‘done’. James Walker, Curiosity’s Director of Technology, and Sérgio Freire, Head of Product Evangelism for Xray, will set out this cutting-edge approach to in-sprint testing. Günther-Matthias Bär, Test Automation Engineer at Sogeti, will then draw on implementation experience to discuss the value of the proposed approach.

1. © Curiosity Software Ireland Ltd. 2021
In-Sprint Testing:
Aligning tests and teams to rapidly changing systems
May 18th 2021
James Walker
Director of Technology, Curiosity.
James.Walker@Curiosity.Software
@CuriositySoft
Sérgio Freire
Solution Architect, Xray
Sergio.Freire@Xpand-It.Com
Günther-Matthias Bär
Test Automation Engineer, Sogeti
Guenther-Matthias.Baer@Sogeti.Com

2. © Curiosity Software Ireland Ltd. 2021
Curiosity
• In 2017, Curiosity Software launched with
technologies and expertise built on decades of
experience.
• We’ve built test data solutions since 1995, when
our MD founded the first of 5 pioneering vendors.
• We specialise in requirements modelling, test data
automation, and model-based testing, and are
launching the first Open Testing Platform.
-Our global presence-
Curiosity locations

3. © Curiosity Software Ireland Ltd. 2021
Xray is the leading Quality Assurance and Test Management
app for Jira.
Improve the quality of your systems through effective and
efficient testing that runs through the entire software
development lifecycle.
Xray supports both manual and automated tests and provides
powerful reports to ensure full requirements coverage.
+ 5,000
Customers
Trust our products
4.5M
Users
100M
Monthly Test Cases
+ 70
Countries
With customers
137
of the Global 500
4 Languages
EN, DE, ES and FR
About Xray

4. Short Sogeti Presentation.pptx
About us
50+
Years of experience —
delivering expertise you
can count on
Local & global
Present in over 100 locations with global
presence in Europe, India, and the US
Made in Tech
Cutting-edge solutions in Digital Assurance
& Testing , Cloud, Cybersecurity , Analytics
& Cognitive Services – underpinned by
Automation and AI
Delivery
excellence
Agile delivery excellence through
DevOps and our global delivery
model, Rightshore®
Global
alliances
Strong global alliances with Microsoft,
IBM, Intel and Micro Focus
16,000
Sogetians with strong
technology expertise, part
of the Capgemini Group

5. © Curiosity Software Ireland Ltd. 2021
In-Sprint Testing:
Aligning tests and teams to rapidly changing systems
1. Common barriers to in-sprint testing
2. Aligning tests and teams for in-sprint testing
3. Demo: In-sprint testing with Xray and Test Modeller
4. Embedding in-sprint testing in enterprise architectures
5. Q&A and next steps

6. © Curiosity Software Ireland Ltd. 2021
Barriers to in-sprint testing
Why does it remain so hard to test and develop complex systems in-sprint?

7. © Curiosity Software Ireland Ltd. 2021
Poll: What are the main barriers to in-
sprint testing at your organisation?

8. © Curiosity Software Ireland Ltd. 2021
Barriers to in-sprint testing
Silo Wall Information Hop Delay
No feedback
or traceability
Dev Automation Engineers
QA/Test
BAs
Effort: I write up lots
of repetitive user stories,
requests and tickets.
Quality:Most bugs
originate in incomplete,
ambiguous requirements.
Effort: I recode after
misunderstood designs
and uncaught bugs.
Quality:Bugs are more
damaging the later they
are found by QA.
Effort: I slowly create
and maintain 1000s of
low-coverage tests.
Quality:Manually
derived tests hit a
fraction of system logic.
Effort: I maintain a
growing mountain of
repetitive test scripts.
Quality: Data clashes,
false positives, false
negatives, low coverage.
Formulate complex requirements
and change requests in
disparate, unconnected formats.
Convert the incomplete designs
into code, trying to fix the impact
of changes on complex systems.
Unsystematically create high
volume, low coverage test cases,
mismatched with system designs.
Choose between creating new
scripts for new logic or
maintaining existing code.

9. © Curiosity Software Ireland Ltd. 2021
5 Barriers to in-sprint testing
1. Repetitive test case creation remains manual (there’s never enough time!).
2. Manual test case design cannot achieve sufficient coverage for complex systems – there’s
not time to test everything!
3. Test data destabilises automation and creates bottlenecks.
− The average QA team spend 44% of their time searching, managing and generating data.1
4. Brittle tests need maintaining manually before every release.
5. Automation code might not be easily re-usable, over-working a team of skilled engineers.
6. Miscommunication creates frustrating and time-consuming rework.
1 Sogeti, Capgemini (2020), Continuous Testing Report 2020

10. © Curiosity Software Ireland Ltd. 2021
Barriers to in-sprint testing
KCT: Knowledge, Collaboration and Tooling
Tooling
• Lack of a good automation foundation (unit & integration test)
• Functional & non-functional testing
• Lack of a true pipeline, with many manual work
Collaboration
• Late testing
• No testing involvement while coding (e.g. during coding, PRs)
Knowledge
• Late clarification of stories
• Lack of understanding about the product and also about the risks

11. © Curiosity Software Ireland Ltd. 2021
Aligning tests and teams for
in-sprint testing
Removing common barriers to in-sprint testing

12. © Curiosity Software Ireland Ltd. 2021
Using flowcharts to drive in-
sprint testing
Recorded
tests activity
Test Cases
BPMN
diagrams
BDD
Scenarios
Business
Requirements
Application Scans
Reactive Model
of the system
Find / Make / Allocate Test Data
Run Results
Tests
New
Iteration
 Perfect sets of tests
 Fast Maintenance
 Reusable Components
 Move faster
 Deliver with confidence
Test Automation
Test Automation

13. © Curiosity Software Ireland Ltd. 2021
Test Modeller for Xray

14. © Curiosity Software Ireland Ltd. 2021
What happens when tests reach Xray?
Test, for each path, are auto-provisioned, with
all the flow steps, and linked to user stories;
results are reported against them.
• Use Xray as single source-of-truth for all
testing efforts, with greater coverage
• Analyse impacts on coverage
• Decide whether to make the release or not: “Is
it good enough?”

15. Demo
© Curiosity Software Ireland Ltd. 2021

16. © Curiosity Software Ireland Ltd. 2021
Embedding in-sprint testing
in enterprise architectures
How can we implement this approach within existing pipelines?

17. Architecture
Docker
Container
Web UI
Custom
application
RESTful Services
Database
Test data
Other Services

18. Architecture
Libraries
Docker
Container
Robot
Framework
Selenium
Request (REST)
Others
DB Libraries
Web UI
Custom
application
RESTful Services
Database
Test data
Other Services

19. Why Robot Framework?
 Its Open Source
 It can be used for test automation and robotic process automation (RPA)
 Its actively supported with many industry-leading companies
 It has an easy syntax utilizing human readable keywords
 Its capabilities can be extended by libraries implemented with Python or Java
 Its operating system and application independent

20. Test creation
Keyword driven
Behavior driven

21. Functional test tagging
Define new test case
[Documentation]
... The intention of this test is to determine the proper
execution
[Tags] XRAYID-123 Smoke CritLevel-3 Successful
• Smoke tests
• Level of criticality
• Cascading test cases
• Enable the communication with Xray
Tagging can be used to define…

22. Architecture
Libraries
RF Dashboard
Docker
Container
Robot
Framework
Continuous
Integration
VCS
Automated
Test Cases
Web UI
Custom
application
RESTful Services
Database
Test data
Other Services
Selenium
Others
DB Libraries
Request (REST)

23. Creating Pipelines

24. Logs, Reports and Stats

25. Architecture
Libraries
REST API
Test plan
Test execution
Tests
User
Stories
RF Dashboard
Test
Management
Docker
Container
Robot
Framework
Continuous
Integration
VCS
Filter Management
QDMLA
Filter Stream Generator
Filter Database
Ingest Processor
Egest Processor
Test results
Automated
Test Cases
Selenium
Others
DB Libraries
Request (REST)

26. Test Management in Xray
User Story • Describing the delivered
functionality
Tests • Generic tests able to communicate
with Robot Framework
• Defining a template for requirements
• Definition of test case path
• Provision of test data

27. Test Management in Xray
User Story • Describing the delivered
functionality
Tests • Generic tests able to communicate
with Robot Framework
Test
Repository
• Structuring the provided tests
• Defining a template for requirements
• Definition of test case path
• Provision of test data

28. Test Repository in Xray

29. Test Management in Xray
User Story • Describing the delivered
functionality
Tests • Generic tests able to communicate
with Robot Framework
Test
Repository
• Structuring the provided tests
Test Sets • Combining similar tests

30. Test Management in Xray
User Story • Describing the delivered
functionality
Tests • Generic tests able to communicate
with Robot Framework
Test
Repository
• Structuring the provided tests
Test Sets • Combining similar tests
Test
Executions
• Assembling tests for execution
Via API
Automatical…
• Creation and Test Executions
• Import of Test Results
• Linking to current Test Plan

31. Test runs

32. Test Management in Xray
User Story • Describing the delivered
functionality
Tests • Generic tests able to communicate
with Robot Framework
Test
Repository
• Structuring the provided tests
Test Sets • Combining similar tests
Test
Executions
• Assembling tests for execution
Test Plan • Sprint test planning

33. Test Plan Board

34. Architecture
Libraries
REST API
Test plan
Test execution
Tests
User
Stories
RF Dashboard
Test
Management
Docker
Container
Robot
Framework
Continuous
Integration
VCS
Test results
Automated
Test Cases
Web UI
Custom
application
RESTful Services
Database
Test data
Other Services
Selenium
Others
DB Libraries
Request (REST)

35. Architecture
Libraries
REST API
Test plan
Test execution
Tests
User
Stories
RF Dashboard
Test
Management
Docker
Container
Robot
Framework
Continuous
Integration
VCS
Test results
Requirements
to
Model
TestModeller
Model-based
test creation
Autom. Code Generation
Automated
Test Cases
Web UI
Custom
application
RESTful Services
Database
Test data
Other Services
Selenium
Others
DB Libraries
Request (REST)

36. Architecture
Libraries
REST API
Test plan
Test execution
Tests
User
Stories
RF Dashboard
Test
Management
Docker
Container
Robot
Framework
Continuous
Integration
VCS
Test results
Requirements
to
Model
TestModeller
Model-based
test creation
Autom. Code Generation
Automated
Test Cases
Web UI
Custom
application
RESTful Services
Database
Test data
Other Services
Selenium
Others
DB Libraries
Request (REST)

37. © Curiosity Software Ireland Ltd. 2021
Achieving in-sprint testing
1. Accelerators build flowcharts – flowcharts generate tests and data. No more repetitive test
case design, unnecessary scripting or test data bottlenecks!
2. Test optimisation focuses testing where it matters before the next release. Reduce test volume
and find bugs when they are quick and affordable to fix!
3. Xray tracks tests. Everyone knows what needs testing for testing to be truly “done”.
4. Updating the flowcharts generates rigorous test suites in-sprint. Don’t waste time updating
existing tests!
5. Flowcharts and Xray provide a shared source of truth. Cross-functional teams work in
parallel, without delays and rework caused by miscommunication.

38. © Curiosity Software Ireland Ltd. 2021
Questions and Discussion

39. © Curiosity Software Ireland Ltd. 2021
Carry on the conversation?
• Try Test Modeller. Visit Testmodeller.io for a free trial and online tutorials.
• Book a Test Modeller demo and discussion with a Curiosity expert.
• Follow Curiosity Software on LinkedIn and @CuriositySoft on Twitter for
resources and news.
• Visit GetXray.app to find out more about the leading Test Management app for Jira
and start your Free Trial.
• Follow Xray on LinkedIn and @XrayApp on Twitter for testing news and events.

40. © Curiosity Software Ireland Ltd. 2021
Thank you
James.Walker@Curiosity.Software
Sergio.Freire@Xpand-It.com
Guenther-Matthias.Baer@Sogeti.com