Exploratory Testing introduction

1. Exploratory Testing
Rail Signalling Solutions

2. Introduction
 The classic testing methods involve testers to create test cases and test
scripts using a test basis such as requirements and specification
documents (e.g. Doors), in a process known as ‘Scripted Testing’ (ST).
 ST led to excessive testing of easy cases, while failing to allow sufficient time
to creative testing of more interesting and complex cases
 ‘Exploratory testing’ (ET) is referred as a new approach
and was introduced by Cem Kaner
2 L905 Overview Presentation
Author: Codrin Pruteanu

3. About Exploratory Testing
“A style of software testing that emphasizes the
personal freedom and responsibility of the individual
tester to continually optimize the quality of his work
by treating test-related learning, test design,
test execution, and test result interpretation”
(Kaner C., 1983)
Keys:
- stimulates the cognitive engagement of the tester
- increases responsibility for managing time properly
3 L905 Overview Presentation
Author: Codrin Pruteanu

4. What is Exploratory Testing (ET) ?
 ET represents:

a technique for designing and executing better tests

simultaneous learning, test design and test execution

adapting the tests as they are performed
 ET does not specify:

how the test activities should be planned, managed and tracked

how the test environment should be set up

risk evaluation for safety critical applications

which tools should be used
4 L905 Overview Presentation
Author: Codrin Pruteanu

5. Exploratory Testing: Pro and Con
 ET – main advantages:
 encourages creativity, adaptability and intuition
 increases the chance of finding bugs in a shorter time period
 the tester is free to explore new areas as test ideas arise during test execution
 is useful when complete documentation is not always available
 is investigative rather than confirmative
 it’s more fun than ST 
 ET – disadvantages:
 depends heavily on the testing skills and domain knowledge of the tester
 when combined with ST, it involves a risk of redundant tests
 is not suitable for complex tests with a long execution time
 does not provide absolute assurance that the most important bugs have been found
5 L905 Overview Presentation
Author: Codrin Pruteanu

6. Exploratory Testing: case study
Complete Testing = ST + ET
ET
Quality
ST
Q
ua
nt
it y
6 L905 Overview Presentation
Author: Codrin Pruteanu

7. When to apply Exploratory Testing
 When ET should be used ?
 the test team includes experienced testers
 validate the work of a tested component (explore the tested feature)
 ‘smoke test’ is desired before starting with large-scale test execution
 isolate and investigate a particular defect
 in an early iteration (product not ready for ST)
 improve ST for safety critical applications
 a new tester enters the team (learning phase)
7 L905 Overview Presentation
Author: Codrin Pruteanu

8. Important Notes
 ET - important notes !
 Preparation, planning and management of test activities
cannot be conducted in an exploratory way!
 Traditional techniques should be used instead: documenting test plans,
test strategies and test summary reports
 ET is not Agile Testing !

AT is how the test activities are to be managed and tracked

ET is a technique for designing and executing tests

ET can be considered to be an agile approach to testing
8 L905 Overview Presentation
Author: Codrin Pruteanu

9. Success stories
 Success stories about companies which use ET:
1. Microsoft
 determine in a limited amount of time if an application works well
on a new version of Windows
 addresses both application functionality and stability
2. Adobe
 Improve the quality of software testing for their main line products on the market
2. Phillips
 Testing new features in safety-critical life-sustaining medical devices
3. Thales
 IM product test (Vlaicu Constantinescu)
 FEC test team (Mihai Niculita)
9 L905 Overview Presentation
Author: Codrin Pruteanu

10. Questions about Exploratory Testing
 Who Does Exploratory Testing?

Any software tester knowingly or unknowingly does it !
 What qualities do I need to posses to be able to perform ET?

the only limit to perform exploratory testing is your imagination and creativity

your ability to explore and create more test cases
 Why do I need to perform ET if I already use ST ?

ET is used to increase the probability of finding defects but doesn’t replace ST !
 What is my job as exploratory tester?

explore a component of the system

think about what should be tested

implement the appropriate tests

write down test results
10 L905 Overview Presentation
Author: Codrin Pruteanu

11. Conclusions
 Conclusions:
1. Many testers are trapped in comfortable ways of doing testing:
 The style of testing is often promoted as “professional”
 Sometimes the lack of experience may produce test results with poor coverage
 Some relatively unskilled testers may produce inefficient tests
 Quantity does not always means quality !
2. In order to achieve a higher product quality (less defects) a tester should:
 develop his creativity and maximize the value of test results
 use both testing techniques: ST and ET !
11 L905 Overview Presentation
Author: Codrin Pruteanu

12. The End
12 L905 Overview Presentation
Author: Codrin Pruteanu