A presentation about how testers add value to a project or organization, and the costs and value-detractors associated with testing.

1. The Value-Adding
Tester
How a Tester Adds Value to an Organization or Project
1
2013-12-13
PA1
Confidential

2. Introduction
▪ The Value-Adding Tester adds value to the project or
organization
▪ The Value-Adding Tester understand and reduces costs
▪ The Value-Adding Tester does not detract value
▪ To be a value-adding tester it is required to continuously
evaluate value and cost
▪ How does a tester add and detract value, and what are the
costs?
2
2013-12-13
PA1
Confidential

3. How to Add Value - Overview
Mandatory Tests
3
2013-12-13
PA1
Fixed Defects
Information to
Stakeholders
Development
Support
Confidential

4. Fixed Defects
▪ One of the major ways a tester can add value is by finding
defects in the product which are actually fixed at some point
during the product life cycle
▪ The value of a defect found can be quantified by estimating
how the defect would impact sales, and if the customer
returns the product because of the defect
4
2013-12-13
PA1
Confidential

5. Information to Stakeholders
▪ Another way to generate value for a tester is to provide
stakeholders with information and material for
tollgates, milestones and decision points
▪ The value is harder to quantify but can be divided into two
parts:
▪ Helping stakeholders taking informed decisions
▪ Allowing stakeholders to feel more confident in their decisions
5
2013-12-13
PA1
Confidential

6. Mandatory Tests
▪ In many cases different governing bodies require that
certain criteria are met for a product to be allowed to be sold
▪ Meeting these criteria often require mandatory tests that
must be executed according to strict requirements
▪ Specific customers can also have mandatory requirements
that must be tested to be able to sell to those customers
▪ The value a tester provides by executing these tests can be
quantified by comparing how much it would cost to
outsource this to an accredited outsourcing partner
6
2013-12-13
PA1
Confidential

7. Development Support
▪ A tester can also provide value by increasing the efficiency of
the development team
▪ This can be done by providing a robust, easy-to-use test
framework, or by creating and maintaining automated
integration and regression test suites which the developers can
use
▪ The tester can also offer support by helping with unit test plans
and strategies
▪ Yet another example could be pushing for and educating in
testability, which can also drive efficiency
7
2013-12-13
PA1
Confidential

8. How to Detract Value - Overview
Low Quality
Reports
8
2013-12-13
PA1
Irrelevant
Defects
Irrelevant
Information
Inefficient Test
Tools
Confidential

9. Low Quality Reports
▪ Testers should not create defect reports that
▪ Are hard to understand because of bad language
▪ Are duplicates of already existing defect reports
▪ Lack information that is critical to understand the defect
▪ Lack critical attachments such as log files or screen shots
▪ Etc.
▪ This will lead to increased report handling and analysis effort for developers
▪ If testers create reports which cannot be understood due to improper
structure, language, formatting, etc., this also costs additional analysis
effort for stakeholders
9
2013-12-13
PA1
Confidential

10. Irrelevant Defects
▪ When a tester reports a defect this has to be
analyzed, prioritized and handled in different ways
▪ If the defect is not fixed, and the information stored in the
defect is not used by stakeholders, submitting the defect
actually detracts value instead of adding
▪ Many irrelevant defects could however together provide
valuable data by revealing trends or problem areas
▪ The value detraction can be reduced by handling less
relevant defects differently than higher priority issues
10
2013-12-13
PA1
Confidential

11. Irrelevant Information
▪ When testers provide reports to stakeholders, the information
presented must be beneficial and help the stakeholders to take
required decisions
▪ Presenting data that is not relevant or even misleading is not
only not valuable, but can be very costly
▪ Burying key information in a mountain of data
▪ A report that states “99 % Pass Rate” as the main
conclusion, when the remaining 1% represents critical quality
issues in the product, will not help stakeholders make the
correct decisions
11
2013-12-13
PA1
Confidential

12. Inefficient Test Tools
▪ The cost that testers impose on stakeholders with inefficient
tools
▪ When testers use tools that impact others, such as
developers, project managers, or other stakeholders, in a
negative way, this detracts value
▪ One example could be inefficient test frameworks or
automation tools which cause developers to write inefficient
tests
▪ Another example could be a reporting tool which is difficult for
stakeholders to extract reports from, or the tool generates
reports which take time to analyze and understand
12
2013-12-13
PA1
Confidential

13. Costs - Overview
Test Design &
Execution
Test Planning
Test Tools &
Frameworks
13
2013-12-13
PA1
Test Reporting
Administrative
Overhead
Confidential

14. Test Design & Execution
▪ Designing, executing, maintaining and porting tests
▪ Execution Effort
Exploratory Test
Scripted Test
Automated Test
Scripted Test
Exploratory Test
▪ Design Cost
Automated Test
14
2013-12-13
PA1
Confidential

15. Test Planning
▪ Planning test activities and creating corresponding test
artifacts such as test plans
▪ Setting scope for different test activities
▪ Risk analysis as impact to scope selection
▪ Probability of failure
▪ Technical risk analysis
▪ Impact of failure
▪ Business risk analysis
15
2013-12-13
PA1
Confidential

16. Test Reporting
▪ Creating test reports
▪ Test result metrics
▪ Qualitative summaries/product stories
▪ Creating defect reports
▪ Analysis of automated test results
16
2013-12-13
PA1
Confidential

17. Test Tools & Frameworks
▪ Creating new tools and test frameworks
▪ Cost of buying test tools and frameworks
▪ Integrating new tools
▪ Maintaining tools and frameworks
17
2013-12-13
PA1
Confidential

18. Administrative Overhead
▪ How much time does the tester actually spent on
design, analysis, execution, tools and reporting, and how
much time does the tester loose to administrative overhead?
▪ Getting the right software artifacts for test, and understand
what to actually test
▪ Coordinating between testers, developers, and
organizations
▪ Non-value adding meetings, etc.
18
2013-12-13
PA1
Confidential

19. Net Value of a Tester
Net Value
19
2013-12-13
PA1
Value
Cost
Value Detraction
Confidential

20. How do we increase value and decrease
cost and value detraction?
▪ We want to maximize value gain, and minimize costs and
value detraction
▪ How we do this depends heavily on context
▪ However we can still give some general guidelines
▪ Of course there are many other ways to reduce costs and
add value, but these are some suggestions
20
2013-12-13
PA1
Confidential

21. How do we increase value and decrease
cost and value detraction?
▪ Evaluate test artifacts and remove the non-value adding
ones
▪ 10-minute Test Plan – don’t create extensive test plans that no
one actually uses or updates
▪ Very detailed scripted test cases cost more than they add value
most of the time
▪ Test Strategy documents that no one reads or uses should not
be created at all
▪ However the discussions which lead to the creation of the documents
are still valuable and important to have
21
2013-12-13
PA1
Confidential

22. How do we increase value and decrease
cost and value detraction?
▪ Evaluate tools among all different stakeholders to secure that
they are efficient
▪ Even if a tool is easy to use for some purposes, it could impose
large costs for other stakeholders to use
▪ A test administration tool that is easy to record data in, may be very
costly to generate reports from
▪ Evaluate the actual value gain of the automated test framework
▪ Don’t just calculate the execution effort saved
▪ Many different costs: Design, Analysis, Maintenance, Porting
▪ Actual gains: Defects Found, Mandatory Tests, Information and Support to
Developers
22
2013-12-13
PA1
Confidential

23. How do we increase value and decrease
cost and value detraction?
▪ Secure that the information and defects testers report is
what stakeholders actually need
▪ Defects must be fixed
▪ Information must be used
▪ Make sure that testers actually work with testing, and not
spend their time on everything else
▪ Let testers list what they spend their time on, and try to reduce
administrative overhead and let the testers actually work with
test
23
2013-12-13
PA1
Confidential

24. How do we increase value and decrease
cost and value detraction?
▪ Proper risk analysis to support scope selection
▪ Risk-based testing
▪ Should be used when the reduction in cost for test execution is
larger than the increase in cost for test planning that the risk
analysis adds
▪ Efficiency gain of risk-based testing is very dependant on the
complexity of the system under test
▪ Show the cost that the system complexity adds to testing – this
can drive actions to reduce the complexity of the system
24
2013-12-13
PA1
Confidential

25. Conclusion
▪ The Value-Adding Tester continuously evaluates what value is
added, what value is detracted, and the costs for providing this
value
▪ There are many ways to add value, but there are also many
ways to detract value and either way there is an associated
cost
▪ Understanding the values and costs is critical to become
efficient
▪ Understanding must be supported by actual metrics, and not
only gut feelings
25
2013-12-13
PA1
Confidential