The document describes a three-step approach to improving defect yield in testing: 1. Conduct a potency assessment to determine which types of defects are being targeted by current test cases and if any important types are missing. 2. Perform potential defect type re-targeting to add new test cases to cover additional defect types identified as missing. 3. Enhance existing test cases through potency improvement to ensure they are complete in uncovering defects.

1. Improving defect yield -
A three step approach
Powered by HBT
T Ashok
Founder & CEO
STAG Software Private Limited
in.linkedin.com/in/AshokSTAG
Ash_Thiru

2. © 2013 STAG Software Private Limited.All rights reserved.
Defect Yield
2
Defect yield is the #defects (of good quality)
uncovered by the test cases
Yield = Outcome/Effort
Defect yield = #Defects / # TestCases

3. © 2013 STAG Software Private Limited.All rights reserved.
Expectation & Challenges
3
Expectation
Increasing the yield - a natural expectation.
Challenges
Defect yield drops/is-low :
- when system matures
- when system of testing is “weak”
- when testing is transitioned to a different team

4. © 2013 STAG Software Private Limited.All rights reserved.
Why Lower Yield ?
4
Are we wasting effort by testing stable parts?
Are we missing doing something else?

5. © 2013 STAG Software Private Limited.All rights reserved.
The 3-step approach to yield improvement
5
Potency assessment1
Potential defect type re-targeting2
Potency improvement3
What types of defects are being targeted by the test cases?
Are the test cases complete/adequate?
Are there any other types of defects to be targeted?
Add new test cases to cover the new types of defects
Enhance existing test cases to ensure that they are complete

6. © 2013 STAG Software Private Limited.All rights reserved.
Hypothesis Based Testing - HBT
6
System Under Test
Cleanliness Criteria
Potential Defect
Types
Test CasesRequirements
traceability
“what to test”
Fault
traceability
“test for what”
should satisfy impeded by

7. © 2013 STAG Software Private Limited.All rights reserved. 7
Quality Levels & PDTs - Powered by HBT
Input cleanliness
Input interface cleanliness
Structural integrity
Behaviour correctness
Environment cleanliness
Attributes met
Flow correctness
Clean Deployment
End user value
L1
L2
L3
L4
L5
L6
L7
L8
L9
That inputs are handled well
PDTs related to Input data correctness
That the functional behaviour is correct
PDTs related to Functionality
That the internal structure is robust
PDTs related to Structural aspects
That the user interface is clean
PDTs related to UI
That end-to-end flows work correctly
PDTs related to Flow behaviour, Interactions
That it does not mess up the environment
PDTs related to Resource leaks, Compatibility...
That the stated attributes are met
PDTs related to Performance, Load, Volume......
That it deploys well in the real environment
PDTs related to Compatibility, Migration
That user expectations are met
PDTs related to User flows, User experience

8. © 2013 STAG Software Private Limited.All rights reserved.
Potency Framework (For Assessment & Re-target)
8
QL9 End user value
End user value test
PDT 57-59
QL8 Clean deployment
Installation test
PDT 54-55
Migration test
PDT 56
Migration test
QL7 Attributes met
LSPS test
PDT 44-51
Reliability test
PDT 52
Reliability test
PDT 52
Security test
PDT 53
QL6 Environment cleanliness
Good citizen test
PDT 39-41
Compatibility test
PDT 42-43
Compatibility test
QL5 Flow correctness
Interaction test
PDT 35-38
QL4 Behaviour correctness
Functionality test
PDT 24-31
Access control test
PDT 32-34
Access control test
QL3 Structural integrity
Structural test
PDT 14-23
QL2 Input interface cleanliness
API validation test
PDT 5-7
GUI validation test
PDT 8-13
GUI validation test
QL1 Input cleanliness
Input validation test
PDT 1-4
Input validation test
QL - Quality Level
PDT - Potential Defect Type
PoweredbyHBT

9. © 2013 STAG Software Private Limited.All rights reserved.
An illustration : A component of a large system
9
Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1Component = C1
BeforeBeforeBeforeBeforeBefore ADDITIONS (After)ADDITIONS (After)ADDITIONS (After)ADDITIONS (After)ADDITIONS (After)
#PDT
s
#TCs#TCs#TCs #Defects '+PDTs '+TCs'+TCs'+TCs #Defects
Total '+ve '-ve Total '+ve '-ve
L8 0 0
L7 1 1 1 0 1 2 1 1
L6 0 0
L5 0 0
L4 7 1116 976 140 8 7 45 26 19 17
L3 1 135 130 5 0
L2 2 29 26 3 0
L1 3 23 9 14 0
Total 14 1304 1142 162 8 8 47 27 20 17
Increase byIncrease byIncrease byIncrease byIncrease by 57% 4% 2% 12% 213%
TC YieldTC Yield 0.01 Defects/TCDefects/TC TC YieldTC Yield 0.36 Defects/TCDefects/TC
Potency assessment PDT re-targeting21
3 Potency improvement

10. © 2013 STAG Software Private Limited.All rights reserved.
An illustration - The entire system
10
Potency assessment PDT re-targeting21
3 Potency improvement
Customer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : SummaryCustomer X : Summary
BeforeBeforeBeforeBeforeBefore ADDITIONS (After)ADDITIONS (After)ADDITIONS (After)ADDITIONS (After)ADDITIONS (After)
#PDTs #TCs#TCs#TCs #Defects '+PDTs '+TCs'+TCs'+TCs #Defects
Total '+ve '-ve Total '+ve '-ve
C1 14 1304 1142 162 8 8 47 27 20 17
C2 20 543 353 190 6 8 176 114 62 7
C2 6 741 494 247 19 7 17 12 5 0
Total 40 2588 1989 599 33 23 240 153 87 24
Increase byIncrease byIncrease byIncrease byIncrease by 58% 9% 8% 15% 73%
TC YieldTC Yield 0.01 Defects/TCDefects/TC TC YieldTC Yield 0.10 Defects/TCDefects/TC
10x yield improvement

11. © 2013 STAG Software Private Limited.All rights reserved.
HBT Overview
SIX staged purposeful activities, powered by EIGHT disciplines of thinking
11
D1
D2
D4
D3
D5D6
D7
D8
Core Concepts
S1
S2
S3S4
S5
S6
SIX Stages of DOING
S1: Understand expectations
S2: Understand context
S3: Formulate hypothesis
S4: Devise proof
S5:Tooling support
S6:Assess & Analyse
EIGHT Disciplines of Thinking
D1: Business value understanding
D2: Defect hypothesis
D3: Strategy & Planning
D4:Test design
D5:Tooling
D6:Visibility
D7: Execution & Reporting
D8:Analysis & Management
Uses 32 Core Concepts
For Problem solving
- Techniques, Principles, Guideline
Click here to know more about HBT.
http://stagsoftware.com/blog?p=570

12. © 2013 STAG Software Private Limited.All rights reserved.
HBT is the intellectual property of STAG Software Private Limited.
STEMTM is the trademark of STAG Software Private Limited.
www.stagsoftware.com
@stagsoft
blog.stagsoftware.com
Connect with us...
Improving defect yield -
A three step approach
Powered by HBT
Potency assessment1
Potential defect type re-targeting2
Potency improvement3
Thank you.