DaKiRy_QAStandUp_Діана Пінчук "Testing oracles"

1. Testing oracles
Diana Pinchuk,
QA Engineer

2. Agenda
• Testing Oracles
• Oracles classification
• Emotions as oracle
• Summary

3. Oracles
An oracle is a heuristic principle or mechanism by which we recognize a problem
An oracles is a method of generation of the expected result
And that, since they’re heuristics, oracles are fallible and context-dependent.
No oracle comes with a guarantee that it’s giving you the right answer. An oracle
can only point you to a possible problem.

4. Full and partial oracles
Full oracle - a mechanism which can provide expected result with 100% sure.
Partial oracle - a mechanism which cannot be fully reliable in providing expected
result, but requires less resources for usage.

5. Failures when using oracles
Mishit - an oracle didn’t find the problem when it exists
False alarm - an oracle found the problem which doesn’t exist

6. Oracles classification
FEW HICCUPS by J.Bach & M.Bolton
Doug Hoffman’s Approach
Your own oracles
Emotions and feelings

7. FEW HICCUPS
Oracle Description
History Correspondence to the previous versions
Image Correspondence to the Company image
Comparable products Correspondence to comparable products
Claims Correspondence to requirements (promises)
User expectations Correspondence to user’s expectations
Product Correspondence to other parts of the product
Purpose Correspondence to the product purposes
Statutes and Standards Correspondence to the statutes and standards

8. FEW HICCUPS
Oracle Description
Familiarity Discrepancy with similar problems
Explainability Correspondence to behaviour which can be explained
World Accordance to beliefs about the world

9. Doug Hoffman’s taxonomy
Oracle
Constraint oracle
Regression oracle
Self-verifying data oracle
Physical model oracle
Oracle
Calculation oracle
Inverse oracle
Reference program
Oracle
Business model oracle
Statistical model oracle
State model oracle
Interaction model oracle

10. Comparison of the both classifications
Areas of usage Strengths Weaknesses
FEW HUCCUPS Logging convincing
bug reports
Test design
Mnemonic
Flexibility
Not suitable for
autotests developing
Doug Hoffman’s
taxonomy
Test design
Developing of
autotests
Oracles are well
programmed
Concreteness
Descriptions are not
well structured

11. And more...
Emotions can be used as oracle
Tester’s emotions are like smoke detector

12. Summary
• An oracles is a mechanism of generation of the expected result
• 2 most popular oracles classifications are (FEW HICCUPPS) and Doug
Hoffman’s taxonomy
• Both classifications help us to detect defects and design our tests
• FEW HICCUPPS is successfully used for logging of convincing bug reports
• Doug Hoffman’s taxonomy can be used for developing autotests
• There are many ideas of the testing oracles usage. For example, you can use
your emotions

13. References
http://www.developsense.com/blog/2012/07/few-hiccupps/
http://www.developsense.com/blog/2011/09/the-cooking-detector/
http://kaner.com/?p=190
http://sqadays.com/talk/12389

14. Thanks, any questions?