Spring Boot vs Quarkus the ultimate battle - DevoxxUK
Fundamentals of testing 1
1. FUNDAMENTALS OF TESTING
Hoang V. Nguyen
startnewday85@gmail.com
startnewday85.blogspot.com
Dept. of Computer Science – FIT - HUA
Tester Training Course
11. What Failures’ effects?
Can harm:
• people
• companies
• the environment
Can lead to:
• Loss of money
• Loss of time
• Loss of business reputation
• Injury
• Death
Tester Training Course 2011
16. Why do defects occur in products?
Made by human
• Who know some thing, but not everything
• Who have skills, but aren’t perfect
==> Who do make mistakes(errors)
Under pressure
• deadline
• complexity ( system , organization)
• individual reasons
==> no time to check
==> systems may be incomplete
==> increased frequency of errors
Tester Training Course 2011
17. Reliability and Defect
Reliability: the probability that software will not
cause the failure of the system for a specified time
under specified conditions
Can a system be defect-free?
Can a system be reliable but still contains defects?
Is a ”defect-free” system always reliable?
Tester Training Course 2011
18. Error/Mistake
a human action which produces an
incorrect result
Defect/Fault/Bug
a manifestation of an error in
software
Failure
deviation of the software from its
expected delivery or service
Tester Training Course 2011
19. Error – Defect – Failure
make
Failure is an event; defect is a state of the software,
caused by an error
Tester Training Course 2011
20. When do defects arise in SD context?
Mistakes
Correct Correct Correct
made in
requirement requirement requirement
requirement
Designed to Designed to Mistakes Designed to
meet meet make in meet
requirement requirement design requirement
Built to meet Mistakes Built to meet Built to meet
design make in built design design
Product Wrong
Product has Product has
works as product
bugs in it design flaws
expected delivered
Tester Training Course 2011
21. What does It cost to fix a defect?
Tester Training Course 2011
22. Question
A bug or defect is:
A mistake made by a person;
A run-time problem experienced by a user;
The result of an error or mistake;
The result of a failure, which may lead to an error?
Tester Training Course 2011
23. Question
When what is visible to end-users is a deviation
from the specific or expected behavior, this is
called:
An Error
A fault
A Failure
A defect
A mistake
Tester Training Course 2011
25. Question
Debugging is:
Testing/checking whether the software performs
correctly;
Checking that a previously reported defect has been
corrected;
Identifying the cause of a defect, repairing the code
and checking the fix is correct;
Checking that no unintended consequences have
occurred as a result of a fix
Tester Training Course 2011
27. Why is testing necessary?
Because software is likely to have faults
To learn about the reliability of the software
To fill the time between delivery of the software and the release
date
To prove that the software has no faults
Because testing is included in the project plan
Because failures can be very expensive
To avoid being used by customers
To stay in business
Tester Training Course 2011
28. Testing and Quality
Quality: get the satisfaction of all stakeholders
Time
Budget
Quality
Satisfy customers
Technically excellent
Testing help us to measure and improve the quality
of software
• give confidence in the quality of software
Tester Training Course 2011
29. What is testing?
The process consisting of all lifecycle activities, both static and
dynamic, concerned with planning, preparation and evaluation
of software products and related work products
to determine that they satisfy specified requirements, to
demonstrate that they are fit for purpose and to detect defects
This is a suitable definition of testing for any level of testing.
Tester Training Course 2011
32. “Test everything”
Avr. 4 menus
3 options / menu
system has Average: 10 fields / screen
20 screens 2 types input / field
(date as Jan 3 or 3/1)
(number as integer or decimal)
Around 100 possible values
Total for 'exhaustive' testing:
20 x 4 x 3 x 10 x 2 x 100 = 480,000 tests
If 1 second per test, 8000 mins, 133 hrs, 17.7 days
(not counting finger trouble, faults or retest)
Tester Training Course 2011
33. Question
What is exhausive testing?
When all the testers are exhausted;
When all the planned tests have been executed;
Exercising all combinations of inputs and
preconditions
Tester Training Course 2011
34. Question
How much time will exhaustive testing take?
Infinite time;
No much time;
Impractical amount of time.
Tester Training Course 2011
35. How much testing is enough?
Testing principle
Exhaustive testing is impossible
Testing everything (all combinations of inputs and
preconditions) is not feasible except for trivial
cases. Instead of exhaustive testing, we use risks
and priorities to focus testing efforts.
Tester Training Course 2011
36. How much testing is enough?
It’s never engough;
When you have done what you planned;
When your customer/user is happy;
When you have proved that the system works correctly;
When you are confident that the system work correctly;
It depends on the risk for your system.
Tester Training Course 2011
38. RISK and TESTING
The number of test depends on RISK
• RISK of missing important faults
• RISK of incurring failure costs
• RISK of releasing untested or under-tested software
• RISK of losing credibility and market share
• RISK of missing a market window
• RISK of over-testing, ineffective testing
use RISK to determine:
• what to test first
• what to test most
• how thoroughly to test each item
• What not to test (this time)
use RISK to:
• Allocate the time available for testing by prioritising
testing ...
Tester Training Course 2011
39. When can we meet our test objectives?
Testing principle
Early testing
Testing activities should start as early as possible
in the software or system development life cycle
and should be focused on defined objectives.
Tester Training Course 2011
41. Is the software defect free?
Testing principle
Testing shows presence of defects
Testing can show that defects are present, but
cannot prove that there are no defects. Testing
reduces the probability of undiscovered defects
remaining in the software but, even if no defects
are found, it is not a proof of correctness.
Tester Training Course 2011
42. How are defects distributed?
Testing principle
Defect clustering
A small number of modules contain most of the
defects discovered during pre-release testing or
show the most operational failures.
Tester Training Course 2011
43. The defect clusters change over time
Testing principle
Pesticide paradox
If the same tests are repeated over and over again,
eventually the same set of test cases will no longer find any
new bugs. To overcome this 'pesticide paradox', the test
cases need to be regularly reviewed and revised, and new
and different tests need to be written to exercise different
parts of the software or system to potentially find more
defects.
Tester Training Course 2011
45. Testing principle
Absence-of-errors fallacy
Finding and fixing defects does not help if the
system built is unusable and does not fulfill the
users' needs and expectations.
Tester Training Course 2011
46. Testing principle
Testing is context dependent
Testing is done differently in different contexts.
For example, safety-critical software is tested
differently from an e-commerce site.
Tester Training Course 2011
48. Question
Which statement is most true?
Different testing is needed depending upon the
application.
All software is tested in the same way.
A technique that finds defects will always find
defects.
A technique that has found no defects is not useful.
Tester Training Course 2011
49. Question
When is testing complete?
When time and budget are exhaused.
When there is enough information for sponsors to
make an informed decision about release.
When there are no remaining high priority defects
outstanding.
When every data combination has been exercised
successfully.
Tester Training Course 2011
50. • Why testing is necessary?
• Failure, Fault, Bug, Defect, Error, Mistake
• What is testing?
• 7 general testing principles
• Testing and Reliability
• Testing and Quality
• Testing and Debugging
Tester Training Course 2011