2. TESTING COSTS
• For unit tests a raw estimate evaluate to an increment of development times
between 10 and 30%
• Usually the reduction in issues is between 40 and 90%
• Summing up there is a reduction of nearly 50% in development costs
3. TOPICS
• Unit tests
• Code Quality
• Safe Refactoring patterns
• Components integration tests
• Code coverage
• Why unit tests are a waste (and confutation)
• References
4. UNIT TESTING
• Needed following a more “DevOps” like approach
• Help tracing bugs, features and code improvements
• Increase confidence in the codebase
• Contribute to a better micro-architecture
• Increase maintainability
5. UNIT TESTING: BUGS HANDLING
• Every bug that involves the source code must, first, be reproduced inside a failing
unit test
• Every bug must be shipped with
• Actions to reproduce the bug (even from an End To End perspective)
• Current (wrong) outcomes
• Expected (correct) outcomes
• Every source code bug should produce a corresponding unit test named
[PRODUCT_ID]_[BUG_ID]
6. UNIT TEST: NEW FEATURES
• Analyst and/or Test team must ship some example data. This is even reinforced when
dealing with bare algorithms
• For every feature a directory will be created [PRODUCT_ID]_[FEATURE_ID], that will
contain all the unit tests for the given feature
• Inside the tests directory a test must exist for each non trivial class implementation in
the form [ClassName]Test
• Non trivial implementations consit in (but not limited to)
• Requirements (even trivial)
• Every artifact not mentioned in requirements that contains logic
7. UNIT TEST: CORRECTNESS
• You can't write any production code until you have first written a failing unit test.
• You can't write more of a unit test than is sufficient to fail, and not compiling is
failing.
• You can't write more production code than is sufficient to pass the currently
failing unit test.
8. UNIT TEST: MICRO-ARCHITECTURE
• A testable class is a simple class (SOLID and DRY)
• A class that can’t be tested should be refactored to allow the testing
• Every tested/testable class must implement an interface.
• Every testable class must depend on interfaces. Never on implementation
9. UNIT TESTS: EXTRA THOUGHTS
• Every request written on the Requirements must have a corresponding test, even
if the implementation seems trivial. Those are the “business rules” and are of the
uttermost importance
• Every non trivial algorithm, functional to the implementation but not explicitely
specified in requirements must be covered by unit tests
• Getters and tester must not contain any business logic
• Unit tests must run on a computer even when disconnected from the network
and without any external server (DB,http etc) …or they become integration tests
10. UNIT TEST: BOY SCOUT RULE
• What happens when dealing with old implementations
• The functionality should be extracted, keeping the algorithms unchanged. The
refactoring rules specified further in this presenetation fall in this category
• When it is not possible to change the constructor a method must be offered to
inject the mocks in the class (see Emergency Refactoring later on)
• If the class become too complicated a specific refactoring task must be issued to
the development team
11. CODE QUALITY: INTRODUCTION
• Unit tests implementation is facilitated by the programming best practices.
Becomes kind of “natural” for well written code
• Unit tests should become the drivers to push developers towards better coding
and a better product
• Code quality principles will become a facilitator for the unit tests implementation
12. CODE QUALITY
• Maximum classes length: 1000 lines
• Maximum functions length: 40 lines
• Maximum function arguments: 3
• Maximum constructor arguments: 4
• Every class should have a clear and bounded role(Single Responsibility Principle)
• No logic should be contained in getters and setters
• Method names must be self-explanatory and not ambiguos
13. REASONS: DIMENSIONS AND METRICS
• Push towards simpler methods and classes
• Reduce the function lengths to “screen size” to increase readability
• Limit unit tests and mocking complexity
• Many parameters smells like too much responsabilities to the class
• Less paramaters in functions and constructors lead to simpler and more focused
implementations
14. REASONS: SRP
• Every class should have a specific role
• Keeps the modifications confined in a small context
• Increases Maintainability
• Improve reuse, small “one role” classes are far more flexible
• Therefore reduces the code size
15. COMMON REFACTORING PATTERNS-1
• Extract Method: Turns a code fragment into a method whose name describes its
purpose. Helps improve the callers' readability, and may help reduce duplication.
• Inline Method: Replaces calls to a method by copies of its body. Brings code into
one place in preparation for other refactorings.
• Rename Method: Can help client code become more readable. Often used after
other refactorings if the method's responsibilities are now better understood.
• Introduce Explaining Variable: Save the result of (part of) a complicated
expression in a temporary variable. Helps to improve an algorithm's readability.
16. COMMON REFACTORING PATTERNS-II
• Inline Temp: Replace uses of a temporary variable by its value. Often used to bring
code together in preparation for other refactorings such as Extract Method.
• Add Parameter: Passes extra information into a method. Often used during larger
code restructuring.
• Remove Parameter: Removes a parameter that is no longer used by a method. Helps
to keep the method's interface clean and readable for client code.
• Extract Class: A class has too many responsibilities, so you split out part of it. The new
class's name contributes to the code's domain language; testability and reuse may
also be improved.
17. COMMON REFACTORING PATTERNS-III
• Introduce Parameter Object: When several methods take the same bunch of
parameters, creates a new object that represents the bunch. This is another way
of finding new classes and thus new domain concepts.
• Replace Magic Number with Symbolic Constant: Gives a name to a literal
number. Improves code readability, and can help to identify subtle dependencies
between algorithms.
19. COMPONENTS INTEGRATION TESTS (CIT)
• Always a kind of unit test but with a larger scope, usually to verify the
coordinators
• Require usually a more complex initialization, that can be composed of several
steps
• These must be descriptive tests, a proposal is the Gherkin Syntax
• And can fall into BDD testing practices
• They do not access any real resource or server
20. CIT: BDD, INTRO AND TEST TEAM
• A tool proposal is Cucumber for Java(https://cucumber.io/ )
• The idea is to let tester specify the preconditions for the test in a clear and
parametrized way with minimal support from developers
• Test are written by tester in Gherkin language on simple text files (.feature
extension)
• GIVEN, WHEN, THEN
• Tester must be parametrized directly by testers
21. CIT: BDD, DEVELOPMENT SIDE
• The developers running the Gherkin scripts generate the stubs for the various
phases.
• Every Gherking line is associated with a parametrized function
• The functions must be filled by the developers. E.G. when the line tells that a line
is insert, the DAO mock should be initialized
• The functions and the parameters are then adapted with parameters based on
regular expressions and tables of possible values are added
22. CIT: BDD, TECHNICALITIES
• The spring context will be filled directly through the GIVEN functions, in a
programmatic way to keep the scope limited (see
https://stackoverflow.com/a/14716542 )
• The various functions can be composed to create more complex ones, e.g. several
given can be coordinated to produce a bigger stratup
• When tester needs a new verification they can simply pick between the already
prepared functions and add the text (.feature) file with the new ordered GIVEN
WHEN THEN
23. WHY MOST UNIT TESTING IS WASTE
• Delete tests that haven’t failed in a year. James argues that unit tests that
haven’t failed in a year provide no information, and can be thrown out. But if a
unit test fails, it fails as you are developing the code. It is similar to a compilation
failure. You fix it immediately. You never check in code where the unit tests are
failing. So the tests fail, but the failures are transient.
24. WHY MOST UNIT TESTING IS WASTE
• https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-Waste.pdf [4.pdf]
• Keep regression tests around for up to a year — but most of those will be system-level tests rather than unit tests.
• Keep unit tests that test key algorithms for which there is a broad, formal, independent oracle of correctness, and for which there is
ascribable business value.
• Except for the preceding case, if X has business value and you can text X with either a system test or a unit test, use a system test — context
is everything.
• Design a test with more care than you design the code.
• Turn most unit tests into assertions.
• Throw away tests that haven’t failed in a year.
• Testing can’t replace good development: a high test failure rate suggests you should shorten development intervals, perhaps radically, and
make sure your architecture and design regimens have teeth
• If you find that individual functions being tested are trivial, double-check the way you incentivize developers’ performance. Rewarding
coverage or other meaningless metrics can lead to rapid architecture decay.
25. WHY MOST UNIT TESTING IS WASTE-RESPONSE
• Delete tests that haven’t failed in a year. James argues that unit tests that
haven’t failed in a year provide no information, and can be thrown out. But if a
unit test fails, it fails as you are developing the code. It is similar to a compilation
failure. You fix it immediately. You never check in code where the unit tests are
failing. So the tests fail, but the failures are transient.
• Complete testing is not possible. In both the original and follow-up article,
James talks about how it is impossible to completely test the code. The state-
space as defined by {Program Counter, System State} is enormous. This is true,
but it applies equally to integration testing, and is thus not an argument against
unit testing.
26. WHY MOST UNIT TESTING IS WASTE-RESPONSE
• We don’t know what parts are used, and how. In the example of the mapin the follow-up
article, James points out that maybe the map will never hold more than five items. That may be
true, but when we do integration testing we are still only testing. Maybe we will encounter more
than five items in production. In any case, it is prudent to make it work for larger values anyway.
It is a trade-off I am willing to make: the cost is low, and a lot of the logic is the same, whether
the maximum usage size is low or high.
• What is correct behavior? James argues that the only tests that have business value are those
directly derived from business requirements. Since unit tests are only testing building blocks, not
the complete function, they cannot be trusted. They are based on programmers’ fantasies about
how the function should work. But programmers break down requirements into smaller
components all the time – this is how you program. Sometimes there are misunderstandings,
but that is the exception, not the rule, in my opinion.
27. WHY MOST UNIT TESTING IS WASTE-RESPONSE
• Refactoring breaks tests. Sometimes when you refactor code, you break tests.
But my experience is that this is not a big problem. For example, a method
signature changes, so you have to go through and add an extra parameter in all
tests where it is called. This can often be done very quickly, and it doesn’t happen
very often. This sounds like a big problem in theory, but in practice it isn’t.
• Asserts. James recommends turning unit tests into asserts. Asserts can be useful,
but they are not a substitute for unit tests. If an assert fails, there is still a failure in
the production system. If it assert on something that can also be unit tested, it is
better to find the problem when testing, not in production.
28. CODE COVERAGE
• Coverage must not be calculated on getter, setter and possibly constructors. All these
kind of methods must not contain any true logic
• The proposed framework is JaCoCo, easy to integrate with Maven, Jenkins and
Atalassian Bamboo
• The coverage can be pushed up to 60% BUT
• Should be adapted to the context
• May be even less but it must cover the functions carrying business value
• THE COVERAGE MUST NOT BE A FIXED METRIC, or some developer can try any kind
of trick to reach the coverage (done that, been there!!).
29. GOALS BY ROLE (OTHER THAN STANDARD ONES!)
• Analysts
• Propose sample data for each UseCase/Story/Algorithm
• Tester
• Define context, expected and wrong outcomes for every bug founded
• Write down the Gherkin scripts to ease regression and integration tests
• Developers
• Create unit tests for bugs and features
• Implement the Gherkin scripts for the testers
• Progressively refactor the old code (unit testing it!)
• Verify the coverage extension and meaningfulness
• Architects/Techinal Leaders/Coaches
• Support all teams to reach the variouos goals
30. REFERENCES
• An Initial Investigation of Test Driven Development in Industry, 2003
https://collaboration.csc.ncsu.edu/laurie/Papers/TDDpaperv8.pdf
• Realizing quality improvement through test driven development: results and experiences of four
industrial teams, 2008 https://people.engr.ncsu.edu/gjin2/Classes/591/Spring2017/case-tdd-
b.pdf
• Guest Editors' Introduction: TDD: The Art of Fearless Programming, 2007
https://www.computer.org/csdl/mags/so/2007/03/s3024.html
• Effects of Test-Driven Development: A Comparative Analysis of Empirical Studies, 2014
http://www.sserg.org/publications/uploads/04b700e040d0cac8681ba3d039be87a56020dd41.pdf
• Why most unit testing is waste https://rbcs-us.com/documents/Why-Most-Unit-Testing-is-
Waste.pdf
• A Response to "Why Most Unit Testing is Waste“ https://dzone.com/articles/response-why-
most-unit-testing
31. REFERENCES
• Rogert C. Martin, Clean Code (ISBN 978-0132350884)
• Martin Fowler, Refactoring: Improving the Design of Existing Code (ISBN 978-
0201485677)
• Robert C. Mayers, Essential Test Driven Development (ISBN 978-0134494159)
• John Ferguson Smart, BDD in Action (ISBN 978-1617291654)
