The document discusses unit testing and its benefits and limitations. It notes that while tests provide confidence that code works as intended, they cannot prevent all problems. The Boeing 737 MAX crashes are discussed as an example of issues despite passing tests due to sensor problems. Proper unit testing involves automated, repeatable, and continuous testing of isolated units with mocked dependencies. Test-driven development and design can lead to more testable and maintainable code, but tests also have costs and limitations.
5. Tests provide some level of confidence
your code does what you think it does…
6. …which is not the same as saying
the code is correct
from a business perspective
7. Tests cannot and will not ever
catch or prevent all possible problems
8. You might have a passing test suite but
still have logic bugs…
e.g. due to poorly defined or
understood requirements, poor
design, etc.
9.
10. “In the 737 Max, only one of the flight
management computers is active at a time -
either the pilot’s computer or the copilot’s
computer.And the active computer takes
inputs only from the sensors on its own side
of the aircraft.”
https://spectrum.ieee.org/aerospace/aviation/how-the-boeing-737-max-disaster-looks-to-a-software-developer
11. “When the two computers disagree, the
solution for the humans in the cockpit is
to look across the control panel to see
what the other instruments are saying and
then sort it out. In the Boeing system, the
flight management computer does not “look
across” at the other instruments. It
believes only the instruments on its side.”
https://spectrum.ieee.org/aerospace/aviation/how-the-boeing-737-max-disaster-looks-to-a-software-developer
12. “This means that if a particular angle-of-
attack sensor goes haywire - which happens
all the time in a machine that alternates
from one extreme environment to another,
vibrating and shaking all the way - the flight
management computer just believes it.”
https://spectrum.ieee.org/aerospace/aviation/how-the-boeing-737-max-disaster-looks-to-a-software-developer
13.
14. “The primary cause of this discrepancy was that one piece of ground
software supplied by Lockheed Martin produced results in a United
States customary unit, contrary to its Software Interface Specification
(SIS), while a second system, supplied by NASA, expected those
results to be in SI units, in accordance with the SIS. Specifically,
software that calculated the total impulse produced by thruster
firings produced results in pound-force seconds.The trajectory
calculation software then used these results – expected to be in
newton seconds – to update the predicted position of the
spacecraft.”
https://en.wikipedia.org/wiki/Mars_Climate_Orbiter#Cause_of_failure
15. The act of testing makes you consider more
than just the “happy path”
18. Good unit testing…
Fast (a few milliseconds?)
Isolated (no “real” dependencies)
Consider success & failure modes
19. What is a “unit” test?
Not always clearly defined
A single class? A single function?
Can it touch external resources?
(e.g. a database, web service, or file system)
20. @Test
fun `contribution should be sum of salary & Roth deferrals`() {
assertThat(payroll.totalContribution)
.isEqualTo(payroll.salaryDeferral +
payroll.rothDeferral)
}
A simple unit test
25. What to test?
More generally you should assume
libraries, frameworks, operating systems,
etc. have been tested and work as
advertised…
(obviously there are caveats to the above)
26. How to test?
TDD (Test-Driven Development aka “test first”)
TAST (test at same time, commit only tested code)
Test after (avoid…or deal with the consequences)
…on “legacy code” this is your only option
27. “Legacy Code”
“To me, legacy code is simply
code without tests”
- Michael Feathers,
Working Effectively with Legacy Code
29. “TDD”
Can you be test-driven without test first?
I think you can…
By letting tests drive the design…
And guide towards “testable” code…
30. Tests & Design
What does “testable” code mean?
* Loosely coupled, e.g. injecting dependencies
* Separation of concerns
* Single Responsibility Principle (SRP)
31. Tests & Design
What does “testable” code look like?
* Smaller classes in OO languages
* Smaller methods/functions (7-10 lines max? 20? 30?)
* Injected dependencies, e.g. via constructors or
function parameters
* More “pure” functions
32. Tests & Design
Why do these things make code testable?
* Facilitates “mocking” dependencies, e.g. a credit
card processing web service, or persistence to a
database
* A function that accepts input, has no side effects,
and produces a consistent output is much easier
to test
36. @ExtendWith(DropwizardExtensionsSupport.class)
class RelationshipResourceTest {
private static final RelationshipService SERVICE = mock(RelationshipService.class);
private static final ResourceExtension RESOURCES = ResourceExtension.builder()
.addResource(new RelationshipResource(SERVICE))
.build();
@AfterEach
void tearDown() {
reset(SERVICE);
}
@Test
@DisplayName("given a valid event should attempt to save the event")
void testRecordEvent() {
ConnectionEvent event =
newConnectionEvent(A_SERVICE_NAME, Direction.OUTBOUND, "some-identifier");
Response response = RESOURCES.target(“/elucidate/event")
.request()
.post(Entity.json(event));
assertThat(response.getStatus()).isEqualTo(202);
verify(SERVICE).createEvent(eq(event));
}
// more tests...
}
37. Testing Async Code
No sleeps!
Mock the environment
“If we go by the book…hours could seem like days…”
Wait for conditions
(e.g. until a status value is changed)
38. Unit vs Integration Tests
* A unit test of a single HTTP endpoint accepts
HTTP requests, but uses mock dependencies, e.g.
objects to persist data, or process payments, or
send asynchronous messages
* An integration test of that single HTTP endpoint
accepts HTTP requests, but uses the actual
dependencies…
39. Code Coverage
How much is “enough”?
Should automated builds fail if
below a threshold? 70%? 80%?
Can/should you test all exceptional
conditions?
40. Code Coverage
By definition, decoupled, “testable” code
is easier to achieve higher coverage
The goal should not be a number (80%)
The goal is ensuring correct behavior,
output, etc.
42. Gain confidence that code is “correct”
Automated, repeatable, continuous (ARC)
Tests cannot catch all problems
Better design through “testable code”
Build & change systems faster
44. Suggested Reading
Clean Code, Robert “Uncle Bob” Martin
Working Effectively with Legacy Code, Michael Feathers
Pragmatic Programmer, Dave Thomas & Andy Hunt
(20th anniversary edition in beta at
https://pragprog.com/book/tpp20/the-pragmatic-programmer-20th-anniversary-edition)