1. Unbearable Test Smells
Steven Mak
steven@odd-e.com
www.odd-e.com
twitter: stevenmak
1
Monday, 15 November 2010

2. Who am I?
2
Name: Steven Mak
Agile Coach at Odd-e
Lives in Hong Kong
Agile/Scrum, TDD Coaching
I love coding - Java, C/C++,
PHP, Perl, C#, VB, and some
weird languages
Monday, 15 November 2010

3. Copy and Paste Code
Long test codes are copied and pasted somewhere else with only a few
lines changing
3
Monday, 15 November 2010

4. DRY
Donʼt Repeat Yourself!
Donʼt Repeat Yourself!
Donʼt Repeat Yourself!
Donʼt Repeat Yourself!
Donʼt Repeat Yourself!
Donʼt Repeat Yourself!
Donʼt Repeat Yourself!
Donʼt Repeat Yourself!
4
Monday, 15 November 2010

5. Not knowing the fixtures
Some initialisation and clean up codes that are repeated in each tests...
5
Monday, 15 November 2010

6. What is fixture?
TEST_GROUP (TEST_thisObject)
{
void setup() {
}
void teardown() {
}
};
6
Monday, 15 November 2010

7. Duplication causing fragile
tests
Where is the duplication?
EXPECT_LOG(“ABC error”);
7
Monday, 15 November 2010

8. Duplication causing fragile
tests
Where is the duplication?
EXPECT_LOG(“ABC error”);
So there is a line in code that
prints this log message
8
Monday, 15 November 2010

9. Duplication causing fragile
tests
Put it under centralise header file:
#define ABC_ERROR_WITH_EC “ABC error”
The test will then look like:
EXPECT_LOG(ABC_ERROR);
9
Monday, 15 November 2010

10. Over-Optimism?
Tests that forgot to cover exceptional cases or just covered the easiest
condition
if (aaa() || bbb() || ccc() {
...
} else {
...
}
10
Monday, 15 November 2010

11. Tests donʼt have assertions
TEST(TEST_GROUP, TEST_THIS)
{
runThisFunctionLaLaLa();
}
11
Monday, 15 November 2010

12. 12
What does it mean by 80% Unit
Test Coverage?
Monday, 15 November 2010

13. Why xUnits donʼt have
CHECK_NOT_EQUAL?
What is the problem with:
CHECK(TRUE, xxx != 3);
13
Monday, 15 November 2010

14. Why xUnits donʼt have
CHECK_NOT_EQUAL?
What is the problem with:
CHECK(TRUE, xxx != 3);
Is there any good reason why you
cannot know the output value?
So, tell me what it is then. 14
Monday, 15 November 2010

15. OK, fine, so I use CHECK with a
specific output value, what now?
What is the problem with:
CHECK(TRUE, xxx == 4);
15
Monday, 15 November 2010

16. OK, fine, so I use CHECK with a
specific output value, what now?
What is the problem with:
CHECK(TRUE, xxx == 4);
In most xUnits, we have
LONGS_EQUAL telling you the actual
value when it goes wrong instead of a
“false”
16
Monday, 15 November 2010

17. 17
Do you know your xUnit harness?
Monday, 15 November 2010

18. Further example
try {
readConfigurationFile();
assertTrue(true);
} catch (IOException e) {
assertTrue(false);
e.printStackTrace();
}
These are the places you know your
team does not know the test harness.
18
Monday, 15 November 2010

19. 19
Some xUnit harness
Java: JUnit
.Net: NUnit
C/C++: CppUTest
PHP: PHPUnit
Monday, 15 November 2010

20. Whatʼs wrong?
What is the problem with:
TEST(TEST_AIH, FAIL_BAD_PARAM)
20
Monday, 15 November 2010

21. Names donʼt really tell
What is the problem with:
TEST(TEST_AIH, FAIL_BAD_PARAM)
Be more precise about how it
triggered the failure
21
Monday, 15 November 2010

22. What names tell us?
• Who
- Name of the SUT class
- Name of the method or feature being exercised
• Input
- Important characteristics of any input values
- Anything relevant about the state
• Output
- The outputs expected
- The expected post-exercise state
22
Monday, 15 November 2010

23. Conditional Test Logic?
// verify Vancouver is in the list
actual = null;
i = flightsFromCalgary.iterator();
while (i.hasNext()) {
FlightDto flightDto = (FlightDto) i.next();
if (flightDto.getFlightNumber().equals(
expectedCalgaryToVan.getFlightNumber()))
{
actual = flightDto;
assertEquals("Flight from Calgary to Vancouver",
expectedCalgaryToVan,
flightDto);
break;
}
}
23
Monday, 15 November 2010

24. Tests that crash 50% of the
time?!!
24
Monday, 15 November 2010

25. public void testFlightMileage_asKm2() throws Exception {
// set up fixture
// exercise constructor
Flight newFlight = new Flight(validFlightNumber);
// verify constructed object
assertEquals(validFlightNumber, newFlight.number);
assertEquals("", newFlight.airlineCode);
assertNull(newFlight.airline);
// set up mileage
newFlight.setMileage(1122);
// exercise mileage translator
int actualKilometres = newFlight.getMileageAsKm();
// verify results
int expectedKilometres = 1810;
assertEquals( expectedKilometres, actualKilometres);
// now try it with a canceled flight
newFlight.cancel();
try {
newFlight.getMileageAsKm();
fail("Expected exception");
} catch (InvalidRequestException e) {
assertEquals( "Cannot get cancelled flight mileage",
e.getMessage());
}
}
25
Testing everything at a time
Monday, 15 November 2010

26. Testing everything at a time
public void testFlightMileage_asKm2() throws Exception {
// set up fixture
// exercise constructor
Flight newFlight = new Flight(validFlightNumber);
// verify constructed object
assertEquals(validFlightNumber, newFlight.number);
assertEquals("", newFlight.airlineCode);
assertNull(newFlight.airline);
// set up mileage
newFlight.setMileage(1122);
// exercise mileage translator
int actualKilometres = newFlight.getMileageAsKm();
// verify results
int expectedKilometres = 1810;
assertEquals( expectedKilometres, actualKilometres);
// now try it with a canceled flight
newFlight.cancel();
try {
newFlight.getMileageAsKm();
fail("Expected exception");
} catch (InvalidRequestException e) {
assertEquals( "Cannot get cancelled flight mileage",
e.getMessage());
}
}
26
Comments as
deodorant
Monday, 15 November 2010

27. Testing everything at a time
public void testFlightMileage_asKm2() throws Exception {
// set up fixture
// exercise constructor
Flight newFlight = new Flight(validFlightNumber);
// verify constructed object
assertEquals(validFlightNumber, newFlight.number);
assertEquals("", newFlight.airlineCode);
assertNull(newFlight.airline);
// set up mileage
newFlight.setMileage(1122);
// exercise mileage translator
int actualKilometres = newFlight.getMileageAsKm();
// verify results
int expectedKilometres = 1810;
assertEquals( expectedKilometres, actualKilometres);
// now try it with a canceled flight
newFlight.cancel();
try {
newFlight.getMileageAsKm();
fail("Expected exception");
} catch (InvalidRequestException e) {
assertEquals( "Cannot get cancelled flight mileage",
e.getMessage());
}
}
27
Duplications with
application logic?
Monday, 15 November 2010

28. Inappropriate dependencies
• Test setup depending on other tests files
• A test file depending on another test file
• Stub functions depending on other tests
extern int reg_ecx; // in the stub program
int reg_exc; // in SUT
28
Monday, 15 November 2010

29. What can we do?
29
Monday, 15 November 2010

30. Try: one test group per file
But why canʼt? is it because of... ?
30
Monday, 15 November 2010

31. Test initialisation hard to read
and shared among test groups
in the same test file
• Fixtures
• Test Data Builder
• Parameterised Creation
• make-it-easy
31
Monday, 15 November 2010

32. Dont forget fixtures
TEST_GROUP (TEST_thisObject)
{
void setup()
{
}
void teardown()
{
}
};
32
Monday, 15 November 2010

33. Test Data Builder
eth_data_buf
->setControl(2)
->withParameterA(3)
->build();
33
Monday, 15 November 2010

34. Parameterised Creation
@Before
public void setUp() throws Exception {
alice = new Person();
alice.setId(1L);
alice.setFirstname("Alice");
alice.setLastname("Adams");
alice.setSsn("111111");
billy = new Person();
billy.setId(2L);
billy.setFirstname("Billy");
billy.setLastname("Burke");
billy.setSsn("222222");
clark = new Person();
clark.setId(3L);
clark.setFirstname("Clark");
clark.setLastname("Cable");
clark.setSsn("333333");
alice.isInLoveWith(billy);
}
34
Monday, 15 November 2010

35. Parameterised Creation
public class ParameterizedCreationMethodExample {
private Person alice, billy, clark;
@Before
public void setUp() throws Exception {
clark = createPerson("Clark", "Cable");
billy = createPerson("Billy", "Burke");
alice = createPerson("Alice", "Adams");
alice.isInLoveWith(billy);
}
private Person createPerson(String firstName, String lastName) {
Person person = new Person();
person.setFirstname(firstName);
person.setLastname(lastName);
person.setId(UniqueNumber.next());
person.setSsn(String.valueOf(UniqueNumber.next()));
return person;
}
@Test
public void aliceShouldAcceptWhenProposedToByBilly()
throws Exception {
billy.proposeTo(alice);
assertTrue(alice.isEngagedWith(billy));
}
}
35
Monday, 15 November 2010

36. make-it-easy
36
http://code.google.com/p/make-it-easy/
Maker<Apple> appleWith2Leaves = an(Apple, with(2, leaves));
Maker<Apple> ripeApple = appleWith2Leaves.but(with(ripeness, 0.9));
Maker<Apple> unripeApple = appleWith2Leaves.but(with(ripeness, 0.125));
Apple apple1 = make(ripeApple);
Apple apple2 = make(unripeApple);
Banana defaultBanana = make(a(Banana));
Banana straightBanana = make(a(Banana, with(curve, 0.0)));
Banana squishyBanana = make(a(Banana, with(ripeness, 1.0)));
Monday, 15 November 2010

37. Try: One assertion per test
37
Monday, 15 November 2010

38. Customised Assertions
#define CHECK_OBJ(a,b) CHECK_OBJ(a,b, __FILE__,__FILE__)
void CHECK_OBJ(struct* yourObj, struct* myObj, const char* file, int line)
{
if (structs are not equal) {
SimpleString errorMessage = StringFromFormat(
“My struct: %d, %p, %s”, myObj->d, myObj->p, myObj->s);
FAIL_LOCATION(errorMessage.asCharString(), file, line);
}
}
38
Monday, 15 November 2010

39. At least: One concept per test
39
Monday, 15 November 2010

40. Hamcrest
• Framework for writing declarative match criteria
40http://code.google.com/p/hamcrest/
String s = "yes we have no bananas today";
Matcher<String> containsBananas = new StringContains("bananas");
Matcher<String> containsMangoes = new StringContains("mangoes");
assertTrue(containsBananas.matches(s));
assertFalse(containsMangoes.matches(s));
assertThat(s, containsString("bananas"));
assertThat(s, not(containsString("mangoes"));
Or even better
Monday, 15 November 2010

41. Meaningful Assertion Messages
41
• Donʼt repeat what the built-in test framework
outputs to the console (e.g. name of the test
method)
• Donʼt repeat what the test name explains
• If you donʼt have anything good to say, you donʼt
have to say anything
• Write what should have happened, or what failed
to happen, and possibly mention when it should
have happened
Monday, 15 November 2010

42. Itʼs Design Smell!!!
42
Monday, 15 November 2010

43. Extra Constructor
43
public class LogFileMerge {
private URL logFileA, logFileB;
public LogFileMerge() {
this(new URL("http://server1/system.log"),
new URL("http://server2/system.log"));
}
LogFileMerge(URL a, URL b) {
this.logFileA = a;
this.logFileB = b;
}
}
Monday, 15 November 2010

44. Test-Specific SubClass
44
public class CreditCardProcessing {
public boolean isValid(String cardnumber) {
return validationCodeMatches(cardnumber)
&& cardIsActive(cardnumber);
}
protected boolean validationCodeMatches(String cardnumber) {
// validation logic omitted for brevity...
}
protected boolean cardIsActive(String cardnumber) {
// access to merchant system's web service
// omitted for brevity...
}
}
Monday, 15 November 2010

45. Still not testable?
45
• Do you follow good design principles?
Monday, 15 November 2010

46. Thinking
46
Test code is not second class citizen
Good design principles apply:
• Responsibility
• Dependency
• Low Coupling
• High Cohesion
• Indirection
• Protected Variations
Watch out for organisational dysfunction!
Monday, 15 November 2010

47. References
• Practical TDD and ATDD for Java Developers - Lasse Koskela
• Growing OO Software, guided by tests - Steve Freeman
• xUnit Test Patterns - Gerard Meszaros
47
Steven Mak
steven@odd-e.com
www.odd-e.com
twitter: stevenmak
Monday, 15 November 2010