This document provides an overview of software testing concepts and the JUnit testing framework. It discusses the importance of testing, different types of testing, unit testing with JUnit, best practices for writing tests, integrating tests into builds with Ant and Maven, and logging. Examples of JUnit tests are also provided. The key points covered are:
- Why testing is important to find bugs, prevent regressions, and allow for refactoring
- Unit testing, functional testing, and other types of testing
- How to write automated unit tests with JUnit
- Best practices like testing edge cases, achieving good code coverage
- Integrating tests into builds with Ant and Maven
- Using logging frameworks like Log
1. Software testing
COMP9321
Julien Ponge (julienp@cse.unsw.edu.au)
Invited lecture: september 7th 2006
Computer School of Engineering
The University of New South Wales, Sydney, Australia
2. • We will see:
• why tests are important
• what habits you should get rid of
• how to create tests
• tips to make your life easier with tests
4. Testing... but what?
• Unit testing
• Functional testing
• Conformance testing
• Performance testing
4
5. Unit testing
• Typically classes
• Isolate as much as possible
• Self-contained and automated
• “Developers love writing tests”
• Metrics: passed tests and code coverage
5
6. What to expect?
• Less stupid bugs + test more often
• Prevent regression bugs
• Reveal bad design early
• Easier deep refactorings
6
7. The cost of tests
• Requires more 4,8
efforts in the early
stages 4
3,2
• A code base
without tests can 2,4
become too difficult 1,6
to manage as it
grows 0,8
0 0,4 0,8 1,2 1,6 2 2,4 2,8 3,2
• Test suites help
when scaling! 7
9. Typical approach
• Write a main method (console or GUI)
• Perform a few checkings and/or output
values (System.out.println(...))
• You may feed data manually and/or
interactively
• Manually check if it works or not
9
10. Example 1
import my.package.NiceClass;
public class OldSchoolTesting {
public static void main(String[] args) {
NiceClass nc = new NiceClass();
System.out.println(quot;Hey!quot;);
nc.doSomething();
System.out.println(quot;It didn't crash!!! (yet)quot;);
System.out.println(nc.getSomeValue());
}
}
10
11. Example 2
public class MyGreatClass {
// (...)
public void doSomethingGreat() {
String str = grabItSomewhereElse();
System.out.println(quot;=> str is quot; + str);
if (!str.equals(ONE_CONSTANT)) {
cry();
goToBed();
} else {
enjoy();
haveOneBeerOrTwo();
}
}
// (...)
} 11
13. Problems
• Not automated:
• you check
• you feed the data
• you need to run it
• System.out.println + messy outputs
• Most of the time... you throw it away
when it works!
13
14. To address this...
• We will see an automated unit testing
framework: JUnit
• We will see how to properly output
data when need be: Logging
14
16. JUnit
• Most famous for Java, many extensions
• Very stupid set of classes simple
• Composite pattern for test cases & test
suites
• Primarily for unit testing, but functional
testing can be conducted as well
• Derivatives: NUnit, PyUnit, CppUnit, ...
16
17. Running JUnit
• It reports:
• passed & failed tests
• errors (exceptions)
• It provides a text-mode and a Swing-
based runners
• Most IDEs embed JUnit and display a
progress bar (green / red)
17
18. How it works
• Base class is TestCase
• Start testing methods names with test
• Override setUp() & tearDown() if
initialization and cleanups are needed
• Check values with assertEquals,
assertNotSame, assertNull,
assertNotNull, ...
18
19. Basic test case canvas
• Create a class that extends TestCase, and
name it as MyClassTest for MyClass
• Create a test method testFoo for each
non-trivial method foo
• Instantiate data, invoke methods
• Use JUnit assertions
19
20. Example 1
public class DifferenceOperatorTest extends TestCase {
private DifferenceOperator operator = new DifferenceOperator(new
BusinessProtocolFactoryImpl());
public void testApply() throws DocumentException {
BusinessProtocol p1 = TestUtils.loadProtocol(quot;difference/p1.wsprotocolquot;);
BusinessProtocol p2 = TestUtils.loadProtocol(quot;difference/p2.wsprotocolquot;);
BusinessProtocol result = operator.apply(p2, p1);
BusinessProtocol expected = TestUtils.loadProtocol(quot;difference/p2-diff-
p1.wsprotocolquot;);
TestCase.assertEquals(expected, result);
}
public void testComputeComplement() throws DocumentException {
BusinessProtocol p1 = TestUtils.loadProtocol(quot;difference/p1.wsprotocolquot;);
BusinessProtocol expected = TestUtils.loadProtocol(quot;difference/compl-p1.wsprotocolquot;);
BusinessProtocol result = operator.computeComplement(p1);
TestCase.assertEquals(expected, result);
}
}
20
21. Example 2
public class OperationImplTest extends TestCase {
public void testEqualsObject() {
State s1 = new StateImpl(quot;s1quot;, false);
State s2 = new StateImpl(quot;s2quot;, false);
State s3 = new StateImpl(quot;s3quot;, false);
State s4 = new StateImpl(quot;s4quot;, false);
Message m1 = new MessageImpl(quot;aquot;, Polarity.POSITIVE);
Message m2 = new MessageImpl(quot;aquot;, Polarity.NEGATIVE);
OperationImpl o1 = new OperationImpl(quot;T1quot;, s1, s2, m1);
OperationImpl o2 = new OperationImpl(quot;T2quot;, s3, s4, m2);
TestCase.assertEquals(o1, o1);
TestCase.assertNotSame(o1, o2);
}
public void testToString() {
State s1 = new StateImpl(quot;s1quot;, false);
State s2 = new StateImpl(quot;s2quot;, false);
Message m1 = new MessageImpl(quot;aquot;, Polarity.POSITIVE);
OperationImpl o1 = new OperationImpl(quot;T1quot;, s1, s2, m1);
TestCase.assertEquals(quot;T1: ((s1),[a](+),(s2),explicit)quot;, o1.toString());
}
}
21
22. What your tests should do
• Test both good and bad cases
• Challenge your code: push it with
misuses and stupid values
• Make sure you get a good coverage:
• use dedicated tools
• Eclipse since version 3.2
22
24. Build integration
• Apache Ant is similar to Make, but for
Java
• There is a Ant task
• Apache Maven is project-oriented, and
expects JUnit tests
• Tests must pass for many Maven goals
24
28. Why?
• System.out and System.err are limited
• Not all events are of the same
importance: informations, debugging
messages, errors, ...
• Activate some messages on-demand,
output to console, network, dbs, files, ...
• Useful for server-side applications, but
also standalone applications or libraries
28
29. Logging in Java
• Log4J is a popular library
• Java 1.4+ has java.util.logging
• Apache commons-logging is a popular
choice to abstract and dynamically
discover the logging API
• Very minor impact on performances
• Never throw messages, keep them!
29
30. Logging levels
• Fatal : critical errors
• Error : errors that can be recovered
• Warn : warnings
• Info : general informations (start,
stop, ...)
• Debug : only-useful for debugging
• Trace : only for critical debugging
30
35. Life of a tests suite
• A test suite is like good wine: it gets better
over time!
• A good example is to reproduce bugs as
they get identified: you will prevent
regressions
35
36. Handling bugs
(ah ah I’m back) (I am a bug)
Test case
Tests suite
36
37. Tests are also...
• Useful to enforce API contracts
• An excellent API usage documentation
• A solid test suite allows deep
refactorings: it acts as a safeguard
against regressions
• Tests reveal bad design early: it is
difficult to test, then you need to
refactor!
37
39. Self-containment
• Your test suites should be as self-
contained as possible
• Avoid environment setup requirements
(databases servers, application
servers, ...)
• Eases automation in any context
39
47. HttpUnit
• Functional testing
• Works like a web browser / HTTP
client (cookies, form submissions, ...)
• Embeds a JavaScript engine (Rhino)
• JUnit integration
• Provides ServletUnit to unit-test servlets
in isolation of servlet containers
47
48. Main classes
• WebConversation: stateful client session
• WebRequest: a single GET or POST
HTTP request
• WebResponse: HTTP response
• WebForm, WebLink, (...): many helper
classes to access the pages content
48
54. Hard-dependencies
• Isolating the class to test is difficult
and/or impossible
• Failures may be induced by a
dependency
• More generally, it reveals potential code
evolution problems: strong coupling
54
56. Interfaces dependencies
• Interfaces introduce loose coupling
• At execution time: use the real class
• At test time: use a mock class
• Future evolution is easier
• BTW, don’t over-use interfaces!
56
57. Mock objects
• Give a simple behavior: fixed values,
random values, values passed by
methods of the mock class, ...
• They allow you to test even if the real
class hasn’t been created yet
• EasyMock, JMockIt allow to generate
mocks on-the-fly
57
58. Object.equals()
public boolean equals(Object obj)
{
if (obj instanceof ComparisonNode)
{
ComparisonNode other = (ComparisonNode) obj;
return symbol.equals(other.symbol)
&& leftChild.equals(other.leftChild)
&& rightChild.equals(other.rightChild);
}
return false;
important for
}
TestCase.assertXXX(...)
58
59. Dependency injection
• 2 types of injection: setter-based &
constructor-based
• Constructor injection is preferable, and
make your objects ready right after
their instanciation
• Clarifies classes dependencies
• Principle used in inversion of control
frameworks: PicoContainer, Spring, ...
59
62. Do
• Try to really write your tests first
• Test as soon as you create a new class
• Test often
• Start with sensible simple scenarios
• Load data from resources
(Class.getResourceAsStream(“/path/to”))
• Check for code coverage
62
63. Don’t
• Write overly exhaustive test cases
• Write tests for trivial methods (getters,
setters, ...) even if it lowers coverage
• Catch exceptions, unless you actually
test that
• Fix bugs without first reproducing them
in a test case
63
64. Traversing object graphs
• Sometimes you will need to walk
through deep object graphs to test
values
• This is really boring
• JXPath uses XPath on objects, and
replaces iterations/loops by requests
64
66. The case of ‘old’ software
• Test cases can be added to existing
applications
• In many situations the design can make
it difficult to test / isolate...
• You cannot break existing APIs
• Don’t expect to introduce tests cases for
100% of the code base...
• It will still allow you to spot new bugs!
66
68. Problem
• Developers work on different portions
of the code base
• Traditional integration may not be
frequent enough (~ weekly)
• It can be difficult to find who, when and
how the build was broken
• Continuous integration aims at
detecting problems in nearly real-time
68