This document discusses whitebox testing of Spring Boot applications. It begins with introductions and backgrounds, then discusses issues with existing testing frameworks like TestNG and JUnit 4. It proposes alternatives like Spock and JUnit 5, highlighting advantages of each. It also provides an overview of Spring Boot testing capabilities, focusing on integration testing support, transaction handling, main components, and reactive support. It concludes with examples of setting up Spring Boot testing with Spock and JUnit 5.
3. About myself
Application Architect @Thomas
Cook
Tech Lead - Platformers team
9+ years Java
*Very happy to leave an amazing teambuilding first to make it for a 5 AM flight back to Kyiv for this presentation
4. What is our current mission and vision in TC?
We are developing a flexible platform that will be able to make full use of CD
practices to improve time-to-market and be resilient enough to support a variety of
new markets.
5. Content
1. What we do in Thomas Cook
2. What problems we are facing with testing
3. Alternative frameworks - Spock and JUnit 5
4. Spring Boot testing
14. TestNG issues
● Test class lifecycle:
@BeforeMethod
private void beforeMethod() {
objectUnderTest = null;
MockitoAnnotations.initMocks(this);
….
}
15. TestNG issues
● Test class lifecycle:
@BeforeMethod
private void beforeMethod() {
objectUnderTest = null;
MockitoAnnotations.initMocks(this);
….
}
*Funnily enough you can use the same lifecycle for JUnit 5 with @TestInstance:
enum Lifecycle
PER_CLASS, PER_METHOD;
16. TestNG issues
● A problem with Mockito related to the way Mockito used Reflection:
https://github.com/mockito/mockito/issues/810
https://github.com/mockito/mockito/pull/948
17. TestNG issues
● A problem with Mockito related to the way Mockito used Reflection:
https://github.com/mockito/mockito/issues/810
https://github.com/mockito/mockito/pull/948
19. Issues with JUnit 4
● Not possible to have several @RunWith
For that reason a list of custom Rules was implemented on our side:
public class MockitoRule implements MethodRule {
@Override
public Statement apply(Statement base, FrameworkMethod frameworkMethod, Object test) {
settings.test = test;
Statement wrappedStatement = new MockitoInitializationStatement(base, settings);
wrappedStatement = new MockitoVerificationStatement(wrappedStatement, settings);
return wrappedStatement;
}
22. Spock: based on JUnit runner
No hassle with the runner: What’s more it extends Junit runner so it can run by
the tools you used for your tests before.
@RunWith(Sputnik.class)
public abstract class Specification extends MockingApi
….
public class Sputnik extends Runner implements Filterable, Sortable
23. Spock: formal semantics
JUnit tests lack formal semantics
The number one reason for using Spock is to make your tests more readable.
25. Spock: test blocks
Having them in your tests is mandatory.
Otherwise a piece of code like this will not complain:
def "CreateBooking"() {
whatever
}
26. Spock: test example
def "should fetch Bob and Alice without errors"() {
given:
def response =
mockMvc.perform(MockMvcRequestBuilders.get("/bookings/$id")).andReturn().response
def content = new JsonSlurper().parseText(response.contentAsString)
expect:
response.status == OK.value()
and:
content.passengerName == result
where:
id || result
'5' || "Bob"
'15' || "Alice"
}
29. Spock: @Shared fields
Objects stored into instance fields are not shared between feature methods. Instead, every feature
method gets its own object. This helps to isolate feature methods from each other, which is often a
desirable goal.
@Shared res = new VeryExpensiveResource()
30. Spock: error reporting
Nice and layered: Condition not satisfied:
content.passengerName == result
| | | |
| Bob | Bob1
| false
| 1 difference (75% similarity)
| Bob(-)
| Bob(1)
Condition not satisfied:
content.passengerName == result
| | | |
| Alice | Alice2
| false
| 1 difference (83% similarity)
| Alice(-)
| Alice(2)
31. Spock: parameterization is way better than in JUnit 4
There is a way to do it for JUnit, but it’s an external lib. Otherwise, it’s just too
verbose
32. Spock: Interactions
A way to express which method invocations are expected to occur:
then: "a rejection email is sent to the customer"
0 * emailService.sendConfirmation(sampleCustomer.getEmailAddress())
1 * emailService.sendRejection(sampleCustomer.getEmailAddress())
//Can also be:
(1..3) * subscriber.receive("hello") // between one and three calls (inclusive)
(1.._) * subscriber.receive("hello") // at least one call
(_..3) * subscriber.receive("hello") // at most three calls
33. Spock: Interactions - powerful matchers
1 * subscriber./r.*e/("hello") // a method whose name matches the given regular expression
// (here: method name starts with 'r' and ends in 'e')
1 * subscriber.receive(_) // any single argument (including null)
1 * subscriber.receive(*_) // any argument list (including the empty argument list)
1 * _._ // any method call on any mock object
1 * _ // shortcut for and preferred over the above
35. Groovy: Drawbacks
If everything inside your test/ folder will be Groovy based there’s a good
chance you will want to have some TestUtils/TestBuilders there at some
point. And you will use Groovy for all that.
37. Groovy: Drawbacks
It’s similar to java, but there will still be a learning curve and some tricky cases.
My use case - copying one object into another based on SO:
https://stackoverflow.com/questions/9072307/copy-groovy-class-properties/9072974#9072974
38. Groovy: Drawbacks
My use case - copying one object into another:
https://stackoverflow.com/questions/46952475/copy-object-properties-to-another-object-in-groovy
43. JUnit 5: module composition
JUnit 5 is composed of several different modules from three different sub-projects.
JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage
44. JUnit 5: native support in Spring 5.0
Spring-boot version 1.5.9-RELEASE is based on Spring 4 and the
SpringExtension is only available since Spring 5.
61. Spring Test: Integration Testing
Spring’s integration testing support has the following primary goals:
● To manage Spring IoC container caching between test execution.
● To provide Dependency Injection of test fixture instances.
● To provide transaction management appropriate to integration testing.
● To supply Spring-specific base classes that assist developers in writing integration
tests.
62. Spring Test: how are transactions handled?
Your tests might run against a real database. It’s rare, but possible. We are talking
about Integration tests after all.
By default, the framework will create and roll back a transaction for each test.
If a test method deletes the contents of selected tables while running within the
transaction managed for the test, the transaction will rollback by default, and the
database will return to its state prior to execution of the test.
63. Spring Test: how are transactions handled?
Your tests might run against a real database. It’s rare, but possible. We are talking
about Integration tests after all.
By default, the framework will create and roll back a transaction for each test.
If a test method deletes the contents of selected tables while running within the
transaction managed for the test, the transaction will rollback by default, and the
database will return to its state prior to execution of the test.
More flexibility is available via @Commit and @Rollback
64. Spring Test: how are transactions handled?
Your tests might run against a real database. It’s rare, but possible. We are talking
about Integration tests after all.
By default, the framework will create and roll back a transaction for each test.
If a test method deletes the contents of selected tables while running within the
transaction managed for the test, the transaction will rollback by default, and the
database will return to its state prior to execution of the test.
Not always that straightforward. Probably depends on implicit commits and
DB engines. Will also only work for MOCK transport.
65. Spring Test: main actors
TestContextManager TestContext
TestExecutionListeners
68. Spring Test: factories for ExecutionListener
Will scan your test class, look at its annotations and add its logic accordingly.
69. Spring Test: Reactive support
The package org.springframework.mock.http.server.reactive contains mock
implementations of ServerHttpRequest and ServerHttpResponse for use in
WebFlux applications.
The WebTestClient builds on the mock request and response to provide support
for testing WebFlux applications without an HTTP server.
70. Spring Test: static cache
Test suites and forked processes
The Spring TestContext framework stores application contexts in a static cache.
This means that the context is literally stored in a static variable. In other words, if
tests execute in separate processes the static cache will be cleared between each
test execution, and this will effectively disable the caching mechanism.
74. Spring Boot + Spock
Integration with Spock: Detached mocks via the DetachedMockFactory and
SpockMockFactoryBean classes.
class TestConfigurationForSpock {
private final detachedMockFactory = new DetachedMockFactory()
@Bean
BookingService bookingService() {
detachedMockFactory.Mock(BookingService);
}
}
Then just use: @Import(TestConfigurationForSpock)
75. Spring Boot + JUnit 5
Will need to include an additional library to use JUnit 5 with Spring Framework 4.3
76. Spring Boot + JUnit 5
There’s nothing at all Spock specific in spring-test, but you can find junit.jupiter is
there:
77. Spring Boot + JUnit 5
Be careful with your surefire version:
79. Spock vs JUnit 5: integration with Spring
Both frameworks provide a class with exact same name: SpringExtension.
However, the name is the only thing they have in common.
80. Spock vs JUnit 5: integration with Spring
SpringExtension
Spock JUnit 5
Tries to find any Spring-specific annotation on a test
class(spec): ContentHierarchy, BootstrapWith,
ContextConfiguration.
Using Spring-provided MetaAnnotaionUtils, so that it
can traverse class hierarchy.
Spring Test SpringExtention implements all possible
JUnit 5 callbacks(.e.g.: BeforeAllCallback,
AfterAllCallback)
If found, creates a TestContextManager and
delegate to it.
Will either get TestContextManager from a store of
create one.
Attaches Spock-specific listener for managing
Mocks:
testContext.setAttribute(MOCKED_BEANS_LIST,
mockedBeans);
Will wrap TestContextManager calls and delegate to
it. E.g.:
public void beforeAll(ExtensionContext context) {
getTestContextManager(context).beforeTestClass();
81. Spring Boot: autoconfigure slicing
Auto-configured tests
The spring-boot-test-autoconfigure module includes a number of annotations that can be used to
automatically configure different ‘slices’ of your app for testing purposes.
Examples:
● @WebMvcTest
● @JsonTest
● @DataJpaTest
● @JdbcTest
● @DataMongoTest
● ...
82. Spring Boot: autoconfigure slicing
@*Test would normally contain several @AutoConfigure* annotations:
For example:
...
@BootstrapWith(SpringBootTestContextBootstrapper.class)
@OverrideAutoConfiguration(enabled = false)
@TypeExcludeFilters(DataJpaTypeExcludeFilter.class)
@Transactional
@AutoConfigureCache
@AutoConfigureDataJpa
@AutoConfigureTestDatabase
@AutoConfigureTestEntityManager
@ImportAutoConfiguration
public @interface DataJpaTest
83. Spring Boot: autoconfigure slicing
To tweak @*Test mechanism, you can use a corresponding @AutoConfigure*
annotation.
For example:
@RunWith(SpringRunner.class)
@DataJpaTest
@AutoConfigureTestDatabase(replace= AutoConfigureTestDatabase.Replace.NONE)
public class ExampleRepositoryTests {
// ...
}
84. Spring Boot: autoconfigure slicing
Have you noticed this relatively unremarkable meta annotation:
@OverrideAutoConfiguration(enabled = false)
It will effectively set:
spring.boot.enableautoconfiguration=false
via AbstractTestContextBootstrapper
87. String Boot: Bootstrapping
As TestContextManager is the main entry point for test frameworks all they need
is to create one:
public TestContextManager(Class<?> testClass) {
this(BootstrapUtils.resolveTestContextBootstrapper(BootstrapUtils.createBootstrapContext(testClass)));
}
88. Spring Boot: Slicing
Do not litter the application’s main class with configuration settings that are are
specific to a particular area of its functionality.
Extract them into specific @Configuration instead. Otherwise they will be picked
up by all slice tests, which might not be what you want:
@SpringBootApplication
@EnableBatchProcessing - DO NOT DO IT THIS WAY
public class SampleApplication { ... }
89. Spring Boot: Custom component scanning
Another source of confusion is classpath scanning. Assume that, while you
structured your code in a sensible way, you need to scan an additional package.
Your application may resemble the following code:
@SpringBootApplication
@ComponentScan({ "com.example.app", "org.acme.another" }) - ALSO BAD
public class SampleApplication { ... }
This effectively overrides the default component scan directive with the side effect
of scanning those two packages regardless of the slice that you’ve chosen.
90. Spring Boot: TypeExcludeFilter
AutoConfigurationExcludeFilter - tells Spring Boot to exclude scanning
autoconfigurations. To use SpringFactoriesLoader instead.
TypeExcludeFilter - is an interesting case. While it’s in spring-boot jar, the doc
actually says: primarily used internally to support spring-boot-test.
93. Spring Boot: TypeExcludeFilter
What a specific test filter effectively does is:
1. Loads a corresponding annotation. E.g.: WebMvcTest for
WebMvcTypeExcludeFilter, DataJpaTest for DataJpaTypeExcludeFilter;
2. Inside *ExcludeFilter add specific annotations for which you want to enable
ComponentScan. For most *ExcludeFilters it is not required, however. They
will simply rely on SpringFactoriesLoader
94. @TestConfiguration :
● Unlike a nested @Configuration class which would be used instead of a your
application’s primary configuration, a nested @TestConfiguration class will
be used in addition to your application’s primary configuration;
● When placed on a top-level class, @TestConfiguration indicates that classes
in src/test/java should not be picked up by scanning. Use explicit @Import to
use them.
Spring Boot: overriding configurations
95. Spring Boot: Mocking and spying beans
For example, you may have a facade over some remote service that’s unavailable
during development.
Spring Boot includes a @MockBean annotation that can be used to define a
Mockito mock for a bean inside your ApplicationContext.
@RunWith(SpringRunner.class)
@SpringBootTest
public class MyTests {
@MockBean
private RemoteService remoteService;
96. Spring Boot: AutoConfigureMockMvc
There is an option to not start the server at all, but test only the layer below that,
where Spring handles the incoming HTTP request and hands it off to your
controller. That way, almost the full stack is used, and your code will be called
exactly the same way as if it was processing a real HTTP request, but without the
cost of starting the server. To do that we will use Spring’s MockMvc, and we can
ask for that to be injected for us by using the @AutoConfigureMockMvc annotation
on the test case.
https://spring.io/guides/gs/testing-web/
97. Spring Boot: WebMvcTest
@WebMvcTest
Only web layer is instantiated, not the whole context.
Often @WebMvcTest will be limited to a single controller and used in combination
with @MockBean to provide mock implementations for required collaborators.
98. Spring Boot: WebMvcTest vs AutoConfigureMockMvc vs
SpringBootTest
The main difference is inside META-INF/spring.factories:
# AutoConfigureMockMvc auto-configuration imports
org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc=
org.springframework.boot.test.autoconfigure.web.servlet.MockMvcAutoConfiguration,
org.springframework.boot.test.autoconfigure.web.servlet.MockMvcSecurityAutoConfiguration,
org.springframework.boot.test.autoconfigure.web.servlet.MockMvcWebClientAutoConfiguration,
org.springframework.boot.test.autoconfigure.web.servlet.MockMvcWebDriverAutoConfiguration
You’ll notice that WebMvcTest has also a @ImportAutoConfiguration *, but there’s
no entry in spring.factories for it.
https://spring.io/blog/2016/08/30/custom-test-slice-with-spring-boot-1-4
99. Spring Boot: WebMvcTest vs AutoConfigureMockMvc vs
SpringBootTest
The main difference is inside META-INF/spring.factories:
# AutoConfigureMockMvc auto-configuration imports
org.springframework.boot.test.autoconfigure.web.servlet.AutoConfigureMockMvc=
org.springframework.boot.test.autoconfigure.web.servlet.MockMvcAutoConfiguration,
org.springframework.boot.test.autoconfigure.web.servlet.MockMvcSecurityAutoConfiguration,
org.springframework.boot.test.autoconfigure.web.servlet.MockMvcWebClientAutoConfiguration,
org.springframework.boot.test.autoconfigure.web.servlet.MockMvcWebDriverAutoConfiguration
You’ll notice that WebMvcTest has also a @ImportAutoConfiguration *, but there’s
no entry in spring.factories for it.
However it has an exclude filter to reduce the amount of scanned objects.
100. Spring Boot: SpringBootTest MOCK WebEnvironment
Embedded servlet containers are not started when using this attribute. Can be
used in conjunction with @AutoConfigureMockMvc for MockMvc-based testing of
your application.
101. Spring Boot: SpringBootTest
RANDOM_PORT/DEFINED_PORT WebEnvironment
A test like this will run perfectly fine with real HTTP transport:
@SpringBootTest(classes = RamlBasedProducerApplication, webEnvironment =
SpringBootTest.WebEnvironment.RANDOM_PORT)
@AutoConfigureMockMvc
@OverrideAutoConfiguration(enabled = true)
class BookingServiceImplSpockTest extends Specification {
@LocalServerPort
private int port;
@Autowired
MockMvc mockMvc;
Spring will inject different ServletContexts into MockMvc depending on webEnv.
102. Spring Boot: Test utilities
● EnvironmentTestUtils(addEnvironment(env, "org=Spring", "name=Boot"))
● TestRestTemplate(behave in a test-friendly way by not throwing exceptions
on server-side errors)
● MockRestServiceServer(part of Spring Test)
If you want a transaction to commit — unusual, but occasionally useful when you want a particular test to populate or modify the database — the TestContext framework can be instructed to cause the transaction to commit instead of roll back via the@Commit annotation.
If you want a transaction to commit — unusual, but occasionally useful when you want a particular test to populate or modify the database — the TestContext framework can be instructed to cause the transaction to commit instead of roll back via the@Commit annotation.
If you want a transaction to commit — unusual, but occasionally useful when you want a particular test to populate or modify the database — the TestContext framework can be instructed to cause the transaction to commit instead of roll back via the@Commit annotation.
If you want a transaction to commit — unusual, but occasionally useful when you want a particular test to populate or modify the database — the TestContext framework can be instructed to cause the transaction to commit instead of roll back via the@Commit annotation.
If you want a transaction to commit — unusual, but occasionally useful when you want a particular test to populate or modify the database — the TestContext framework can be instructed to cause the transaction to commit instead of roll back via the@Commit annotation.
Talk about TestContextmanager and TestContext
Talk about TestContextmanager and TestContext
All test scope
public TestContextManager(Class<?> testClass) {
this(BootstrapUtils.resolveTestContextBootstrapper(BootstrapUtils.createBootstrapContext(testClass)));
}
public TestContextManager(Class<?> testClass) {
this(BootstrapUtils.resolveTestContextBootstrapper(BootstrapUtils.createBootstrapContext(testClass)));
}