Integration testing is hard, and often teams are tempted to do it in production. Testcontainers allows writing meaningful integration tests spawning Docker containers for databases, queue systems, kv-store, other services. The talk, a blend of slides and live code, will show how we are able to deploy without fear while integrating with a dozen of different datastores. Don't mock your database with fake data anymore, work with real data

1. Integration tests:
use the containers, Luke!
Roberto “Frank” Franchini
@robfrankie

2. whoami(1)
https://www.linkedin.com/in/robfrank/

3. Former CTO of Arcade Analytics
Joined Activiti team in Alfresco

4. More use cases!

5. Arcade Analytics

9. Same output from different data-sources

10. One new connector per month

11. New features every week for every connector

12. Tests: features x connectors

13. Test Feature B
Test Feature A
Test Feature C
Test Feature D

14. Deploy without

15. How to avoid fear?

16. Integration tests

18. Activiti Cloud

19. Activiti Cloud is the first Cloud Native BPM framework built to
provide a scalable and transparent solution for BPM
implementations in cloud environments.

22. Each microservice has its own integration testing suite that
needs RabbiMQ and KeyCloack to be started before

23. Every morning a developer wakes up and starts containers
docker run -it -p 5672:5672 -p 15672:15672 --rm rabbitmq:management
docker run -it --rm -p 8180:8180 activiti/activiti-keycloak

24. Then run tests in the IDE
(Poor developer experience)

25. Don’t start containers manually anymore

26. Integration testing is hard

27. Image credits https://magnasoma.com
Monolith era

28. Microservices

29. Unit tests
Integration tests
UI
Tests
Slower
Faster
More
integration
More
isolation

31. Testcontainers is a Java library that supports JUnit tests,
providing lightweight, throwaway instances of common
databases, Selenium web browsers, or anything else that can
run in a Docker container.

32. Prerequisites

34. Ports for GO, .NET, Rust, Js

35. Main features

36. Temporary database containers: specialized PostgreSQL,
MySQL, MsSQL Server, MariaDB, Oracle XE, Virtuoso

37. RDBMS containers
@Container
public static PostgreSQLContainer container = new PostgreSQLContainer();
@Container
public static MySQLContainer container = new MySQLContainer();
@Container
public static OracleContainer container = new OracleContainer();

38. Others containers: neo4j, couchbase, toxy-proxy, kafka, redis,
influxDB, elasticSearch, rabbitMQ

39. Others containers
@Container
public static Neo4jContainer container = new Neo4jContainer();
@Container
public static InfluxDBContainer container = new InfluxDBContainer();
@Container
public static KafkaContainer container = new KafkaContainer();
@Container
public static RabbitMQContainer container = new RabbitMQContainer();

40. Webdriver containers: run a Dockerized Chrome or Firefox
browser ready for Selenium/Webdriver operations - complete
with automatic video recording

41. Selenium
private val chrome: BrowserWebDriverContainer<Nothing> = BrowserWebDriverContainer<Nothing>()
.apply {
withDesiredCapabilities(DesiredCapabilities.chrome())
withRecordingMode(RECORD_ALL, File("target"))
start()
}

42. Generic containers: run any Docker container as a test
dependency

43. Generic container, image based
@Container
public static GenericContainer container = new GenericContainer("orientdb:3.0.23")
.withExposedPorts(2424, 2480)
.withEnv("ORIENTDB_ROOT_PASSWORD", "rootpassword")
.waitingFor(Wait.forListeningPort());

44. ROM debian:stretch-slim
LABEL maintainer="NGINX Docker Maintainers
<docker-maint@nginx.com>"
ENV NGINX_VERSION 1.15.5-1~stretch
ENV NJS_VERSION 1.15.5.0.2.4-1~stretch
RUN set -x
&& apt-get update
&& apt-get install --no-install-recommends
--no-install-suggests -y gnupg1 apt-transport-https
ca-certificates
&&
NGINX_GPGKEY=573BFD6B3D8FBC641079A6ABABF5BD827BD9B
F62;
found='';
for server in
ha.pool.sks-keyservers.net
hkp://keyserver.ubuntu.com:80
hkp://p80.pool.sks-keyservers.net:80
version: '2'
services:
elasticsearch:
build:
context: elasticsearch/
args:
ELK_VERSION: $ELK_VERSION
volumes:
-
./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch
.yml:ro
ports:
- "9200:9200"
- "9300:9300"
environment:
ES_JAVA_OPTS: "-Xmx256m -Xms256m"
networks:
- elk
logstash:
build:
context: logstash/
args:
ELK_VERSION: $ELK_VERSION
volumes:
- ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml:ro
- ./logstash/pipeline:/usr/share/logstash/pipeline:ro
ports:
- "5000:5000"
- "9600:9600"
environment:
LS_JAVA_OPTS: "-Xmx256m -Xms256m"
networks:
- elk
depends_on:
- elasticsearch
kibana:
build:
Use a compose
Start from a Dockerfile

45. Generic container from Dockerfile
private val container: GenericContainer<Nothing> = GenericContainer<Nothing>(
ImageFromDockerfile("robfrank/ngnix")
.withFileFromPath("Dockerfile", Paths.get("./src/main/docker/nginx/Dockerfile"))
).apply {
withExposedPorts(80)
waitingFor(Wait.forListeningPort())
start()
followOutput(Slf4jLogConsumer(log))
}

46. Whatever is containerized by your team(s)

47. DB’s containers? Isn’t
enough?

48. H2 is fast, BUT doesn’t emulate specific features

49. Testcontainers is slower*, BUT gives 100% db compatibility
*not so slower

50. Use in your CI env

51. Jenkins
DOOD: Docker outside of Docker
DIND: Docker inside of Docker

52. DOOD: Dockerfile
FROM jenkins/jenkins:lts
USER root
RUN apt-get update
&& apt-get install -y apt-transport-https ca-certificates curl gnupg2 software-properties-common
&& curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")/gpg
| apt-key add -
&& add-apt-repository
"deb [arch=amd64] https://download.docker.com/linux/$(. /etc/os-release; echo "$ID")
$(lsb_release -cs)
stable"
&& apt-get update
&& apt-get install -y docker-ce
&& apt-get clean
&& rm -rf /var/lib/apt/lists/*
RUN usermod -aG docker jenkins
USER jenkins

53. version: '2'
services:
jenkins:
image: robfrank/jenkins:latest
ports:
- 8080:8080
- 50000:50000
privileged: false
volumes:
- ~/volumes/jenkins_home:/var/jenkins_home
- /usr/bin/docker:/usr/bin/docker
- /var/run/docker.sock:/var/run/docker.sock
- /usr/local/bin/docker-compose:/usr/local/bin/docker-compose

54. /usr/bin/docker
/usr/bin/docker

55. Travis: declare the service docker

56. language: java
service: docker
notifications:
email:
- builder@mycompany.com
before_install:
- docker version
- docker info
- cd ./src/main/docker/orientdb && ./build.sh && cd -
- cd ./src/main/docker/postgresql-dvdrental && ./build.sh && cd -
jdk:
- openjdk8
- openjdk12

57. Github Actions: just works

58. Scenarios

59. Test over different versions of a single database with
parametric test

60. Test a feature over multiple databases

61. Test your app against (or supported by) a complex env:
queue, kv-store, log aggregator, search engine

62. Use a compose file if necessary

63. Let’s code!

64. PostgreSQL container
@Container
public static PostgreSQLContainer container = new PostgreSQLContainer();
@Test
public void shouldTestSimpleQuery() throws SQLException {
Connection conn = DriverManager.getConnection(container.getJdbcUrl(),
container.getUsername(), container.getPassword());
Statement stmt = conn.createStatement();
stmt.execute("SELECT 1");
ResultSet resultSet = stmt.getResultSet();
resultSet.next();
assertThat(resultSet.getInt(1)).isEqualTo(1);
}

65. Jdbc url and init method
@Test
public void shouldSelectFromBar() throws SQLException {
String jdbcUrl =
"jdbc:tc:postgresql:9.6.8://hostname/databasename?&TC_INITFUNCTION=io.github.robfrank.testc
ontainers.JavaJdbcUrlTest::sampleInitFunction";
Connection conn = DriverManager.getConnection(jdbcUrl);
Statement stmt = conn.createStatement();
stmt.execute("SELECT * FROM bar");
ResultSet resultSet = stmt.getResultSet();
resultSet.next();
assertThat(resultSet.getString("foo")).isEqualTo("hello world");
}

66. Jdbc url and init method
public static void sampleInitFunction(Connection connection) throws SQLException {
connection.createStatement().execute("CREATE TABLE bar (n" +
" foo VARCHAR(255)n" +
");");
connection.createStatement().execute("INSERT INTO bar (foo) VALUES ('hello world');");
connection.createStatement().execute("CREATE TABLE my_counter (n" +
" n INTn" +
");");
}

67. Jdbc url script
@Test
public void shouldSelectFromBar() throws SQLException {
String jdbcUrl =
"jdbc:tc:postgresql:9.6.8://hostname/databasename?&TC_INITSCRIPT=initdb.sql";
Connection conn = DriverManager.getConnection(jdbcUrl);
Statement stmt = conn.createStatement();
stmt.execute("SELECT * FROM bar");
ResultSet resultSet = stmt.getResultSet();
resultSet.next();
assertThat(resultSet.getString("foo")).isEqualTo("hello world");
}

68. Generic container, image based
@Container
public static GenericContainer container = new GenericContainer("orientdb:3.0.23")
.withExposedPorts(2424, 2480)
.withEnv("ORIENTDB_ROOT_PASSWORD", "rootpassword")
.waitingFor(Wait.forListeningPort());

69. Generic container, image based
@Test
internal fun `should select beers vertexes`() {
OrientDB("remote:${container.containerIpAddress}:${container.firstMappedPort}", OrientDBConfig.defaultConfig()).use {
orientDB ->
orientDB.open("openbeer", "admin", "admin").use { db ->
db.query("select from Beer limit 10").use { resultSet ->
resultSet.asSequence()
.toList().apply {
assertThat(this).hasSize(10)
}.map { record ->
assertThat(record.isVertex).isTrue()
assertThat(record.hasProperty("name")).isTrue()
assertThat(record.hasProperty("descript")).isTrue()
record.vertex.get()
}.forEach { vertex: OVertex ->
assertThat(vertex.getEdges(ODirection.OUT)).isNotEmpty
}
}
}
}

70. Recap

71. Add to your project
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers</artifactId>
<version>1.12.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>MODULE_NAME</artifactId>
<version>1.12.0</version>
<scope>test</scope>
</dependency>

72. Use in test
@Container
val container = PostgreSQLContainer<Nothing>()
@Test fun `should perform simple query`() {
val conn = DriverManager.getConnection(container.jdbcUrl,
container.username,
container.password)
val stmt = conn.createStatement()
stmt.execute("SELECT 1")
val resultSet = stmt.resultSet
resultSet.next()
assertThat(resultSet.getInt(1)).isEqualTo(1)
}

73. https://github.com/robfrank/testcontainers-examples

74. ArcadeAnalytics connectors: https://github.com/arcadeAnalytics/arcade-connectors/
Kotlin and Java
Single container for multiple test classes
Neo4j, Postgres, Mysql, OrientDB, JanusGraph test and custom images
Alfresco Activiti: https://github.com/Activiti/activiti-cloud-query-service
Use of Testcontainers instead of maven plugin for lifecycle
Other examples

75. Thank you!