Docker Power for Spring Boot Integration Testing

2 years ago
7 minutes

Credits

Spring Boot is a well-known and overpowerful server-side framework. And nowadays it has been used more and more in microservices systems. And no matter how the system is built it has to be tested. So how would you prefer to test your microservices system with hundreds of isolated components? I would like to take a bit of your attention for Spring Boot & Docker combinations for integration testing of different system components

Before going to the real examples I would like to give a bit of the context. Like many other engineers, I encountered problems in the testing of individual microservices in isolation from an entire system. So here are several problems teams faced:

  • Almost all business logic was covered only by unit tests, and each change in the code without changes in business logic led to additional overhead with changes in the behavior of many unit tests, which did not make any sense
  • On one hand, the company has been fast-growing, therefore the majority of engineers were new and not familiar with the system. On the other hand, some of the guys with most of the knowledge about the system already left, and the rest of them simply didn’t have enough knowledge about already implemented features. Therefore code refactoring was a risky, complicated, and unsafe procedure. So most of the feature decisions have followed the rule “works — do not touch”. Because of it, the technical dept has grown massively
  • Most of the “integration tests” were useless, they had a lot of dependencies within other existing and live running systems and because of it the behavior of most of the tests was unpredictable and unstable

So how could I improve the confidence and reliability of our services? As the first step, I decided to study whether anyone already faced the same problem and whether there are ready-made solutions in the modern world of microservice development. I read a lot of articles, and the next J.B. Rainsberger’s and André Schaffer’s articles impressed me the most

They focus on rethinking the classic testing pyramid

Integrated test is a test that will pass or fail based on the correctness of another system

If we consider the microservice structure as one big monolith, then we will conclude that classes in a classical monolithic service perform similar functions as microservices in a modern distributed architecture. And the most significant is testing the interaction between components, microservices in our case. Based on this, the idea of Honeycomb was born

But what can be used to recreate external dependencies? At the moment, the easiest way to configure and deploy components is the Docker, so I began to explore its capabilities for integration testing of services. So I concluded that docker-compose allows you to quite conveniently describe all the dependencies in a single file and run them if necessary

Spring Boot & Maven Docker Compose

Configurations for external dependencies can be described at docker-compose.yaml file

version: "3"
services:
  rabbitmq:
    image: "rabbitmq:3-management"
    environment:
      - RABBITMQ_DEFAULT_USER=guest
      - RABBITMQ_DEFAULT_PASS=guest
    ports:
      - "27403:15672"
      - "27404:5672"
  localstack:
    image: localstack/localstack:0.10.5
    environment:
      - SERVICES=s3
      - DEFAULT_REGION=eu-west-1
      - DATA_DIR=/tmp/localstack/data
    ports:
      - "4572:4572"
      - "8055:8080"
    volumes:
      - './.localstack:/tmp/localstack'

The standard set of different external dependencies used by our services

If the project builds by Maven, the next plugin will take all docker up and down management. It runs all containers before verify stage and shutdowns them after

<plugin>
    <groupId>com.dkanejs.maven.plugins</groupId>
    <artifactId>docker-compose-maven-plugin</artifactId>
    <version>2.4.0</version>
    <configuration>
        <composeFile>${project.basedir}/docker-compose.yml</composeFile>
        <detachedMode>true</detachedMode>
        <removeVolumes>true</removeVolumes>
    </configuration>
    <executions>
        <execution>
            <id>docker-compose-up</id>
            <phase>pre-integration-test</phase>
            <goals>
                <goal>up</goal>
            </goals>
        </execution>
        <execution>
            <id>docker-compose-down</id>
            <phase>post-integration-test</phase>
            <goals>
                <goal>down</goal>
            </goals>
        </execution>
    </executions>
</plugin>

And as always the decision had its pros and cons

Pros:

  • Single description place of all dependencies
  • All data migrations are isolated and run only once
  • The solution allows you to deploy the service locally along with all the necessary external dependencies

Cons:

  • It is necessary to clear containers contents after every test run
  • The more complex process of running tests in the IDE
  • Constant ports conflicts

The current solution already solves all mentioned above problems, however, it has some drawbacks. Therefore I come back to the alternatives research. And this led me to a rather interesting project — Testcontainers. The main idea is elegant and super simple, why do we need to use static YAML files when we can control the container’s dynamic life cycle from code

Spring Boot & Testcontainers

The current approach requires an additional dependency at the project pom file and integration of all external components in the tests codebase

<dependency>
    <groupId>org.testcontainers</groupId>
    <artifactId>testcontainers</artifactId>
    <version>1.14.2</version>
    <scope>test</scope>
</dependency>

public class RedisBackedCacheIntTestStep0 {
    private RedisBackedCache underTest;

    @Before
    public void setUp() {
        underTest = new RedisBackedCache("localhost", 6379);
    }

    @Test
    public void testSimplePutAndGet() {
        underTest.put("test", "example");

        String retrieved = underTest.get("test");
        assertEquals("example", retrieved);
    }
}

Although the current solution did solve some issues, it still had several drawbacks

Pros:

  • All data migrations are isolated and run only once
  • New containers will be launched for each new Spring Context
  • Run tests from the IDE in one click

Cons:

  • Bulky constructions to describe container launch rules
  • Opaque tincture of Spring Application Context files

The combination of Testcontainers and Spring Boot seems to be super promising and I decided to dig deeper to solve existing shortcomings

Spring Boot & Testcontainers Spring Boot Library

The result of all investigations and researches in the implementation of the next library — Testcontainers for Spring Boot

If the service external dependency is in the supported technologies list, then all you need to do is to add a maven dependency to your pom.xml and provide application configuration to application-test.yaml The library will take care of the rest of the work

<dependency>
    <groupId>com.playtika.testcontainers</groupId>
    <artifactId>embedded-rabbitmq</artifactId>
    <version>1.67</version>
</dependency>

spring:
  rabbitmq:
    host: ${embedded.rabbitmq.host}
    port: ${embedded.rabbitmq.port}
    username: ${embedded.rabbitmq.user}
    password: ${embedded.rabbitmq.password}

The configuration of RabbitMQ for Spring Test Context

Pros:

  • All data migrations are isolated and run only once
  • New containers will be launched for each new Spring Context
  • Run tests from the IDE in one click

Cons:

  • Not all technologies are supported by the library

Solved problems:

  • A lot of useless unit tests have been removed. Only critical parts of the implementation are covered by unit tests
  • The refactoring process is more confident and prevents a lot of problems before service deployment to a real environment
  • All integrated tests have been modified to be independent of external systems and performed stable and constant regardless of the state of a third-party system

Conclusion

This trip was exciting and help to solve many of the problems of the existing product. As an ideal option for developing microservice integration testing, I would recommend using a combination of all the approaches above, docker-compose for local deployment of dependencies, Testcontainers Spring Boot Library for integration testing, and pure Testcontainers for dependencies that are not yet supported by the library

Relevant links for further reading

0 0 votes
Article Rating
Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments