Integration Testing on Kubernetes with JUnit5
With Hoverfly you can easily mock HTTP traffic during automated tests. Kubernetes is also based on the REST API. Today, I’m going to show you how to use both these tools together to improve integration testing on Kubernetes.
In the first step, we will build an application that uses the fabric8 Kubernetes Client. We don’t have to use it directly. Therefore, I’m going to include Spring Cloud Kubernetes. It uses the fabric8 client for integration with Kubernetes API. Moreover, the fabric8 client provides a mock server for the integration tests. In the beginning, we will use it, but then I’m going to replace it with Hoverfly. Let’s begin!
Source code
The source code is available on GitHub. If you want to clone the repository or just give me a star go here 🙂
Building applications with Spring Cloud Kubernetes
Spring Cloud Kubernetes provides implementations of well known Spring Cloud components based on Kubernetes API. It includes a discovery client, load balancer, and property sources support. We should add the following Maven dependency to enable it in our project.
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-kubernetes-all</artifactId>
</dependency>
Our application connects to the Mongo database, exposes REST API, and communicates with other applications over HTTP. Therefore we need to include some additional dependencies.
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>
The overview of our system is visible in the picture below. We need to mock communication between applications and with Kubernetes API. We will also run an embedded in-memory Mongo database during tests. For more details about building microservices with Spring Cloud Kubernetes read the following article.
Testing API with Kubernetes MockServer
First, we need to include a Spring Boot Test starter, that contains basic dependencies used for JUnit tests implementation. Since our application is connected to Mongo and Kubernetes API, we should also mock them during the test. Here’s the full list of required dependencies.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>de.flapdoodle.embed</groupId>
<artifactId>de.flapdoodle.embed.mongo</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.fabric8</groupId>
<artifactId>kubernetes-server-mock</artifactId>
<version>4.10.3</version>
<scope>test</scope>
</dependency>
Let’s discuss what exactly is happening during our test.
(1) First, we are enabling fabric8 Kubernetes Client JUnit5 extension in CRUD mode. It means that we can create a Kubernetes object on the mocked server.
(2) Then the KubernetesClient
is injected to the test by the JUnit5 extension.
(3) TestRestTemplate
is able to call endpoints exposed by the application that is started during the test.
(4) We need to set the basic properties for KubernetesClient
like a default namespace name, master URL.
(5) We are creating ConfigMap
that contains application.properties
file. ConfigMap
with name employee
is automatically read by the application employee
.
(6) In the test method we are using TestRestTemplate
to call REST endpoints. We are mocking Kubernetes API and running Mongo database in the embedded mode.
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@EnableKubernetesMockClient(crud = true) // (1)
@TestMethodOrder(MethodOrderer.Alphanumeric.class)
class EmployeeAPITest {
static KubernetesClient client; // (2)
@Autowired
TestRestTemplate restTemplate; // (3)
@BeforeAll
static void init() {
System.setProperty(Config.KUBERNETES_MASTER_SYSTEM_PROPERTY,
client.getConfiguration().getMasterUrl());
System.setProperty(Config.KUBERNETES_TRUST_CERT_SYSTEM_PROPERTY,
"true");
System.setProperty(
Config.KUBERNETES_AUTH_TRYKUBECONFIG_SYSTEM_PROPERTY, "false");
System.setProperty(
Config.KUBERNETES_AUTH_TRYSERVICEACCOUNT_SYSTEM_PROPERTY, "false");
System.setProperty(Config.KUBERNETES_HTTP2_DISABLE, "true");
System.setProperty(Config.KUBERNETES_NAMESPACE_SYSTEM_PROPERTY,
"default"); // (4)
client.configMaps().inNamespace("default").createNew()
.withNewMetadata().withName("employee").endMetadata()
.addToData("application.properties",
"spring.data.mongodb.uri=mongodb://localhost:27017/test")
.done(); // (5)
}
@Test // (6)
void addEmployeeTest() {
Employee employee = new Employee(1L, 1L, "Test", 30, "test");
employee = restTemplate.postForObject("/", employee, Employee.class);
Assertions.assertNotNull(employee);
Assertions.assertNotNull(employee.getId());
}
@Test
void addAndThenFindEmployeeByIdTest() {
Employee employee = new Employee(1L, 2L, "Test2", 20, "test2");
employee = restTemplate.postForObject("/", employee, Employee.class);
Assertions.assertNotNull(employee);
Assertions.assertNotNull(employee.getId());
employee = restTemplate
.getForObject("/{id}", Employee.class, employee.getId());
Assertions.assertNotNull(employee);
Assertions.assertNotNull(employee.getId());
}
@Test
void findAllEmployeesTest() {
Employee[] employees =
restTemplate.getForObject("/", Employee[].class);
Assertions.assertEquals(2, employees.length);
}
@Test
void findEmployeesByDepartmentTest() {
Employee[] employees =
restTemplate.getForObject("/department/1", Employee[].class);
Assertions.assertEquals(1, employees.length);
}
@Test
void findEmployeesByOrganizationTest() {
Employee[] employees =
restTemplate.getForObject("/organization/1", Employee[].class);
Assertions.assertEquals(2, employees.length);
}
}
Integration Testing on Kubernetes with Hoverfly
To test HTTP communication between applications we usually need to use an additional tool for mocking API. Hoverfly is an ideal solution for such a use case. It is a lightweight, open-source API simulation tool not only for REST-based applications. It allows you to write tests in Java and Python. In addition, it also supports JUnit5. You need to include the following dependencies to enable it in your project.
<dependency>
<groupId>io.specto</groupId>
<artifactId>hoverfly-java-junit5</artifactId>
<version>0.13.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.specto</groupId>
<artifactId>hoverfly-java</artifactId>
<version>0.13.0</version>
<scope>test</scope>
</dependency>
You can enable Hoverfly in your tests with @ExtendWith
annotation. It automatically starts Hoverfly proxy during a test. Our main goal is to mock the Kubernetes client. To do that we still need to set some properties inside @BeforeAll
method. The default URL used by KubernetesClient
is kubernetes.default.svc
. In the first step, we are mocking configmap
endpoint and returning predefined Kubernetes ConfigMap
with application.properties
. The name of ConfigMap
is the same as the application name. We are testing communication from the department
application to the employee
application.
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@ExtendWith(HoverflyExtension.class)
public class DepartmentAPIAdvancedTest {
@Autowired
KubernetesClient client;
@BeforeAll
static void setup(Hoverfly hoverfly) {
System.setProperty(Config.KUBERNETES_TRUST_CERT_SYSTEM_PROPERTY, "true");
System.setProperty(Config.KUBERNETES_AUTH_TRYKUBECONFIG_SYSTEM_PROPERTY, "false");
System.setProperty(Config.KUBERNETES_AUTH_TRYSERVICEACCOUNT_SYSTEM_PROPERTY,
"false");
System.setProperty(Config.KUBERNETES_HTTP2_DISABLE, "true");
System.setProperty(Config.KUBERNETES_NAMESPACE_SYSTEM_PROPERTY, "default");
hoverfly.simulate(dsl(service("kubernetes.default.svc")
.get("/api/v1/namespaces/default/configmaps/department")
.willReturn(success().body(json(buildConfigMap())))));
}
private static ConfigMap buildConfigMap() {
return new ConfigMapBuilder().withNewMetadata()
.withName("department").withNamespace("default")
.endMetadata()
.addToData("application.properties",
"spring.data.mongodb.uri=mongodb://localhost:27017/test")
.build();
}
// TESTS ...
}
After application startup, we may use TestRestTemplate
to call a test endpoint. The endpoint GET /organization/{organizationId}/with-employees
retrieves data from the employee
application. It finds the department by organization id and then finds all employees assigned to the department. We need to mock a target endpoint using Hoverfly. But before that, we are mocking Kubernetes APIs responsible for getting service and endpoint by name. The address and port returned by the mocked endpoints must be the same as the address of a target application endpoint.
@Autowired
TestRestTemplate restTemplate;
private final String EMPLOYEE_URL = "employee.default:8080";
@Test
void findByOrganizationWithEmployees(Hoverfly hoverfly) {
Department department = new Department(1L, "Test");
department = restTemplate.postForObject("/", department, Department.class);
Assertions.assertNotNull(department);
Assertions.assertNotNull(department.getId());
hoverfly.simulate(
dsl(service(prepareUrl())
.get("/api/v1/namespaces/default/endpoints/employee")
.willReturn(success().body(json(buildEndpoints())))),
dsl(service(prepareUrl())
.get("/api/v1/namespaces/default/services/employee")
.willReturn(success().body(json(buildService())))),
dsl(service(EMPLOYEE_URL)
.get("/department/" + department.getId())
.willReturn(success().body(json(buildEmployees())))));
Department[] departments = restTemplate
.getForObject("/organization/{organizationId}/with-employees", Department[].class, 1L);
Assertions.assertEquals(1, departments.length);
Assertions.assertEquals(1, departments[0].getEmployees().size());
}
private Service buildService() {
return new ServiceBuilder().withNewMetadata().withName("employee")
.withNamespace("default").withLabels(new HashMap<>())
.withAnnotations(new HashMap<>()).endMetadata().withNewSpec().addNewPort()
.withPort(8080).endPort().endSpec().build();
}
private Endpoints buildEndpoints() {
return new EndpointsBuilder().withNewMetadata()
.withName("employee").withNamespace("default")
.endMetadata()
.addNewSubset().addNewAddress()
.withIp("employee.default").endAddress().addNewPort().withName("http")
.withPort(8080).endPort().endSubset()
.build();
}
private List<Employee> buildEmployees() {
List<Employee> employees = new ArrayList<>();
Employee employee = new Employee();
employee.setId("abc123");
employee.setAge(30);
employee.setName("Test");
employee.setPosition("test");
employees.add(employee);
return employees;
}
private String prepareUrl() {
return client.getConfiguration().getMasterUrl()
.replace("/", "")
.replace("https:", "");
}
Conclusion
The approach described in this article allows you to create integration tests without running a Kubernetes instance. On the other hand, you could start a single-node Kubernetes instance like Microk8s and deploy your application there. You could as well use an existing cluster, and implement your tests with Arquillian Cube. It is able to communicate directly to the Kubernetes API.
Another key point is testing communication between applications. In my opinion, Hoverfly is the best tool for that. It is able to mock the whole traffic over HTTP in the single test. With Hoverfly, fabric8 and Spring Cloud you can improve your integration testing on Kubernetes.
Leave a ReplyCancel reply