Quarkus Microservices with Consul Discovery

Quarkus Microservices with Consul Discovery

In this article, I’ll show you how to run Quarkus microservices outside Kubernetes with Consul service discovery and a KV store. Firstly, we are going to create a custom integration with Consul discovery, since Quarkus does not offer it. On the other hand, we may take advantage of built-in support for configuration properties from the Consul KV store. We will also learn how to customize the Quarkus REST client to integrate it with an external service discovery mechanism. The client will follow a load balancing pattern based on a round-robin algorithm.

If you feel you need to enhance your knowledge about the Quarkus framework visit the site with guides. For more advanced information you may read the articles Guide to Quarkus with Kotlin and Guide to Quarkus on Kubernetes.

The Architecture

Before proceeding to the implementation, let’s take a look at the diagram with the architecture of our system. There are three microservices: employee-service, departament-service, and organization-service. They are communicating with each other through a REST API. They use the Consul KV store as a distributed configuration backend. Every single instance of microservice is registering itself in Consul. A load balancer is on the client-side. It reads a list of registered instances of a target service from Consul. Then it chooses a single instance using a round-robin algorithm.

Source code

If you would like to try it by yourself, you may always take a look at my source code. In order to do that, you need to clone my repository sample-quarkus-microservices-consul. Then you should just follow my instructions. 🙂

Run the Consul instance

In order to run Consul on the local machine, we use its Docker image. By default, Consul exposes API and a web console on port 8500. We just need to expose that port outside the container.

$ docker run -d --name=consul \
   -e CONSUL_BIND_INTERFACE=eth0 \
   -p 8500:8500 \
   consul

Register Quarkus Microservice in Consul

Our application exposes a REST API on the HTTP server and connects to an in-memory database H2. It also uses the Java Consul client to interact with a Consul API. Therefore, we need to include at least the following dependencies.

<dependencies>
   <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-resteasy-jackson</artifactId>
   </dependency>
   <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-hibernate-orm-panache</artifactId>
   </dependency>
   <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-jdbc-h2</artifactId>
   </dependency>
   <dependency>
      <groupId>com.h2database</groupId>
      <artifactId>h2</artifactId>
      <scope>runtime</scope>
   </dependency>
   <dependency>
      <groupId>com.orbitz.consul</groupId>
      <artifactId>consul-client</artifactId>
      <version>${consul-client.version}</version>
   </dependency>
</dependencies>

Since we will run all our applications locally it is worth enabling an HTTP random port feature. To do that we should set the property quarkus.http.port to 0.

quarkus.http.port=0

Then we create the Consul client bean. By default, it is trying to connect with a server on the localhost and port 8500. So, we don’t need to provide any additional configuration.

@ApplicationScoped
public class EmployeeBeansProducer {

   @Produces
   Consul consulClient = Consul.builder().build();

}

Every single instance of a Quarkus application should register itself in Consul just after startup. Consequently, it needs to be able to deregister itself on shutdown. Therefore, you should first implement a bean responsible for intercepting startup and shutdown events. It is not hard with Quarkus.

The bean responsible for catching the startup and shutdown events is annotated with @ApplicationScoped. It defines two methods: onStart and onStop. It also injects the Consul client bean. Quarkus generates the number of the HTTP listen port on startup and saves it in the quarkus.http.port property. Therefore, the startup task needs to wait a moment to ensure that the application is running. We will run it 5 seconds after receiving the startup event. In order to register an application in Consul, we need to use the ConsulAgent object. Every single instance of the application needs to have a unique id in Consul. Therefore, we retrieve the number of running instances and use that number as the id suffix. The name of the service is taken from the quarkus.application.name property. The instance of the application should save id in order to be able to deregister itself on shutdown.

@ApplicationScoped
public class EmployeeLifecycle {

   private static final Logger LOGGER = LoggerFactory
         .getLogger(EmployeeLifecycle.class);
   private String instanceId;

   @Inject
   Consul consulClient;
   @ConfigProperty(name = "quarkus.application.name")
   String appName;
   @ConfigProperty(name = "quarkus.application.version")
   String appVersion;

   void onStart(@Observes StartupEvent ev) {
      ScheduledExecutorService executorService = Executors
            .newSingleThreadScheduledExecutor();
      executorService.schedule(() -> {
         HealthClient healthClient = consulClient.healthClient();
         List<ServiceHealth> instances = healthClient
               .getHealthyServiceInstances(appName).getResponse();
         instanceId = appName + "-" + instances.size();
         ImmutableRegistration registration = ImmutableRegistration.builder()
               .id(instanceId)
               .name(appName)
               .address("127.0.0.1")
               .port(Integer.parseInt(System.getProperty("quarkus.http.port")))
               .putMeta("version", appVersion)
               .build();
         consulClient.agentClient().register(registration);
         LOGGER.info("Instance registered: id={}", registration.getId());
      }, 5000, TimeUnit.MILLISECONDS);
   }

   void onStop(@Observes ShutdownEvent ev) {
      consulClient.agentClient().deregister(instanceId);
      LOGGER.info("Instance de-registered: id={}", instanceId);
   }

}

Run Quarkus microservices locally

Thanks to the HTTP random port feature we don’t have to care about port conflicts between applications. So, we can run as many instances as we need. To run a single instance of application we should use the quarkus:dev Maven command.

$ mvn compile quarkus:dev

Let’s see at the logs after employee-service startup. The application successfully called the Consul API using a Consul agent. With 5 second delay is sends an instance id and a port number.

Let’s take a look at the list of services registered in Consul.

I run two instances of every microservice. We may take a look at list of instances registered, for example by employee-service.

Integrate Quarkus REST client with Consul discovery

Both departament-service and organization-service applications use the Quarkus REST module to communicate with other microservices.

<dependency>
   <groupId>io.quarkus</groupId>
   <artifactId>quarkus-rest-client</artifactId>
</dependency>

Let’s take a look at the EmployeeClient interface inside the departament-service. We won’t use @RegisterRestClient on it. It is just annotated with @Path and contains a single @GET method.

@Path("/employees")
public interface EmployeeClient {

    @GET
    @Path("/department/{departmentId}")
    @Produces(MediaType.APPLICATION_JSON)
    List<Employee> findByDepartment(@PathParam("departmentId") Long departmentId);

}

We won’t provide a target address of the service, but just its name from the discovery server. The base URI is available in the application.properties file.

client.employee.uri=http://employee

The REST client uses a filter to detect a list of running instances registered in Consul. The filter implements a round-robin load balancer. Consequently, it replaces the name of service in the target URI with a particular IP address and a port number.

public class LoadBalancedFilter implements ClientRequestFilter {

   private static final Logger LOGGER = LoggerFactory
         .getLogger(LoadBalancedFilter.class);

   private Consul consulClient;
   private AtomicInteger counter = new AtomicInteger();

   public LoadBalancedFilter(Consul consulClient) {
      this.consulClient = consulClient;
   }

   @Override
   public void filter(ClientRequestContext ctx) {
      URI uri = ctx.getUri();
      HealthClient healthClient = consulClient.healthClient();
      List<ServiceHealth> instances = healthClient
            .getHealthyServiceInstances(uri.getHost()).getResponse();
      instances.forEach(it ->
            LOGGER.info("Instance: uri={}:{}",
                  it.getService().getAddress(),
                  it.getService().getPort()));
      ServiceHealth instance = instances.get(counter.getAndIncrement());
      URI u = UriBuilder.fromUri(uri)
            .host(instance.getService().getAddress())
            .port(instance.getService().getPort())
            .build();
      ctx.setUri(u);
   }

}

Finally, we need to inject the filter bean into the REST client builder. After that, our Quarkus application is fully integrated with the Consul discovery.

@ApplicationScoped
public class DepartmentBeansProducer {

   @ConfigProperty(name = "client.employee.uri")
   String employeeUri;
   @Produces
   Consul consulClient = Consul.builder().build();

   @Produces
   LoadBalancedFilter filter = new LoadBalancedFilter(consulClient);

   @Produces
   EmployeeClient employeeClient() throws URISyntaxException {
      URIBuilder builder = new URIBuilder(employeeUri);
      return RestClientBuilder.newBuilder()
            .baseUri(builder.build())
            .register(filter)
            .build(EmployeeClient.class);
   }

}

Read configuration properties from Consul

Although Quarkus does not provide built-in integration with a Consul discovery, it is able to read configuration properties from there. Firstly, we need to include the Quarkus Consul Config module to the Maven dependencies.

<dependency>
   <groupId>io.quarkus</groupId>
   <artifactId>quarkus-consul-config</artifactId>
</dependency>

Then, we enable the mechanism with the quarkus.consul-config.enable property.

quarkus.application.name=employee
quarkus.consul-config.enabled=true
quarkus.consul-config.properties-value-keys=config/${quarkus.application.name}

The Quarkus Config client reads properties from a KV store based on the location set in quarkus.consul-config.properties-value-keys property. Let’s create the settings responsible for a database connection and for enabling a random HTTP port feature.

Finally, we can run the application. The effect is the same as they would be stored in the standard application.properties file. The configuration for departament-service and organization-service looks pretty similar, but it also contains URLs used by the HTTP clients to call other microservices. For some reasons the property quarkus.datasource.db-kind=h2 always needs to be set inside application.properties file.

Testing Quarkus Consul discovery with gateway

All the applications listen on the random HTTP port. In order to simplify testing, we should run an API gateway. It will listen on a defined port. Since Quarkus does not provide any implementation of an API gateway, we are going to use Spring Cloud Gateway. We can easily integrate it with Consul using a Spring Cloud discovery client.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-loadbalancer</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-consul-discovery</artifactId>
</dependency>

The configuration of Spring Cloud Gateway contains a list of routes. We need to create three routes for all our sample applications.

spring:
  application:
    name: gateway-service
  cloud:
    gateway:
      discovery:
        locator:
          enabled: true
      routes:
        - id: employee-service
          uri: lb://employee
          predicates:
            - Path=/api/employees/**
          filters:
            - StripPrefix=1
        - id: department-service
          uri: lb://department
          predicates:
            - Path=/api/departments/**
          filters:
            - StripPrefix=1
        - id: organization-service
          uri: lb://organization
          predicates:
            - Path=/api/organizations/**
          filters:
            - StripPrefix=1
    loadbalancer:
      ribbon:
        enabled: false

Now, you may perform some test calls by yourself. The API gateway is available on port 8080. It uses prefix /api. Here are some curl commands to list all available employees, departments and organizations.

$ http://localhost:8080/api/employees
$ http://localhost:8080/api/departments
$ http://localhost:8080/api/organizations

Conclusion

Although Quarkus is a Kubernetes-native framework, we can use it to run microservices outside Kubernetes. The only problem we may encounter is a lack of support for external discovery. This article shows how to solve it. As a result, we created microservices architecture based on our custom discovery mechanism and built-in support for configuration properties in Consul. It is worth saying that Quarkus also provides integration with other third-party configuration solutions like Vault or Spring Cloud Config. If you are interested in a competitive solution based on Spring Boot and Spring Cloud you should read the article Microservices with Spring Boot, Spring Cloud Gateway and Consul Cluster.

15 COMMENTS

Alec

Do you try the consul discovery function in native image? i got a reflect error with consul-client and consul-api when integration in prod model.

    piotr.minkowski

    Hi. No, I didn’t. What problems exactly do you have?

Olivier

Hi,
first of all, thank you for all your valuable posts.
I’ve set up the architecture with two replicas for each service.
I can initiate two requests without any any error, but at the third request, I have an java.lang.IndexOutOfBoundsException.
It seems that the line ServiceHealth instance = instances.get(counter.getAndIncrement()) causes the problem.
I would suggest to modulo the counter by the number of instances.
What is your opinion ?

Thanks

    piotr.minkowski

    Hi,
    I think it is a good idea. Maybe, you could create PR with such a change in GitHub?

Ale Vouilloz

Hi. Excellent blog and very useful posts.
It looks like the consul-client library doesn’t work in native mode. Any ideas if it supports Quarkus in native mode?

Last stacktrace lines:

com.oracle.svm.core.jdk.UnsupportedFeatureError: Proxy class defined by interfaces [interface com.orbitz.consul.AgentClient$Api] not found. Generating proxy classes at runtime is not supported. Proxy classes need to be defined at image build time by specifying the list of interfaces that they implement. To define proxy classes use -H:DynamicProxyConfigurationFiles= and -H:DynamicProxyConfigurationResources= options.
at com.oracle.svm.core.util.VMError.unsupportedFeature(VMError.java:87)
at com.oracle.svm.reflect.proxy.DynamicProxySupport.getProxyClass(DynamicProxySupport.java:113)
at java.lang.reflect.Proxy.getProxyConstructor(Proxy.java:66)
at java.lang.reflect.Proxy.newProxyInstance(Proxy.java:1006)
at retrofit2.Retrofit.create(Retrofit.java:133)
at com.orbitz.consul.AgentClient.(AgentClient.java:42)
at com.orbitz.consul.Consul$Builder.build(Consul.java:717)
at com.saasmesh.customer.infrastructure.config.ConsulConfig.consulClient(ConsulConfig.kt:16)

Greetings!!

    piotr.minkowski

    Well, that’s a problem with the consul-client library. It probably does not support GraalVM since it uses proxies. You can as well replace it with your own code that just calls Consul REST API to work with the native image builds


    com.orbitz.consul
    consul-client
    ${consul-client.version}

Ale Vouilloz

Consul-client uses the retrofit2 library that uses proxies. It is possible to define proxy interfaces with parameters de GraalVM. However, there are other problems that prevent using the client on native mode.
The definitive solution was to use vertx-consul-client that allows the compilation of the Quarkus application in native mode without problems. I hope this helps someone.

repdom809

Hello, I have a problem. When I register my service, it didn’t deregister. Is because I am using windows?

    piotr.minkowski

    Hello,
    OS shouldn’t have any impact on it. But if you use Windows, the question is how do you run Docker? It is available on localhost? What error you have during app startup?

icedeveloper93

Hello !
When adding quarkus healthcheck & setting up a http check.
I take an error on the check : tcp dial cannot connect to 127.0.0.1:65300. The endpoint works well when I test it but fails on consul.

What do I have to do to fix this error ?

    piotr.minkowski

    Maybe start Consul e.g. on Docker

Rutul Shah

I have already implemented spring cloud gateway with quarkus and consul as per this blog. But I need to implement weighted strategy in load balancer and not the round robin. Can quarkus stork be used in this architecture so it can help us to decide the best suitable instance ? Can quarkus stork be used as a gateway ?

    piotr.minkowski

    Stork is for intergating with external discovery. Quarkus dosn’t have any dedicated project for API gateway (smth similar to Spring Cloud Gateway)

Rutul Shah

Thanks for making this. I am able to implement it. But I am looking to implement load balancer based on response time or weighted metric and not round robin.
As stork gives by default the best instance to pick to distribute the load, can we use stork with quarkus for weighted or response time load balancing in the architecture which you have shown ? Ofcourse on gateway side we may have to implement custom load balancer without stork as it is spring cloud and not quarkus. Can Stork be used or act as API gateway ?

    piotr.minkowski

    Yes, this article was created before Stork module for Quarkus has been implemented. Definitely, you can try to implement it with Stork

Exit mobile version