In this article I’m going to introduce my newest library for registering Spring Boot applications running outside the Kubernetes cluster. The motivation for creating this library has already been described in the details in my article Spring Cloud Kubernetes for Hybrid Microservices Architecture. Since Spring Cloud Kubernetes doesn’t implement registration in the service registry in any way, and just delegates it to the platform, it will not provide many benefits to applications running outside the Kubernetes cluster. To take an advantage of Spring Cloud Kubernetes Discovery you may just include library spring-cloud-kubernetes-discovery-ext-client to your Spring Boot application running externally.
The current stable version of this library is 1.0.1.RELEASE.

<dependency>
  <groupId>com.github.piomin</groupId>
  <artifactId>spring-cloud-kubernetes-discovery-ext-client</artifactId>
  <version>1.0.1.RELEASE</version>
</dependency>

The registration feature is still disabled, since we won’t set property spring.cloud.kubernetes.discovery.register to true.

spring:
  cloud:
    kubernetes:
      discovery:
        register: true

If you are running an application you need to set a target namespace in Kubernetes, where it will be registered after startup. Here we are using a mechanism provided by Spring Cloud Kubernetes, which allows to set a default namespace for Fabric8 Kubernetes Client by setting environment variable KUBERNETES_NAMESPACE.

A registration mechanism is based on Kubernetes objects: Service and Endpoints. It creates a service with the name taken from property spring.application.name. In the annotations field of Service object it puts a path of health check endpoint. By default it is /actuator/health. The new Service object is created only if it does not exist. The following screen shows the details about Service created by library for application api-test.

The path of the health check endpoint may be overridden using property spring.cloud.kubernetes.discovery.healthUrl.

spring:
  cloud:
    kubernetes:
      discovery:
        healthUrl: /actuator/liveness 

The next step is to create an Endpoints object. Normally, you don’t have a lot to deal with Endpoints, since it just tracks the IP addresses of the pods the service send traffic to. The name of Endpoints is the same as the name of Service. The IP address of the application is stored in the Subset section. To distinguish Endpoints created by the library for external applications from Endpoints registered by the platform automatically each of them is labeled with external flag with value true.

We may display details about selected Endpoints by command kubectl describe to see the structure of this object.

The IP address of Spring Boot application is automatically detected by the library by calling Java method InetAddress.getLocalHost().getHostAddress(). You may set a static IP address by using property spring.cloud.kubernetes.discovery.ipAddress as shown below.

spring:
  cloud:
    kubernetes:
      discovery:
        ipAddress: 192.168.99.1

The library spring-cloud-kubernetes-discovery-ext-client is based on the Spring Cloud Kubernetes project. It uses the Kubernetes API client provided within this library. The version of Spring Cloud Release Train used by the library is Hoxton.RELEASE.

<dependencies>
   <dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-starter-kubernetes</artifactId>
   </dependency>
</dependencies>
<dependencyManagement>
   <dependencies>
      <dependency>
         <groupId>org.springframework.cloud</groupId>
         <artifactId>spring-cloud-dependencies</artifactId>
         <version>Hoxton.RELEASE</version>
         <type>pom</type>
         <scope>import</scope>
      </dependency>
   </dependencies>
</dependencyManagement>

Assuming you have currently run a local instance of your Kubernetes cluster, you should at least provide an address of master API, and set property spring.cloud.kubernetes.client.trustCerts just for the development purposes. Here’s bootstrap.yml for my Spring Boot demo application.

spring:
  application:
    name: api-test
  cloud:
    kubernetes:
      discovery:
        register: true
      client:
        masterUrl: 192.168.99.100:8443
        trustCerts: true

If you shutdown your Spring Boot application gracefully spring-cloud-kubernetes-discovery-ext-client will unregister it from Kubernetes API. However, we always have to consider situations like forceful kill of application or network problems, that may cause unregistered instances in our Kubernetes API. Such situations should be handled on the platform side. Since Kubernetes Discovery does provide any built-in mechanisms for that (like heartbeat for applications running outside cluster), you may provide your implementation within Kubernetes Job or you can just use my library spring-cloud-kubernetes-discovery-ext-watcher responsible for detecting and removing inactive Endpoints.
The main idea behind that library is illustrated on the picture below. The module spring-cloud-kubernetes-discovery-ext-watcher is in fact Spring Boot application that needs to be run on Kubernetes. It periodically queries Kubernetes API in order to fetch the current list of external Endpoints registered by applications using spring-cloud-kubernetes-discovery-ext-client library. Then it is trying to call health endpoints registered for each application using IP address and ports taken from master API. If it won’t receive any response or receive response with HTTP status 5XX several times in row, it removes IP address from Subset section of Endpoints object.

By default, spring-cloud-kubernetes-discovery-ext-watcher application checks out endpoints registered in the same Kubernetes namespace as that application. This behaviour may be customized using configuration properties. We can set the default target namespace by setting property spring.cloud.kubernetes.watcher.targetNamespace or just enable watching for Endpoints labeled with external=true across all the namespace by setting property spring.cloud.kubernetes.watcher.allNamespaces to true. We can also override some default retry properties for calling application health endpoints like number of retries (by default 3) or connect timeout (by default 1000ms). The configuration settings need to be delivered to the Watcher application as application.yml or bootstrap.yml file.

spring:
  cloud:
    kubernetes:
      watcher:
        targetNamespace: test
        retries: 5
        retryTimeout: 5000

To deploy spring-cloud-kubernetes-discovery-ext-watcher application on your Kubernetes cluster you just need to apply the following Deployment definition using kubectl apply command.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: spring-cloud-discovery-watcher
spec:
  selector:
    matchLabels:
      app: spring-cloud-discovery-watcher
  template:
    metadata:
      labels:
        app: spring-cloud-discovery-watcher
    spec:
      containers:
      - name: watcher
        image: piomin/spring-cloud-discovery-watcher
        ports:
        - containerPort: 8080

Since that application uses Spring Cloud Kubernetes for accessing Master API you need to grant privileges to read objects like Service, Endpoints or ConfigMap. For development purposes you can just assign cluster-admin to the default ServiceAccount in target namespace.

$ kubectl create clusterrolebinding admin-external --clusterrole=cluster-admin --serviceaccount=external:default

In case you would like to override some default configuration settings you should define application.yml file and place it inside ConfigMap. Since spring-cloud-kubernetes-discovery-ext-watcher uses Spring Cloud Kubernetes Config we may take an advantage of its integration with ConfigMap. To do that just create ConfigMap with the same metadata.name as the application name.

apiVersion: v1
kind: ConfigMap
metadata:
  name: api-test
data:
  application.yaml: |-
    spring:
      cloud:
        kubernetes:
          watcher:
            allNamespaces: true
            retries: 5
            retryTimeout: 5000

Here’s our sample deployment in the external namespace.

Now, let’s take a look on the logs generated by the spring-cloud-kubernetes-discovery-ext-watcher application. Before running it I have started my sample application that uses spring-cloud-kubernetes-discovery-ext-client outside Kubernetes. I has been registered under address 192.168.99.1:8080. As you in the following logs in the beginning Watcher Application was able to communicate with sample application Actuator endpoint. Then I killed the sample application. Since Watcher was unable to call Actuator endpoint of previously checked application it finally remove that address from Subset section of api-test Endpoints

Here’s the Endpoints object after removal of address 192.168.99.1:8080.

Summary

The repository with Client library and Watcher application is available on GitHub: https://github.com/piomin/spring-cloud-kubernetes-discovery-ext.git. The library spring-cloud-kubernetes-discovery-ext-client is available in Maven Central Repository. The Docker image with Watcher application is available on Docker Hub: https://hub.docker.com/repository/docker/piomin/spring-cloud-discovery-watcher. You can also run it directly from a source code using Skaffold.