Using Spring Cloud Kubernetes External Library
In this article I’m going to introduce my newest library for registering Spring Boot applications running outside the Kubernetes cluster. The motivation for creating this library has already been described in the details in my article Spring Cloud Kubernetes for Hybrid Microservices Architecture. Since Spring Cloud Kubernetes doesn’t implement registration in the service registry in any way, and just delegates it to the platform, it will not provide many benefits to applications running outside the Kubernetes cluster. To take an advantage of Spring Cloud Kubernetes Discovery you may just include library spring-cloud-kubernetes-discovery-ext-client
to your Spring Boot application running externally.
The current stable version of this library is 1.0.1.RELEASE
.
<dependency>
<groupId>com.github.piomin</groupId>
<artifactId>spring-cloud-kubernetes-discovery-ext-client</artifactId>
<version>1.0.1.RELEASE</version>
</dependency>
The registration feature is still disabled, since we won’t set property spring.cloud.kubernetes.discovery.register
to true
.
spring:
cloud:
kubernetes:
discovery:
register: true
If you are running an application you need to set a target namespace in Kubernetes, where it will be registered after startup. Here we are using a mechanism provided by Spring Cloud Kubernetes, which allows to set a default namespace for Fabric8 Kubernetes Client by setting environment variable KUBERNETES_NAMESPACE
.
A registration mechanism is based on Kubernetes objects: Service
and Endpoints
. It creates a service with the name taken from property spring.application.name
. In the annotations field of Service
object it puts a path of health check endpoint. By default it is /actuator/health
. The new Service
object is created only if it does not exist. The following screen shows the details about Service
created by library for application api-test
.
The path of the health check endpoint may be overridden using property spring.cloud.kubernetes.discovery.healthUrl
.
spring:
cloud:
kubernetes:
discovery:
healthUrl: /actuator/liveness
The next step is to create an Endpoints
object. Normally, you don’t have a lot to deal with Endpoints
, since it just tracks the IP addresses of the pods the service send traffic to. The name of Endpoints
is the same as the name of Service
. The IP address of the application is stored in the Subset
section. To distinguish Endpoints
created by the library for external applications from Endpoints
registered by the platform automatically each of them is labeled with external
flag with value true
.
We may display details about selected Endpoints
by command kubectl describe
to see the structure of this object.
The IP address of Spring Boot application is automatically detected by the library by calling Java method InetAddress.getLocalHost().getHostAddress()
. You may set a static IP address by using property spring.cloud.kubernetes.discovery.ipAddress
as shown below.
spring:
cloud:
kubernetes:
discovery:
ipAddress: 192.168.99.1
The library spring-cloud-kubernetes-discovery-ext-client
is based on the Spring Cloud Kubernetes project. It uses the Kubernetes API client provided within this library. The version of Spring Cloud Release Train used by the library is Hoxton.RELEASE
.
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-kubernetes</artifactId>
</dependency>
</dependencies>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>Hoxton.RELEASE</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
Assuming you have currently run a local instance of your Kubernetes cluster, you should at least provide an address of master API, and set property spring.cloud.kubernetes.client.trustCerts
just for the development purposes. Here’s bootstrap.yml
for my Spring Boot demo application.
spring:
application:
name: api-test
cloud:
kubernetes:
discovery:
register: true
client:
masterUrl: 192.168.99.100:8443
trustCerts: true
If you shutdown your Spring Boot application gracefully spring-cloud-kubernetes-discovery-ext-client
will unregister it from Kubernetes API. However, we always have to consider situations like forceful kill of application or network problems, that may cause unregistered instances in our Kubernetes API. Such situations should be handled on the platform side. Since Kubernetes Discovery does provide any built-in mechanisms for that (like heartbeat for applications running outside cluster), you may provide your implementation within Kubernetes Job
or you can just use my library spring-cloud-kubernetes-discovery-ext-watcher
responsible for detecting and removing inactive Endpoints
.
The main idea behind that library is illustrated on the picture below. The module spring-cloud-kubernetes-discovery-ext-watcher
is in fact Spring Boot application that needs to be run on Kubernetes. It periodically queries Kubernetes API in order to fetch the current list of external Endpoints
registered by applications using spring-cloud-kubernetes-discovery-ext-client
library. Then it is trying to call health endpoints registered for each application using IP address and ports taken from master API. If it won’t receive any response or receive response with HTTP status 5XX
several times in row, it removes IP address from Subset
section of Endpoints
object.
By default, spring-cloud-kubernetes-discovery-ext-watcher
application checks out endpoints registered in the same Kubernetes namespace as that application. This behaviour may be customized using configuration properties. We can set the default target namespace by setting property spring.cloud.kubernetes.watcher.targetNamespace
or just enable watching for Endpoints
labeled with external=true
across all the namespace by setting property spring.cloud.kubernetes.watcher.allNamespaces
to true
. We can also override some default retry properties for calling application health endpoints like number of retries (by default 3) or connect timeout (by default 1000ms). The configuration settings need to be delivered to the Watcher application as application.yml
or bootstrap.yml
file.
spring:
cloud:
kubernetes:
watcher:
targetNamespace: test
retries: 5
retryTimeout: 5000
To deploy spring-cloud-kubernetes-discovery-ext-watcher
application on your Kubernetes cluster you just need to apply the following Deployment
definition using kubectl apply
command.
apiVersion: apps/v1
kind: Deployment
metadata:
name: spring-cloud-discovery-watcher
spec:
selector:
matchLabels:
app: spring-cloud-discovery-watcher
template:
metadata:
labels:
app: spring-cloud-discovery-watcher
spec:
containers:
- name: watcher
image: piomin/spring-cloud-discovery-watcher
ports:
- containerPort: 8080
Since that application uses Spring Cloud Kubernetes for accessing Master API you need to grant privileges to read objects like Service
, Endpoints
or ConfigMap
. For development purposes you can just assign cluster-admin
to the default ServiceAccount
in target namespace.
$ kubectl create clusterrolebinding admin-external --clusterrole=cluster-admin --serviceaccount=external:default
In case you would like to override some default configuration settings you should define application.yml
file and place it inside ConfigMap
. Since spring-cloud-kubernetes-discovery-ext-watcher
uses Spring Cloud Kubernetes Config we may take an advantage of its integration with ConfigMap
. To do that just create ConfigMap
with the same metadata.name
as the application name.
apiVersion: v1
kind: ConfigMap
metadata:
name: api-test
data:
application.yaml: |-
spring:
cloud:
kubernetes:
watcher:
allNamespaces: true
retries: 5
retryTimeout: 5000
Here’s our sample deployment in the external namespace.
Now, let’s take a look on the logs generated by the spring-cloud-kubernetes-discovery-ext-watcher
application. Before running it I have started my sample application that uses spring-cloud-kubernetes-discovery-ext-client
outside Kubernetes. I has been registered under address 192.168.99.1:8080
. As you in the following logs in the beginning Watcher Application was able to communicate with sample application Actuator endpoint. Then I killed the sample application. Since Watcher was unable to call Actuator endpoint of previously checked application it finally remove that address from Subset
section of api-test
Endpoints
Here’s the Endpoints
object after removal of address 192.168.99.1:8080
.
Summary
The repository with Client library and Watcher application is available on GitHub: https://github.com/piomin/spring-cloud-kubernetes-discovery-ext.git. The library spring-cloud-kubernetes-discovery-ext-client
is available in Maven Central Repository. The Docker image with Watcher application is available on Docker Hub: https://hub.docker.com/repository/docker/piomin/spring-cloud-discovery-watcher. You can also run it directly from a source code using Skaffold.
Leave a ReplyCancel reply