Guide to Quarkus on Kubernetes
Quarkus is usually described as a Kubernetes-native Java framework. It allows us to automatically generate Kubernetes resources based on the defaults and user-provided configuration. It also provides an extension for building and pushing container images. Quarkus can create a container image and push it to a registry before deploying the application to the target platform. It also provides an extension that allows developers to use Kubernetes ConfigMap
as a configuration source, without having to mount them into the pod. We may use fabric8 Kubernetes Client directly to interact with the cluster, for example during JUnit tests.
In this guide, you will learn how to:
- Use Quarkus Dekorate extension to automatically generate Kubernetes manifests basing on the source code and configuration
- Build and push images to Docker registry with Jib extension
- Deploy your application on Kubernetes without any manually created YAML in one click
- Use Quarkus Kubernetes Config to inject configuration properties from
ConfigMap
This guide is the second in series about Quarkus framework. If you are interested in the introduction to building Quarkus REST applications with Kotlin you may refer to my article Guide to Quarkus with Kotlin.
Source code
The source code with the sample Quarkus applications is available on GitHub. First, you need to clone the following repository: https://github.com/piomin/sample-quarkus-applications.git. Then, you need to go to the employee-service directory. We use the same repository as in my previous article about Quarkus.
1. Dependencies
Quarkus does not implement mechanisms for generating Kubernetes manifests, deploying them on the platform, or building images. It adds some logic to the existing tools. To enable extensions to Dekorate and Jib we should include the following dependencies.
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-kubernetes</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-container-image-jib</artifactId>
</dependency>
Jib builds optimized images for Java applications without a Docker daemon, and without deep mastery of Docker best-practices. It is available as plugins for Maven and Gradle and as a Java library. Dekorate is a Java library that makes generating and decorating Kubernetes manifests as simple as adding a dependency to your project. It may generate manifests basing on the source code, annotations, and configuration properties.
2. Preparation
In the first part of my guide to Kotlin, we were running our application in development mode with an embedded H2 database. In this part of the tutorial, we will integrate our application with Postgres deployed on Kubernetes. To do that we first need to change configuration settings for the data source. H2 database will be active only in dev and test mode. The configuration of Postgresql data source would be based on environment variables.
# kubernetes
quarkus.datasource.db-kind=postgresql
quarkus.datasource.username=${POSTGRES_USER}
quarkus.datasource.password=${POSTGRES_PASSWORD}
quarkus.datasource.jdbc.url=jdbc:postgresql://${POSTGRES_HOST}:5432/${POSTGRES_DB}
# dev
%dev.quarkus.datasource.db-kind=h2
%dev.quarkus.datasource.username=sa
%dev.quarkus.datasource.password=password
%dev.quarkus.datasource.jdbc.url=jdbc:h2:mem:testdb
# test
%test.quarkus.datasource.db-kind=h2
%test.quarkus.datasource.username=sa
%test.quarkus.datasource.password=password
%test.quarkus.datasource.jdbc.url=jdbc:h2:mem:testdb
3. Configure Kubernetes extension
With Quarkus Kubernetes extension we may customize the behavior of the manifest generator. To do that we need to provide configuration settings with the prefix quarkus.kubernetes.*
. There are pretty many options like defining labels, annotations, environment variables, Secret
and ConfigMap
references, or mounting volumes. First, let’s take a look at the Secret
and ConfigMap
prepared for Postgres.
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-config
labels:
app: postgres
data:
POSTGRES_DB: quarkus
POSTGRES_USER: quarkus
POSTGRES_HOST: postgres
---
apiVersion: v1
kind: Secret
metadata:
name: postgres-secret
labels:
app: postgres
data:
POSTGRES_PASSWORD: YWRtaW4xMjM=
In this fragment of configuration, besides simple label and annotation, we are adding reference to all the keys inside postgres-config
and postgres-secret
.
quarkus.kubernetes.labels.app-type=demo
quarkus.kubernetes.annotations.app-type=demo
quarkus.kubernetes.env.secrets=postgres-secret
quarkus.kubernetes.env.configmaps=postgres-config
4. Build image and deploy
Before executing build and deploy we need to apply manifest with Postgres. Here’s Deployment
definition of Postgres.
apiVersion: apps/v1
kind: Deployment
metadata:
name: postgres
spec:
replicas: 1
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
containers:
- name: postgres
image: postgres:latest
imagePullPolicy: "IfNotPresent"
ports:
- containerPort: 5432
env:
- name: POSTGRES_DB
valueFrom:
configMapKeyRef:
key: POSTGRES_DB
name: postgres-config
- name: POSTGRES_USER
valueFrom:
configMapKeyRef:
key: POSTGRES_USER
name: postgres-config
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
key: POSTGRES_PASSWORD
name: postgres-secret
volumeMounts:
- mountPath: /var/lib/postgresql/data
name: postgredb
volumes:
- name: postgredb
persistentVolumeClaim:
claimName: postgres-claim
Let’s apply it on Kubernetes together with required ConfigMap
, Secret
, PersistenceVolume
and PersistenceVolumeClaim
. All the objects are available inside example repository in the file employee-service/k8s/postgres-deployment.yaml
.
$ kubectl apply -f employee-service\k8s\postgresql-deployment.yaml
After deploying Postgres we may proceed to the main task. In order to build a Docker image with the application, we need to enable option quarkus.container-image.build
during Maven build. If you also want to deploy and run a container with the application on your local Kubernetes instance you need to enable option quarkus.kubernetes.deploy
.
$ clean package -Dquarkus.container-image.build=true -Dquarkus.kubernetes.deploy=true
If your Kubernetes cluster is located on the hosted cloud you should push the image to a remote Docker registry before deployment. To do that we should also activate option quarkus.container-image.push
during Maven build. If you do not push to the default Docker registry you have to set parameter quarkus.container-image.registry=gcr.io
inside the application.properties
file. The only thing I need to set for building images is the following property, which is the same as my login to docker.io site.
quarkus.container-image.group=piomin
After running the required Maven command our application is deployed on Kubernetes. Let’s take a look at what happened during the Maven build. Here’s the fragment of logs during that build. You see that Quarkus extension generated two files kubernetes.yaml
and kubernetes.json
inside target/kubernetes
directory. Then it proceeded to build a Docker image with our application. Because we didn’t specify any base image it takes a default one for Java 11 – fabric8/java-alpine-openjdk11-jre
.
Let’s take a look on the Deployment
definition automatically generated by Quarkus.
- It adds some annotations like port or path to metrics endpoint used by Prometheus to monitor application and enabled scraping. It also adds Git commit id, repository URL, and our custom annotation defined in
application.properties
. - It adds labels with the application name, version (taken from Maven pom.xml), and our custom label
app-type
. - It injects Kubernetes namespace name into the container.
- It injects the reference to the
postgres-secret
defined inapplication.properties
. - It injects the reference to the
postgres-config
defined inapplication.properties
. - The name of the image is automatically created. It is based on Maven
artifactId
andversion
. - The definition of liveness and readiness is generated if Maven module
quarkus-smallrye-health
is present
apiVersion: apps/v1
kind: Deployment
metadata:
annotations: # (1)
prometheus.io/path: /metrics
prometheus.io/port: 8080
app.quarkus.io/commit-id: f6ae37288ed445177f23291c921c6099cfc58c6e
app.quarkus.io/vcs-url: https://github.com/piomin/sample-quarkus-applications.git
app.quarkus.io/build-timestamp: 2020-08-10 - 13:22:32 +0000
app-type: demo
prometheus.io/scrape: "true"
labels: # (2)
app.kubernetes.io/name: employee-service
app.kubernetes.io/version: 1.1
app-type: demo
name: employee-service
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/name: employee-service
app.kubernetes.io/version: 1.1
template:
metadata:
annotations:
prometheus.io/path: /metrics
prometheus.io/port: 8080
app.quarkus.io/commit-id: f6ae37288ed445177f23291c921c6099cfc58c6e
app.quarkus.io/vcs-url: https://github.com/piomin/sample-quarkus-applications.git
app.quarkus.io/build-timestamp: 2020-08-10 - 13:22:32 +0000
app-type: demo
prometheus.io/scrape: "true"
labels:
app.kubernetes.io/name: employee-service
app.kubernetes.io/version: 1.1
app-type: demo
spec:
containers:
- env:
- name: KUBERNETES_NAMESPACE # (3)
valueFrom:
fieldRef:
fieldPath: metadata.namespace
envFrom:
- secretRef: # (4)
name: postgres-secret
- configMapRef: # (5)
name: postgres-config
image: piomin/employee-service:1.1 # (6)
imagePullPolicy: IfNotPresent
livenessProbe: # (7)
failureThreshold: 3
httpGet:
path: /health/live
port: 8080
scheme: HTTP
initialDelaySeconds: 0
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 10
name: employee-service
ports:
- containerPort: 8080
name: http
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /health/ready
port: 8080
scheme: HTTP
initialDelaySeconds: 0
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 10
serviceAccount: employee-service
Once the image has been built it is available in local registry. Quarkus automatically deploy it to the current cluster using already generated Kubernetes manifests.
Here’s the list of pods in default
namespace.
5. Using Kubernetes Config extension
With Kubernetes Config extension you can use ConfigMap
as a configuration source, without having to mount them into the pod with the application. To use that extension we need to include the following Maven dependency.
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-kubernetes-config</artifactId>
</dependency>
This extension works directly with Kubernetes API using fabric8 KubernetesClient
. That’s why we should set the proper permissions for ServiceAccount
. Fortunately, all the required configuration is automatically generated by Quarkus Kubernetes extension. The RoleBinding
object is appied automatically if quarkus-kubernetes-config
module is present.
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
annotations:
prometheus.io/path: /metrics
prometheus.io/port: 8080
app.quarkus.io/commit-id: a5da459af01637657ebb0ec3a606eb53d13b8524
app.quarkus.io/vcs-url: https://github.com/piomin/sample-quarkus-applications.git
app.quarkus.io/build-timestamp: 2020-08-10 - 14:25:20 +0000
app-type: demo
prometheus.io/scrape: "true"
labels:
app.kubernetes.io/name: employee-service
app.kubernetes.io/version: 1.1
app-type: demo
name: employee-service:view
roleRef:
kind: ClusterRole
apiGroup: rbac.authorization.k8s.io
name: view
subjects:
- kind: ServiceAccount
name: employee-service
Here’s our example ConfigMap
that contains a single property property1
.
apiVersion: v1
kind: ConfigMap
metadata:
name: employee-config
data:
application.properties: |-
property1=one
The same property is defined inside application.properties
available on the classpath, but there it has a different value.
property1=test
Before deploying a new version of application we need to add the following properties. First of them enables Kubernetes ConfigMap
injection, while the second specifies the name of injected ConfigMap
.
quarkus.kubernetes-config.enabled=true
quarkus.kubernetes-config.config-maps=employee-config
Finally we just need to implement a simple endpoint that injects and returns configuration property.
@ConfigProperty(name = "property1")
lateinit var property1: String
@GET
@Path("/property1")
fun property1(): String = property1
The properties obtained from the ConfigMap
have a higher priority than any properties of the same name that are found in application.properties
available on the classpath. Let’s test it.
3 COMMENTS