Microservices with Kubernetes and Docker

Microservices with Kubernetes and Docker

In one of my previous posts, I described an example of a continuous delivery configuration for building microservices with Docker and Jenkins. It was a simple configuration where I decided to use only Docker Pipeline Plugin for building and running containers with microservices. That solution had one big disadvantage – we had to link all containers between each other to provide communication between microservices deployed inside those containers. Today I’m going to show you one the smart solution which helps us to avoid that problem – Kubernetes.

Kubernetes is an open-source platform for automating deployment, scaling, and operations of application containers across clusters of hosts, providing container-centric infrastructure. It was originally designed by Google. It has many features especially useful for applications running in production like service naming and discovery, load balancing, application health checking, horizontal auto-scaling or rolling updates. There are several important concepts around Kubernetes we should know before going into the sample.

Pod – this is basic unit in Kubernetes. It can consist of one or more containers that are guaranteed to be co-located on the host machine and share the same resources. All containers deployed inside the pod can see other containers via the localhost. Each pod has a unique IP address within the cluster

Service – is a set of pods that work together. By default, a service is exposed inside a cluster but it can also be exposed onto an external IP address outside your cluster. We can expose it using one of four available behaviors: ClusterIP, NodePort, LoadBalancer and ExternalName.

Replication Controller – it is specific type of Kubernetes controllers. It handles replication and scaling by running a specified number of copies of a pod across the cluster. It is also responsible for pods replacement if the underlying node fails.

Minikube

Configuration of highly available Kubernetes cluster is rather not easy task to perform. Fortunately, there is a tool that makes it easy to run Kubernetes locally – Minikube. It can run a single-node cluster inside a VM, which is really important for developers who want to try it out. The beginning is really easy. For example on Windows, you have to download minikube.exe and kubectl.exe and add them to PATH environment variable. Then you can start it from command line using minikube start command and use almost all of Kubernetes features available by calling kubectl command. An alternative for the command-line option is Kubernetes Dashboard. It can be launched by calling minikube dashboard command. We can create, update, or delete deployment from the UI dashboard, and also list and view a configuration of all pods, services, ingresses, replication controller,s etc. Here’s the Kubernetes Dashboard with the list of deployments for our sample.

kube1

Application

The concept of microservices architecture for our sample is pretty similar to the concept from my article about continuous delivery with Docker and Jenkins which I mentioned at the beginning of that article. We also have accounts and customer microservices. Customer service is interacting with account service while searching for customer accounts. We do not use the gateway (Zuul) and discovery (Eureka) Spring Boot services, because we have such mechanisms available on Kubernetes out of the box. Here’s the picture illustrating the architecture of the presented solution. Each microservice’s pod consists of two containers: first with a microservice application and second with a Mongo database. Account and customer microservices have their own database where all data is stored. Each pod is exposed as a service and can be searched by name on Kubernetes. We also configure Kubernetes Ingress which acts as a gateway for our microservices.

kube_micro

Sample application source code is available on GitHub. It consists of two modules account-service and customer-service. It is based on the Spring Boot framework but doesn’t use any of Spring Cloud projects except the Feign client. Here’s dockerfile from account service. We use small openjdk image – alpine. Thanks to that our result image will have about ~120MB instead of ~650MB when using standard openjdk as a base image.

FROM openjdk:alpine
MAINTAINER Piotr Minkowski <piotr.minkowski@gmail.com>
ADD target/account-service.jar account-service.jar
ENTRYPOINT ["java", "-jar", "/account-service.jar"]
EXPOSE 2222

To enable MongoDB support I add spring-boot-starter-data-mongodb dependency to pom.xml. We also have to provide connection data to application.yml and annotate entity class with @Document. The last think is to declare repository interface extending MongoRepository which has basic CRUD methods implemented. We add two custom find methods.

public interface AccountRepository extends MongoRepository<Account, String> {

   public Account findByNumber(String number);
   public List<Account> findByCustomerId(String customerId);

}

In customer service, we are going to call the API method from the account-service. Here’s declarative REST client @FeignClient declaration. All the pods with account service are available under the account-service name and default service port – 2222. Such settings are the results of the service configuration on Kubernetes. I will describe it in the next section.

@FeignClient(name = "account-service", url = "http://account-service:2222")
public interface AccountClient {

   @RequestMapping(method = RequestMethod.GET, value = "/accounts/customer/{customerId}")
   List<Account> getAccounts(@PathVariable("customerId") String customerId);

}

The docker image of our microservices can be built with the command visible below. After the build, you should push that image to the official docker hub or your private registry. In the next section, I’ll describe how to use them on Kubernetes. Docker images of the described microservices are also available on my Docker Hub public repository as piomin/account-service and piomin/customer-service.

$docker build -t piomin/account-service .
$ docker push piomin/account-service

Kubernetes deployment

You can create deployment on Kubernetes using kubectl run command, Minikube dashboard or JSON configuration files with kubectl create command. I’m going to show you how to create all resources from JSON configuration files, because we need to create multi-containers deployments in one step. Here’s the deployment configuration file for the account-service. We have to provide a deployment name, image name, and exposed port. In the replicas property we are setting requested number of created pods.

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: account-service
  labels:
    run: account-service
spec:
  replicas: 1
  template:
    metadata:
      labels:
        run: account-service
    spec:
      containers:
      - name: account-service
         image: piomin/account-service
         ports:
         - containerPort: 2222
           protocol: TCP
      - name: mongo
        image: library/mongo
        ports:
        - containerPort: 27017
           protocol: TCP

We are creating a new deployment by running the command below. The same command is used for creating services and ingress. Only a JSON file format is different.

$ kubectl create -f deployment-account.json

Now, let’s take o look at service configuration file. We have already created deployment. As you could see in the dashboard image has been pulled from Docker Hub, pod and replica set has been created. Now, we would like to expose our microservice outside. That’s why service is needed. We are also exposing the Mongo database on its default port, to be able to connect the database and create collections from a MongoDB client.

kind: Service
apiVersion: v1
metadata:
  name: account-service
spec:
  selector:
    run: account-service
  ports:
  - name: port1
    protocol: TCP
    port: 2222
    targetPort: 2222
  - name: port2
    protocol: TCP
    port: 27017
    targetPort: 27017
    type: NodePort

kube-2

After creating a similar configuration for customer service we have our microservices exposed. Inside Kubernetes they are visible on default ports (2222 and 3333) and service name. That’s why inside customer service REST client (@FeignClient) we declared URL http://account-service:2222. No matter how many pods have been created service will always be available on that URL and requests are load balanced between all pods be Kubernetes out of the box. If we would like to access each service outside Kubernetes, for example in the web browser we need to call it with port visible below the container default port – in that sample for account service it is 31638 port and for customer service 31171 port. If you have run Minikube on Windows your Kubernetes is probably available under 192.168.99.100 address, so you could try to call the account-service using URL http://192.168.99.100:31638/accounts. Before such test you need to create collection on Mongo database and user micro/micro which is set for that service inside application.yml.

kube-3

Ok, we have our two microservices available under two different ports. It is not exactly what we need. We need some kind of gateway available under on IP which proxies our requests to exact service by matching the request path. Fortunately, such an option is also available on Kubernetes. This solution is Ingress. Here’s the JSON ingress configuration file. There are two rules defined, first for account-service and second for customer service. Our gateway is available under micro.all hostname and default HTTP port.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: gateway-ingress
spec:
  backend:
    serviceName: default-http-backend
    servicePort: 80
  rules:
  - host: micro.all
    http:
    paths:
    - path: /account
      backend:
        serviceName: account-service
        servicePort: 2222
    - path: /customer
      backend:
        serviceName: customer-service
        servicePort: 3333

The last thing that needs to be done to make the gateway working is to add following entry to system hosts file (/etc/hosts for Linux and C:\Windows\System32\drivers\etc\hosts for windows). Now, you could try to call from your web browser http://micro.all/accounts or http://micro.all/customers/{id}, which also calls account service in the background.

[MINIKUBE_IP] micro.all

Conclusion

Kubernetes is a great tool for microservices clustering and orchestration. It is still a relatively new solution under active development. It can be used together with Spring Boot stack or as an alternative for Spring Cloud Netflix OSS, which seems to be the most popular solution for microservices now. It has also a UI dashboard where you can manage and monitor all resources. Production grade configuration is probably more complicated than a single host development configuration with Minikube, but I don’t that it is a solid argument against Kubernetes.

0 COMMENTS

comments user
Naive Geek

Nice post!!, Thanks for sharing knowledge. I was struggling to find a way to do service discovery with kubernetes, your post came in handy!!!

comments user
WH

Could you please show the setup of the “default-http-backend” and the ingress controller setup before ingress is working?

    comments user
    Piotr Mińkowski

    What you mean by show setup? It is all what you need inside Ingress declaration in yml file.

      comments user
      WH

      According to the Kubernetes ingress document, the ingress controller needs to be setup before any ingress works. But the ingress controller setup is not documented. Could you please show me all the steps to setup the ingress controller (including the “default-http-backend” object) in your example?

        comments user
        Piotr Mińkowski

        Did you try to set it up on your minikube instance? In my opinion you don’t have to do nothing more than steps which I described in my post.

comments user
Leonardo

Have i to create a mongoDB locally? Please, can you be more specific in that step?, i am a beginner and in this step “http://192.168.99.100:31638/accounts” with my port number i can’t access to the service, and in the step of the host, i am not using minikube environment, i am in my cluster of kubernetes directly, and i am trying to access to the services with nginx, but i can’t get the deployment yet

    comments user
    Piotr Mińkowski

    The first case is that it is deployed on Kubernetes with minikube. On Windows minikube start Kubernetes by default on 192.168.99.100. Mongo is starting there just by running its Docker image on Kubernetes where access port is assigned automatically.

comments user
Leonardo

yeah, but if i don’t want to use Minikube, i can deploy it directly in a kubernetes cluster, because i would like to adapt it, but when i use my public ip with the port of the accounts service, the page shows me this error:
There was an unexpected error (type=Internal Server Error, status=500).
{ “serverUsed” : “localhost:27017” , “ok” : 0.0 , “errmsg” : “Authentication failed.” , “code” : 18 , “codeName” : “AuthenticationFailed”}; nested exception is com.mongodb.CommandFailureException: { “serverUsed” : “localhost:27017” , “ok” : 0.0 , “errmsg” : “Authentication failed.” , “code” : 18 , “codeName” : “AuthenticationFailed”}
Could you help me?

    comments user
    Piotr Mińkowski

    If you have authentication it probably means you didn’t create user with credentials on MongoDB

      comments user
      Leonardo

      i tried to install mongo locally, creating an user, but i have the same issue, have i to run mongo on my pc ?

      comments user
      Asha Rajan

      Do we need to create a Mongo DB datastore? I thought the deployment should deploy mongoDB as a service using the docker image

        comments user
        Piotr Mińkowski

        Yes, I have also deployed it as a service

comments user
Ganesh Kumar

Hi

I tried your Yaml to create a ingress rule but I am not able able to access application, what could be the issue

# kubectl describe ing gateway-ingress
Name: gateway-ingress
Namespace: default
Address:
Default backend: default-http-backend:8080 ()
Rules:
Host Path Backends
—- —- ——–
kuber-poc-app1.com
/Review_JCS.html tomcat-svc:8080 ()
/KUBEADMIN-TOOL.html nginx-svc:80 ()
Annotations:
Events:
FirstSeen LastSeen Count From SubObjectPath Type Reason Message
——— ——– —– —- ————- ——– —— ——-
44m 44m 1 ingress-controller Normal CREATE Ingress default/gateway-ingress
44m 38m 2 ingress-controller Normal UPDATE Ingress default/gateway-ingress

Please advise

Best Regards
GAnesh

    comments user
    Ganesh Kumar

    W1109 08:31:14.931341 5 controller.go:794] error obtaining service endpoints: error getting service default/tomcat-svc from the cache: service default/tomcat-svc was not found
    W1109 08:31:14.931355 5 controller.go:794] error obtaining service endpoints: error getting service default/nginx-svc from the cache: service default/nginx-svc was not found

    comments user
    Piotr Mińkowski

    Ok, but did you create an application, expose it as services etc.?

comments user
Ganesh Kumar

I1110 09:02:12.163757 5 controller.go:316] ingress backend successfully reloaded…
I1110 09:02:21.389194 5 event.go:218] Event(v1.ObjectReference{Kind:”Ingress”, Namespace:”tomcat”, Name:”gateway-ingress”, UID:”dc6c23a8-c5f5-11e7-aece-00505601302d”, APIVersion:”extensions”, ResourceVersion:”1243451″, FieldPath:””}): type: ‚Normal’ reason: ‚CREATE’ Ingress tomcat/gateway-ingress
I1110 09:02:21.389510 5 controller.go:307] backend reload required
I1110 09:02:21.447376 5 controller.go:316] ingress backend successfully reloaded…
I1110 09:03:20.851318 5 status.go:364] updating Ingress tomcat/gateway-ingress status to [{ }]
I1110 09:03:20.854845 5 event.go:218] Event(v1.ObjectReference{Kind:”Ingress”, Namespace:”tomcat”, Name:”gateway-ingress”, UID:”dc6c23a8-c5f5-11e7-aece-00505601302d”, APIVersion:”extensions”, ResourceVersion:”1243538″, FieldPath:””}): type: ‚Normal’ reason: ‚UPDATE’ Ingress tomcat/gateway-ingress
I1110 09:45:38.080234 5 controller.go:307] backend reload required
I1110 09:45:38.081209 5 event.go:218] Event(v1.ObjectReference{Kind:”Ingress”, Namespace:”tomcat”, Name:”gateway-ingress”, UID:”dc6c23a8-c5f5-11e7-aece-00505601302d”, APIVersion:”extensions”, ResourceVersion:”1247174″, FieldPath:””}): type: ‚Normal’ reason: ‚UPDATE’ Ingress tomcat/gateway-ingress
I1110 09:45:38.175862 5 controller.go:316] ingress backend successfully reloaded…
I made Ingress run successfully but not able to access still
I checked entry for host its there

comments user
sp

Piotr, good example. Shows strengths of kubernetes well. I’ve gotten the docker containers up and running, but am getting auth issues on Mongo access. New to mongo, so pls elaborate on „create collection on mongo database and user micro/micro „

    comments user
    Piotr Mińkowski

    Well, did you run Mongo on docker? Then you need to create user for authentication. After all you should create account and customer collections

      comments user
      sailakshmi

      Can you update the steps here. To create user in mongo db

        comments user
        Piotr Mińkowski

        I have created user o Mongo with MongoClient

comments user
Asha Rajan

Can you update the steps here. To create user in mongo db .

Leave a Reply