As I promised in the last post, I’ll show you how to set up our first Kubernetes application deployment. For the purpose of this demo, I have selected the Jenkins. I hope this tool is well known by all of you interested in modern DevOps CI/CD approach. Building the Jenkins on the top of Kubernetes is not only very easy, but also extremely powerful because it gives us the possibility to autoscale our build environment by automatically spin up new docker instances for the requested jobs.

minikube1

In my previous posts, I’ve been focused on building Kubernetes components based on Rancher and AWS. But for the purpose of this post, I’ll use Minikube which is extremely easy to use Kubernetes “sandbox”.

Before you can start using Minikube, you need to install a few dependencies. I’ve have done this on my Ubuntu 18.04 LTS Linux and these steps are intended for this environment. If you’re using Windows, MacOS or other Linux distro, please refer to official Minikube documentation available at https://kubernetes.io/docs/tasks/tools/install-minikube/

# Installing VirtualBox and Curl
$ sudo apt-get update 
$ sudo apt-get install virtualbox virtualbox-ext-pack curl -y
# Accept the license agreement during the installation and default options

# Download the Minikube
$ sudo curl -Lo /usr/local/bin/minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
$ sudo chmod +x /usr/local/bin/minikube-test

# Download the Kubectl
$ sudo curl -Lo /usr/local/bin/kubectl https://storage.googleapis.com/kubernetes-release/release/v1.8.0/bin/linux/amd64/kubectl
$ sudo chmod +x /usr/local/bin/kubectl

Now the installation is done. Time to start the Minikube and wait a while until it will spin up new VM under VirtualBox and configure it.

$ minikube start

minikube2

Minikube will automatically configure kubectl and other things for you. You can run below command to check whether the cluster is operational

$ kubectl get nodes

minikube3

Before we start to deploy our application to Kubernetes, we must define our Docker image with Jenkins. I have prepared below Dockerfile which uses official docker image as well as install kubernetes plugins and some other tools.

FROM jenkins/jenkins:lts

# Sets the number of jenkins executors to 0
COPY executors.groovy /usr/share/jenkins/ref/init.groovy.d/executors.groovy

# Install kubernetes plugin - scaling
RUN /usr/local/bin/install-plugins.sh kubernetes

# Install Maven and net-tools 
USER root

RUN apt-get update && apt-get install -y maven net-tools

USER jenkins

This file requires also file named executors.groovy, which sets the number of jenkins executors to 0 (from default 2). We want to have only external executors based on our docker images.

import jenkins.model.*
Jenkins.instance.setNumExecutors(0)

Before you start to build a new image, I’d recommend to you use below command which allows you to use locally built images straight into the Minikube environment. Otherwise, you must build, tag and push the image to an external repository e.g. hub.docker.com, from where the Kubernetes can pull the image.

$ eval $(minikube docker-env)

Now let’s build our image (Make sure you’re in the folder where Dockerfile and executors.groovy files exist). It will take a while because it must pull original Jenkins image from Dockerhub and compile the new one. The time depends on your internet speed.

# You can use your own tag (-t)
$ docker build -t mycloudfun/jenkins:1.0 .

You can skip these steps and use my image available as mycloudfun/jenkins:1.0 at Dockerhub.

When our image is ready to use, we can start to configure our Jenkins on top of the Kubernetes cluster. To do this, we must define a few components: Deployment, Service, Ingress, and Role. The source code is available at my Github repository https://github.com/mycloudfun/kubernetes-jenkins

Simply grab my code from the Github and run below command to start the application

$ kubectl create -f jenkins-cluster.yaml

Let’s walk through created components and describe them a bit. First one is Deployment

apiVersion: apps/v1 
kind: Deployment
metadata:
  name: jenkins
spec:
  replicas: 1
  selector:
   matchLabels:
    app: jenkins
  template:
    metadata:
      labels:
        app: jenkins
    spec:
      containers:
      - name: jenkins
        image: mycloudfun/jenkins:1.0
        env:
         - name: JAVA_OPTS
           value: "-Djenkins.install.runSetupWizard=false"
         - name: JENKINS_OPTS
           value: "--prefix=/jenkins"
        ports:
        - name: http-port
          containerPort: 8080
        - name: jnlp-port
          containerPort: 50000
        volumeMounts:
          - name: jenkins-home
            mountPath: /var/jenkins_home
      volumes:
        - name: jenkins-home
          emptyDir: {} 

Here I define the POD declaration. First, I set the number of running replicas to 1, which is sufficient for this type of application. I’m going to use my newly created image mycloudfun/jenkins:1.0. I inject two environment variables. The “JAVA_OPTS” with a given value is responsible for starting our application without configuration wizard. The “JENKINS_OPTS” with a configured prefix, will make sure that our application is available from the URL behind the ingress. Next, I open the connection to this POD on ports 8080 (http access) and 50000 (JNLP communication). Finally, I set the persistent volume to make sure that our settings are will survive the POD fails.

The next component is Service.

apiVersion: v1
kind: Service
metadata:
  name: jenkins-service
spec:
  ports:
    - port: 80
      targetPort: 8080
      protocol: TCP
      name: http
    - port: 50000
      targetPort: 50000
      protocol: TCP
      name: jnlp
  selector:
    app: jenkins

Simply speaking, the service is responsible for manage the access to our POD from outside the Node. I bind this service with my deployment jenkins based on label “jenkins” and configured ports.

Because I’m running my Kubernetes cluster with a kind of on-premise solution, I need to setup ingress to make sure we can access our application. Of course, we can use only service, but then we must find the randomly assigned port to our service in order to access the Jenkins. Ingress gives us also more interesting capabilities, which are out of the scope of this tutorial.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: jenkins-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: "/jenkins"
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
    nginx.ingress.kubernetes.io/configuration-snippet: |
      rewrite ^(/jenkins)$ http://$best_http_host$1/ permanent;
spec:
  rules:
  - http:
      paths:
        - path: /jenkins
          backend:
            serviceName: jenkins-service
            servicePort: 8080

As you can see, I’m using some tricks in ingress, which allows me to access my Jenkins using URL: http://minikube/jenkins/

Finally, we need an extra role to be created in order to allow Jenkins to spin up new instances in our cluster.

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: fabric8-rbac
subjects:
  - kind: ServiceAccount
    # Reference to upper's `metadata.name`
    name: default
    # Reference to upper's `metadata.namespace`
    namespace: default
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

That’s pretty much all about Kubernetes configuration for now. We can verify the status of our components using below set of commands

minikube4

Now open your browser with the address of your http://minikube_ip/jenkins/, which is usually 192.168.99.100 but can be different. You can always check it using the command:

$ minikube ip

You will see the Jenkins main dashboard similar to one below.

minikube5

If you go to Manage Jenkins -> Manage Plugins -> Installed, you will see that our instance has already kubernetes plugin installed. It means that our image is working good.

minikube6

Now I must configure the Jenkins to use our Kubernetes images for distributed builds. Go to Manage Jenkins -> Configure System and scroll down to the bottom of this page. You can see the section Cloud. Click on Add a new cloud and select Kubernetes. Fill in the below fields:

Name: kubernetes

Kubernetes URL: https://kubernetes

Jenkins URL: http://jenkins_pod_ip:8080/jenkins/

You can find the IP of your jenkins POD using commands:

$ kubectl get pods
$ kubectl describe pod jenkins-<some numbers> |grep IP:

minikube7

You can verify the connection by clicking on the Test Connection button

minikube8

Next, scroll down to Images section and click on Add Pod Template. Fill in the below fields as follow:

Name: jenkins-slave

Namespace: default

Labels: jenkins-slave

Usage: Use this node as much as possible

Containers:

Name: jenkins-slave

Docker image: jenkins/jnlp-slave

Other leave as default.

minikube9

Save the configuration. Now let’s create a simple job and verify if my configuration works. Click on the New Item → Enter the name and select Freestyle project

minikube10

Unselect the Restrict where this project can be run. In the Build section, click on Add build step -> Execute Shell, and type few commands like

minikube11

Next, save and click on “Build Now”. The build process will start, spinning up new docker images. You can see it in the Build Executor Status in main dashboard as well as using kubectl command. For the first time, it will take some time as the Kubernetes must pull new jenkins/jnlb-slave image.

minikube12

minikube13

Finally, click on build and the Console Output to see your job execution results:

minikube14

As you can see everything works like a charm! I hope you share my excitement on how the Kubernetes works and how easily it scales our Jenkins workloads.

Thank you for your time and I want to invite you for my next posts.