Implement a CI/CD Pipeline with Jenkins and Vultr Kubernetes Engine

Introduction

Rapid and reliable delivery of applications is an important aspect of software development that ensures the timely execution of solutions in production environments. Jenkins is a popular open-source automation service that enables Continuous Integration (CI) and Continuous Deployment (CD) in software development. It plays a crucial role in automating the application building, testing, and deployment processes to streamline the software development lifecycle.

Implementing a CI/CD pipeline with Jenkins and Kubernetes offers many software development and delivery advantages. These include streamlining development processes, enhancing collaboration, and speed up the delivery of high-quality applications. In this guide, you will deploy a Node.js containerized application to a Vultr Kubernetes Engine (VKE) cluster using the Jenkins CI/CD Pipeline.

Prerequisites

Before you start, you need to:

• Deploy a Vultr Kubernetes Engine (VKE) cluster with at least 3 nodes.
• Create a Vultr Container Registry to store Docker images.
• Deploy a One-Click Docker instance with at least 4 GB RAM and 2 vCPUs to use as the management server.
• Access the server using SSH as a non-root user with sudo privileges.
• Install and Configure Kubectl to access the VKE cluster.
• Log in to your GitHub account and generate a personal access token to access your repositories using Git.

Install Jenkins

1. Install the Java OpenJDK 11 dependecy package.

   console Copy

   $ sudo apt install openjdk-11-jdk -y
   

2. Download the Jenkins repository key to your server.

   console Copy

   $ curl -fsSL https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key | sudo tee /usr/share/keyrings/jenkins-keyring.asc > /dev/null
   

3. Add Jenkins repository to APT repository sources.

   console Copy

   $ echo deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc] https://pkg.jenkins.io/debian-stable binary/ | sudo tee /etc/apt/sources.list.d/jenkins.list > /dev/null
   

4. Update the server repository index.

   console Copy

   $ sudo apt update -y
   

5. Install Jenkins on the server.

   console Copy

   $ sudo apt install jenkins -y
   

6. View the Jenkins status to verify that it's successfully installed and running.

   console Copy

   $ sudo systemctl status jenkins
   

   Output:

   ● jenkins.service - Jenkins Continuous Integration Server
        Loaded: loaded (/lib/systemd/system/jenkins.service; enabled; vendor preset: enabled)
        Active: active (running) since Wed 2023-11-15 06:23:57 UTC; 5min ago
      Main PID: 5491 (java)
         Tasks: 36 (limit: 1006)
        Memory: 313.3M
           CPU: 1min 12.574s
        CGroup: /system.slice/jenkins.service
                └─5491 /usr/bin/java -Djava.awt.headless=true -jar /usr/share/java/jenkins.war --webroot=/var/cache/jenkins/war --httpPort=8080
   
   Nov 15 06:22:50 ubuntu jenkins[5491]: ff1f768ed659416695b3decdff0e9473
   Nov 15 06:22:50 ubuntu jenkins[5491]: This may also be found at: /var/lib/jenkins/secrets/initialAdminPassword
   

7. Add the Jenkins user to the Docker users group to enable communications with the Docker daemon.

   console Copy

   $ sudo usermod -aG docker jenkins
   

8. Allow the default Jenkins port 8080 through the default UFW firewall to enable access to the web interface.

   console Copy

   $ sudo ufw allow 8080
   

9. View the default Jenkins administrator password.

   console Copy

   $ sudo cat /var/lib/jenkins/secrets/initialAdminPassword
   

   Copy the generated password similar to the output below:

   243e47a2921746db9f14c95e264b3eb1

10. Access the Jenkins port 8080 using your Server IP Address to access the Jenkins dashboard in a web browser such as Google Chrome.

    http://SERVER-IP:8080

11. Enter the default password you copied earlier and click Continue to start the Jenkins Set Up process.

    

12. Click Install suggested plugins to start installing all necessary Jenkins packages on your server.

    

13. Enter a new Jenkins administrator username and password. Then, click Save and Continue to access the instance configuration page.

    

14. Keep your Server IP address structure in the Jenkins URL field, and click Save and Finish to save the access URL structure.

    

15. Click Start using Jenkins to finish the configuration process and access the Jenkins dashboard.

    

Install Jenkins Plugins

To integrate Jenkins with your VKE cluster, GitHub, and the Vultr container registry, install all necessary plugins to enable all pipeline resources. Follow the below steps to install all required Docker, GitHub, and Kubernetes plugins within your Jenkins dashboard.

1. Access your Jenkins dashboard.

2. Navigate to Manage Jenkins on the main navigation menu.

3. Click Manage Plugins to access the plugins page.

4. Click Available Plugins, and enter the keyword Docker in the search field.

5. Select Docker, Docker Pipeline, and docker-build-step from the plugin search results.

   

6. Enter Kubernetes in the search field, and select Kubernetes, Kubernetes Credentials, Kubernetes CLI, and Kubernetes Pipeline from the plugin results.

7. Click Install to start the installation process.

8. Scroll and check the Restart Jenkins when installation is complete and no jobs are running option on the installation progress page to automatically restart Jenkins.

   

Create a New GitHub Repository Webhook

A GitHub repository webhook triggers Jenkins to automatically start the build process whenever push and pull events occur in the linked repository. This integration facilitates Continuous Integration and Delivery (CI/CD) workflows. Follow the steps below to create a new GitHub repository webhook to integrate with your Jenkins dashboard.

1. Log in to your GitHub account.

2. Click the + sign in the top right corner and select New repository to set up a new repository.

3. Enter your desired repository name, for example, node-application. Then, choose your desired visibility type, and click Create repository to publish the new repository.

   

4. Navigate to the repository page, and click Settings on the top navigation bar.

5. Click Webhooks on the left navigation menu.

   

6. Click Add webhook to set up a new webhook.

   

   • Enter your Jenkins server URL in the Payload URL to receive payloads.
   • Keep the Just the push event radio button selected.

7. Click Add webhook to apply your new repository webhook.

Create New Jenkins Credentials

To communicate with your GitHub repository, the Vultr Container Registry, and your VKE cluster, set up new credentials within your Jenkins dashboard to store the access details for each platform as described in the steps below.

1. Navigate to your Jenkins dashboard and click Manage Jenkins.

2. Click Credentials from the list of available options to set up a new profile.

   

3. Click the (global) Domains value and select Add credentials from the list of options.

   

4. Keep Username with password as the credential type and enter your Vultr Container Registry username and password in the respective fields.

5. Enter vcr-credentials in the ID field and click Create to save the new credential.

   

6. When successful, click Add credentials to set up a new GitHub credential.

7. Keep Username with password selected as the credential type, and enter your GitHub username, and password in the respective fields.

8. Enter github-credential in the ID field and click Create to save the new credential.

   

9. When successful, click Add credentials to set up a new GitHub credential. To set up the Kubernetes credential, click Add credentials.

10. Click the Kind drop-down and select Secret file from the list of options.

11. Click the Choose File button in the File section, browse, and upload your VKE cluster YAML file.

12. Enter kubeconfig in the ID field and click Create to save the new credential.

    

Create a Node.js Application

1. Clone your GitHub node-application repository you created earlier.

   console Copy

   $ git clone https://github.com/example-user/node-application.git
   

2. Switch to the new repository directory.

   console Copy

   $ cd node-application
   

3. Create a new package.json file to define the application dependencies.

   console Copy

   $ nano package.json
   

4. Add the following configurations to the file.

   json Copy

   {
   
     "name": "simple-node-app",
     "version": "1.0.0",
     "description": "Node Hello World Application",
     "main": "app.js",
     "scripts": {
       "start": "node app.js"
     },
     "dependencies": {
       "express": "4.17.1"
     }
   }
   

   Save and close the file.

5. Create a new app.js file to define the application structure.

   console Copy

   $ nano app.js
   

6. Add the following contents to the file.

   js Copy

   'use strict';
   const express = require('express');
   const PORT = process.env.PORT || 8080;
   const app = express();
   app.get('/', function (req, res) {
     res.send('Node Hello World Application!\n');
   });
   app.listen(PORT);
   console.log('Running on http://localhost:' + PORT);
   

   Save and close the file.

   The above Node.js application code sets up a basic server that listens on port 8080 and responds with Node Hello World Application! when accessed.

7. Create a new Dockerfile to define the application image structure.

   console Copy

   $ nano Dockerfile
   

8. Add the following configurations to the file.

   dockerfile Copy

   FROM node:18-alpine as builder
   ENV NODE_ENV="production"
   
   # Copy app's source code to the /app directory
   COPY . /app
   
   # The application's directory will be the working directory
   WORKDIR /app
   
   # Install Node.js dependencies defined in '/app/package.json'
   RUN npm install
   FROM node:18-alpine
   ENV NODE_ENV="production"
   COPY --from=builder /app /app
   WORKDIR /app
   ENV PORT 8080
   EXPOSE 8080
   
   # Start the application
   CMD ["npm", "start"]
   

   Save and close the file.

Set Up the Pipeline Kubernetes Resources

In this section, create the necessary Kubernetes resources that pull your Node.js application container image from the Vultr Container Registry and deploy it to your cluster. Each time Jenkins starts the build process, the resource files run again to update your cluster.

1. Create a deployment.yml file.

   console Copy

   $ nano deployment.yml
   

2. Add the following configurations to the file. Replace example-user with your actual GitHub username.

   yaml Copy

   kind: Deployment
   apiVersion: apps/v1
   metadata:
     name: node-app
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: node-app
     template:
       metadata:
         labels:
           app: node-app
       spec:
         containers:
         - name: node-app
           image: example-user/node-app:latest
           ports:
           - containerPort: 8080
   

   Save and close the file.

   The above deployment configuration defines your application replicas, Docker image, and port to expose the application within the cluster.

3. Create a new Service resource file service.yml to expose the Node.js application outside the Kubernetes cluster.

   console Copy

   $ nano service.yml
   

4. Add the following contents to the file.

   yaml Copy

   apiVersion: v1
   kind: Service
   metadata:
     name: load-balancer
     labels:
       app: node-app
   spec:
     type: LoadBalancer
     ports:
     - port: 8080
       targetPort: 8080
     selector:
       app: node-app
   

   Save and close the file.

Create the Pipeline Jenkinsfile

A Jenkinsfile file defines a Jenkins Pipeline for your CI/CD processes. It includes the entire build process steps from pulling the source code to deploying the application to your cluster. Follow the steps in this section to set up a new Jenkinsfile that includes all build processes to prepare the Node.js application for deployment in your cluster.

1. Create a new Jenkinsfile.

   console Copy

   $ nano Jenkinsfile
   

2. Add the following configurations to the file. Replace https://sjc.vultrcr.com/jenkinsregistry with your actual Vultr Container Registry URL.

   json Copy

   pipeline {
   
   agent any
    environment {
        PROJECT_ID = 'jenkins'
        CLUSTER_NAME = 'cluster'
        LOCATION = 'usa'
        CREDENTIALS_ID = 'vultr'
    }
    stages {
        stage("Checkout code") {
            steps {
                checkout scm
            }
        }
        stage("Build image") {
            steps {
                script {
                    myapp = docker.build("sjc.vultrcr.com/jenkinsregistry/node-app:${env.BUILD_ID}")
                }
            }
        }
        stage("Push image") {
            steps {
                script {
                    docker.withRegistry('https://sjc.vultrcr.com/jenkinsregistry', 'vcr-credentials') {
                            myapp.push("latest")
                            myapp.push("${env.BUILD_ID}")
                    }
                }
            }
        }
   
    stage("Deploy to Vultr Kubernetes") {
   
        steps {
            sh "sed -i 's/node-app:latest/node-app:${env.BUILD_ID}/g' deployment.yml"
                    script {
        withKubeConfig([credentialsId: 'kubeconfig']) 
            {
          sh "kubectl apply -f deployment.yml"
          sh "kubectl apply -f service.yml"
   
          }                
        }
   
      }
    }
   
    }
    }
   

   Save and close the file.

   Below is what the above pipeline executes:

   • agent any: Specifies that the pipeline can run on any available agent (Jenkins node).
   • stage("Checkout code"): Checks out the source code from the version control system (specified by scm), and enables the pipeline to access the application code.
   • stage("Build image"): Builds a new Docker image of the Node.js applicatiom tagged with a version identifier based on the Jenkins build number (${env.BUILD_ID}).
   • stage("Push image"): Pushes the application Docker image to your defined registry sjc.vultrcr.com/jenkinsregistry using the vcr-credentials variable.
   • stage("Deploy to Vultr Kubernetes"): Updates the Kubernetes deployment configuration deployment.yml to use the new Docker image tagged with the latest Jenkins build number. Then, it applies the updated deployment and service configurations to your cluster using kubectl.
   • sed: Replaces all occurrences of the previous image tag node-app:latest with the new image tag node-app:${env.BUILD_ID} in the deployment.yml file.
   • withKubeConfig: Specifies the Kubernetes configuration kubeconfig to use in the pipeline processes. The sh command runs kubectl to deploy the deployment and service configurations to your VKE cluster.

   ![NOTE] If you applied different ID values in your Jenkins dashboard, replace vcr-credentials with your Vultr Container Registry credential, and kubeconfig with your actual cluster credential ID.

Upload the Application Files to your GitHub Repository

All files, including your Node.js application, Dockerfile, Jenkinsfile, and Kubernetes resources, are ready for deployment. Configure Git to commit and push all files from your management machine to the GitHub repository as described in the steps below.

1. Configure a global username to associate with your Git operations. Replace Example User with your actual username.

   console Copy

   $ git config --global user.name "Example User"
   

2. Configure a global email. Replace user@example.com with your active email address.

   console Copy

   $ git config --global user.name "user@example.com"
   

3. Initialize the Git repository.

   console Copy

   $ git init
   

4. Add your GitHub repository as the remote repository to push changes.

   console Copy

   $ git remote add origin https://github.com/example-user/node-application.git
   

5. Stage all files in your working directory.

   console Copy

   $ git add .
   

6. Commit the staged changes with a descriptive commit message of your choice. For example, Initial Commit.

   console Copy

   $ git commit -m "Initial commit"
   

7. Push the changes to your GitHub repository.

   console Copy

   $ git push origin HEAD:master
   

   When prompted, enter your GitHub username and personal access token to successfully push changes to your repository.

8. When successful, visit your GitHub repository and verify that the new files are available.

   

Create a Jenkins Pipeline for Kubernetes Deployment

At this point, you have Jenkins integrated with the Kubernetes cluster. Now, you will create a Jenkins pipeline to deploy your application. Follow the steps below to create a new pipeline.

1. Access your Jenkins dashboard and click New Item.

2. Enter a name for your project. For example myproject.

3. Select Multibranch Pipeline, and click OK to save changes.

   

4. Apply the following changes on the project configuration page:

   • Enter a name and description for your new project.
   • In the Branch Sources section, click the Add Source drop-down and select GitHub.
   • Select your Jenkins GitHub credentials ID.

   

   • Enter your GitHub repository URL in the Repository HTTPS URL field.
   • Scroll to the Properties section, and enter your desired Docker label.
   • Enter your Vultr Container Registry URL in the Docker registry URL field, and click the Registry credentials drop-down to select the credential you created earlier.

   

5. Click Save to apply your new project configuration.

6. Monitor the new Jenkins scan progress and verify that the scan log includes a FINISHED SUCCESS result.

   

Run the Jenkins Pipeline

1. Navigate to the pipeline project Status page.

2. Verify that a green checkmark is available and the Last Success field includes a time value that represents a successful build process.

   

3. Click the main value to view the pipeline processes page and verify that all processes are successful.

   

4. Access your server terminal session to verify the new deployment in your cluster.

   console Copy

   $ kubectl get deployment
   

   Output:

   NAME       READY   UP-TO-DATE   AVAILABLE   AGE
   node-app   1/1     1            1           36m

   As displayed in the above output, the Node.js application deployment is available in your cluster.

5. View the cluster services and verify that a new LoadBalancer service is available.

   console Copy

   $ kubectl get services
   

   Output:

   NAME            TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)          AGE
   kubernetes      ClusterIP      10.96.0.1        <none>          443/TCP          5h4m
   load-balancer   LoadBalancer   10.101.168.242   172.16.10.10   8080:32732/TCP   35m

   Keep note of the Node.js application LoadBalancer EXTERNAL-IP value to use when accessing your Node.js application.

6. Access your Node.js application using the Load Balancer IP on port 8080. Replace the example IP 172.16.10.10 with your actual IP Address.

   http://172.16.10.10:8080

   Verify that the Node Hello World Application prompt displays in your browser session.

   

Verify the CI/CD Pipeline

1. Switch to your node-application directory.

   console Copy

   $ cd node-application
   

2. Change the Node Hello World Application message in the app.js file using the sed command to the Node Hello World Application Updated.

   console Copy

   $ sed -i 's/Node Hello World Application/Node Hello World Application Updated/g' app.js
   

3. Stage changes in your local repository.

   console Copy

   $ git add .
   

4. Commit new changes.

   console Copy

   $ git commit -m "Modify Node App"
   

5. Push the changes to your GitHub repository.

   console Copy

   $ git push origin HEAD:master
   

6. Wait for at least 2 minutes for Jenkins to trigger the pipeline, rebuild the Node.js container image, and deploy it to your Kubernetes cluster.

7. When the pipeline execution process is successful. Visit your cluster Load Balancer IP to view the updated application.

   

Conclusion

You have implemented a CI/CD pipeline with Jenkins on a Vultr Kubernetes Engine (VKE) cluster. By setting up a CI/CD pipeline with Jenkins and Kubernetes, you can generate an efficient, reliable, and scalable software delivery cycle. This integration automates the software deployment process and allows you to focus on innovation and collaboration. For more information, visit the official Jenkins documentation.