Create and Manage Kubernetes Jobs in Go with client-go API

Introduction

The Kubernetes API Server provides a REST API interface for clients to interact with it and execute operations such as creating Deployments. But a client application does not have to understand and use the lower-level HTTP operations such as GET, PUT, POST, or PATCH because there are multiple client libraries for different languages. One of the most popular ones is the Go client library.

This guide covers creating a Job and Cron Job using the Kubernetes Go client. Before we get into the details, let's briefly understand these concepts.

Job

A Kubernetes Job can be used for types of applications that must be completed after running for a finite time. These short-lived tasks (such as batch jobs) might run as a one-off process or at regular intervals. Just like a Deployment, a Job also creates one or more Pods. For the Job to be considered complete, its Pods should terminate successfully. If a Job execution fails, Kubernetes retries the execution. Jobs can also be deleted or suspended -- in both cases, the Pods created by the Job are deleted.

Cron Job

A CronJob can be used for tasks that need to be run at a certain interval, such as once a day/week/month. A new Job object is created periodically based on a Cron format-based schedule.

Prerequisites

To work through this guide, you need the following:

• A Vultr Kubernetes Engine (VKE) cluster.

• Install Go programming language (version 1.18 or higher) on your local workstation.
• Install a recent version of kubectl on your local workstation.

Initialize the project

On your local machine, create a directory and switch to it:

mkdir k8s-jobs-go-client
cd k8s-jobs-go-client

Create a new Go module:

go mod init k8s-jobs-go-client

This will create a new go.mod file

Create a new file main.go:

touch main.go

Import libraries

To import required Go modules, add the following to main.go file:

package main

import (
    "context"
    "flag"
    "fmt"
    "log"
    "path/filepath"

    batchv1 "k8s.io/api/batch/v1"
    apiv1 "k8s.io/api/core/v1"

    metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"

    "k8s.io/client-go/kubernetes"
    "k8s.io/client-go/tools/clientcmd"
    "k8s.io/client-go/util/homedir"
)

Add the init function

Add the code below to main.go:

var clientset *kubernetes.Clientset

const jobName = "test-job-clientgo"
const cronJobName = "test-cronjob-clientgo"

func init() {
    var kubeconfig *string
    if home := homedir.HomeDir(); home != "" {
        kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
    } else {
        kubeconfig = flag.String("kubeconfig","  ", "absolute path to the kubeconfig file")
    }
    flag.Parse()

    config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
    if err != nil {
        log.Fatal(err)
    }
    clientset, err = kubernetes.NewForConfig(config)
    if err != nil {
        log.Fatal(err)
    }
}

The init function takes care of the following:

1. Access the config file in the .kube folder under the user's home directory.
2. Uses the path to kube config file to create a ***rest.Config** by invokin clientcmd.BuildConfigFromFlags.
3. Creates a kubernetes.Clientset by invoking the kubernetes.NewForConfig passing in the ***rest.Config**

Add the function to create a Job

Add the createJob function to main.go file:

func createJob() {
    fmt.Println("creating job", jobName)
    jobsClient := clientset.BatchV1().Jobs(apiv1.NamespaceDefault)

    job := &batchv1.Job{ObjectMeta: metav1.ObjectMeta{
        Name: jobName,
    },
        Spec: batchv1.JobSpec{
            Template: apiv1.PodTemplateSpec{
                Spec: apiv1.PodSpec{
                    Containers: []apiv1.Container{
                        {
                            Name: "pi",
                            Image: "perl:5.34.0",
                            Command: []string{"perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"},
                        },
                    },
                    RestartPolicy: apiv1.RestartPolicyNever,
                },
            },
        },
    }

    _, err := jobsClient.Create(context.Background(), job, metav1.CreateOptions{})
    if err != nil {
        log.Fatal("failed to create job", err)
    }

    fmt.Println("created job successfully")
}

If successful, This function creates a Job named test-job-clientgo

• First, it gets a v1.JobInterface instance by calling clientset.BatchV1().Jobs() with the default namespace
• Prepares a new Job definition
• Invokes JobInterface.Create to initiate Job creation

Add the main function

Finally, add the main function, which will invoke the createJob function.

func main() {
    createJob()
}

Run the program to create a new Job

Make sure you have configured kubectl to point to the Kubernetes cluster you want to use for this guide. For example, use kind cluster create to create a new Kubernetes cluster using kind. This will automatically configure kubectl as well.

kubectl get nodes

You should see the following output:

NAME                 STATUS   ROLES           AGE   VERSION
kind-control-plane   Ready    control-plane   23h   v1.24.0

Run the program:

go run main.go

You should see the following output:

creating job test-job-clientgo
created job successfully

Confirm that the Job was created:

kubectl get job/test-job-clientgo

You should see the following output:

NAME                COMPLETIONS   DURATION   AGE
test-job-clientgo   1/1           7s         66s

The Job created a Pod that took around 7s seconds to compute the value of Pi (to 2000 places) and print it out.

To confirm the result, check the Pod logs:

kubectl logs $(kubectl get pods --selector=job-name=test-job-clientgo --output=jsonpath='{.items[*].metadata.name}')

Add the function to delete a Job

Add the deleteJob function to main.go:

func deleteJob() {
    fmt.Println("deleting job", jobName)

    jobsClient := clientset.BatchV1().Jobs(apiv1.NamespaceDefault)

    pp := metav1.DeletePropagationBackground
    err := jobsClient.Delete(context.Background(), jobName, metav1.DeleteOptions{PropagationPolicy: &pp})
    if err != nil {
        log.Fatal("failed to delete job", err)
    }
    fmt.Println("deleted job successfully")
}

Update the main function: Comment out the createJob function and add the deleteJob function.

func main() {
    //createJob()
    deleteJob()
}

Run the program to delete an existing Job

To run the program:

go run main.go

You should see the following output:

deleting job test-job-clientgo
deleted job successfully

Confirm that the Job was deleted:

kubectl get job/test-job-clientgo

You should see the following output:

Error from server (NotFound): jobs.batch "test-job-clientgo" not found

Confirm that Pod created by the Job was also deleted:

kubectl get pods --selector=job-name=test-job-clientgo

You should see the following output:

No resources found in default namespace.

Add the function to create a CronJob

Add the createCronJob function to main.go:

func createCronJob() {
    fmt.Println("creating cron job", cronJobName)

    cronJobsClient := clientset.BatchV1().CronJobs(apiv1.NamespaceDefault)

    cronJob := &batchv1.CronJob{ObjectMeta: metav1.ObjectMeta{
        Name: cronJobName},
        Spec: batchv1.CronJobSpec{
            Schedule: "* * * * *",
            JobTemplate: batchv1.JobTemplateSpec{
                Spec: batchv1.JobSpec{
                    Template: apiv1.PodTemplateSpec{
                        Spec: apiv1.PodSpec{
                            Containers: []apiv1.Container{
                                {
                                    Name: "hello",
                                    Image: "busybox:1.28",
                                    Command: []string{"/bin/sh", "-c", "date; echo Hello from the Kubernetes cluster"},
                                },
                            },
                            RestartPolicy: apiv1.RestartPolicyOnFailure,
                        },
                    },
                },
            },
        },
    }

    _, err := cronJobsClient.Create(context.Background(), cronJob, metav1.CreateOptions{})
    if err != nil {
        log.Fatal("failed to create cron job", err)
    }

    fmt.Println("created cron job successfully")
}

If successful, This function creates a CronJob named test-cronjob-clientgo.

• First, it gets a v1.CronJobInterface instance by calling clientset.BatchV1().CronJobs() with the default namespace
• Prepares a new CronJob definition
• Invokes CronJobInterface.Create to initiate CronJob creation

Update the main function: Comment out the deleteJob function and add the createCronJob function.

func main() {
    //createJob()
    //deleteJob()
    createCronJob()
}

Run the program to create a CronJob

To run the program:

go run main.go

You should see the following output:

creating cron job test-cronjob-clientgo
created cron job successfully

Confirm that the CronJob was created:

kubectl get cronjob/test-cronjob-clientgo

You should see the following output:

NAME                    SCHEDULE    SUSPEND   ACTIVE   LAST SCHEDULE   AGE
test-cronjob-clientgo   * * * * *   False     0        <none>          17s

This CronJob creates a Pod every minute and prints the current time along with a message. Check the pods:

kubectl get pods

You should see the following output:

NAME                                   READY   STATUS      RESTARTS   AGE
test-cronjob-clientgo-27804391-qcjzd   0/1     Completed   0          2m49s

You should see Pods with names starting with test-cronjob-clientgo. To check the Pod logs:

kubectl logs <enter Pod name>

You should see an output with the current date along with a message:

Fri Sep 12 14:34:00 UTC 2022
Hello from the Kubernetes cluster

Add the function to delete a CronJob

Add the deleteCronJob function:

func deleteCronJob() {
    fmt.Println("deleting cron job", cronJobName)

    cronJobsClient := clientset.BatchV1().CronJobs(apiv1.NamespaceDefault)

    pp := metav1.DeletePropagationBackground
    err := cronJobsClient.Delete(context.Background(), cronJobName, metav1.DeleteOptions{PropagationPolicy: &pp})
    if err != nil {
        log.Fatal("failed to delete cron job", err)
    }
    fmt.Println("deleted cron job successfully")
}

Update the main function: Comment out createCronJob function and add the deleteCronJob function.

func main() {
    //createJob()
    //deleteJob()
    //createCronJob()
    deleteCronJob()
}

Run the program to delete an existing CronJob

Run the program:

go run main.go

You should see the following output:

deleting cron job test-cronjob-clientgo
deleted cron job successfully

Confirm that the Job was deleted:

kubectl get cronjob/test-cronjob-clientgo

You should see the following output:

Error from server (NotFound): cronjobs.batch "test-cronjob-clientgo" not found

Confirm that Pod created by the cron Job was also deleted

kubectl get pods

You should not see Pods with names starting with test-cronjob-clientgo.

Conclusion

This guide covered how to create and delete a Job and CronJob using the Kubernetes Go client library.