How to Configure Kubernetes Readiness and Liveness Probes

This article explains the readiness and liveness probes in Kubernetes and how you can configure them for a container.

What Is a Probe in Kubernetes

Applications can become unreliable because of many reasons. You need to make sure that the application is in the desired state. Kubernetes offers a way to check the health of a container. This health diagnosis is called a probe.

Ways to Probe a Container

A probe is done by the kubelet upon its container using one of the following ways:

1. exec

Kubelet executes a command within the container. The result depends on the exit code of the command. An exit code of 0 indicates a successful diagnosis.

2. httpGet

Kubelet sends a GET request to the specified port of the container's IP address. A response code between 200 and 400 (inclusive) indicates a successful diagnosis.

3. tcpSocket

Kubelet attempts to establish a TCP connection on the specified port of the container's IP address. An established connection indicates a successful diagnosis.

Types of Probes

There are three types of probes in Kubernetes:

1. Liveness probe
2. Readiness probe
3. Startup probe

The startup probe is a recent addition to this list, and it is not discussed in this article.

Liveness Probe

The liveness probe detects if the container is alive. This is used to ensure the availability of a pod. The kubelet does nothing if the liveness probe succeeds because the container is already in its desired state. If it fails, the kubelet kills the container, and the further course of action depends upon the specified restart policy of the container.

Why Use Liveness Probe?

Kubelet checks a container's health by looking at the status of its PID 1. The problem is that the state of PID 1 does not reflect the health of the child processes of a container. You don't need to specify a liveness probe if you have a single process container. Otherwise, you should use a liveness probe.

Readiness Probe

A readiness probe detects if the container is ready to serve the incoming network requests. You do not want to send traffic to a pod that is not ready to function. If the readiness probe fails, the control pane stops the network traffic to the pod. Before the first readiness probe is run on the pod, the default readiness state is set as Failure. The readiness probe is run periodically on the container during its entire lifecycle.

Why Use Readiness Probe?

Network traffic should be sent to the container once all the child processes start. Without a readiness probe, Kubernetes will send network traffic to the container as soon as PID 1 starts. This does not mean that the child processes have started. Sending network traffic to the container in this state can cause errors. The application will face errors every time the container restarts or a new instance of the application is created.

How to Configure a Probe in Kubernetes

A probe is defined inside the pods.spec.containers field. Every probe has the following five parameters:

1. initialDelaySeconds: This specifies the time (in seconds) to wait after the container starts for the probe to run. The default value is 0.
2. periodSeconds: This specifies the time interval between each periodic probe. The default value is 10.
3. timeoutSeconds: This specifies the time each probe has to wait to get the response. The default value is 0.
4. successThreshold: This specifies the count of successful probes that would indicate a successful diagnostic. The default value is 1.
5. failureThreshold: This specifies the count of failed probes that would indicate a failed diagnostic. The default value is 3.

All these parameters are used to configure the probe. To follow along with this guide, you will need a Kubernetes cluster and kubectl client configured. You can use Vultr Kubernetes Engine to deploy a Kubernetes cluster in the cloud and test the probes. Configure kubectl on your machine and follow along.

Creating a Liveness Probe

Look at the following configuration file:

apiVersion: v1
kind: Pod
metadata:
  labels:
    test: liveness
  name: liveness-demo
spec:
  containers:
  - name: basic-container
    image: registry.k8s.io/busybox
    args:
    - /bin/sh
    - -c
    - touch /tmp/healthy; sleep 20; rm -f /tmp/healthy; sleep 1000
    livenessProbe:
      exec:
        command:
        - cat
        - /tmp/healthy
      initialDelaySeconds: 5
      periodSeconds: 10

This configuration creates a single container pod and defines a liveness probe. This probe uses exec to perform the diagnosis. After the container is created, touch /tmp/healthy; sleep 20; rm -f /tmp/healthy; sleep 100 command is executed. This command will:

• Create a /tmp/healthy file
• Wait for 20 seconds.
• Delete the /tmp/healthy file.
• Keep the container alive for another 1000 seconds.

Notice the livenessProbe field under the containers field. This is where you define the liveness probe. Here, the liveness probe will wait for 5 seconds after the container is created to start the periodic diagnosis at a time interval of 10 seconds. It uses the exec way to perform the probe. If the command cat /tmp/healthy returns 0, the probe is successful.

Create the pod:

kubectl apply -f config.yaml

Check the pod's events:

kubectl describe liveness-demo

At the end of the output, you can expect to see something like:

Events:
    Type    Reason     Age   From               Message
      ----    ------     ----  ----               -------
    Normal  Scheduled  25s   default-scheduler  Successfully assigned default/liveness-demo to probescluster-3a397dd3c5ec
    Normal  Pulling    25s   kubelet            Pulling image "registry.k8s.io/busybox"
    Normal  Pulled     22s   kubelet            Successfully pulled image "registry.k8s.io/busybox" in 3.292188263s
    Normal  Created    22s   kubelet            Created container basic-container
    Normal  Started    22s   kubelet            Started container basic-container

Till now, the pod is healthy and running. After 25 seconds, check the pod's events again:

kubectl apply -f config.yaml

You can expect to see something like:

Events:
    Type     Reason     Age               From               Message
      ----     ------     ----              ----               -------
    Normal   Scheduled  56s               default-scheduler  Successfully assigned default/liveness-demo to probescluster-3a397dd3c5ec
    Normal   Pulling    55s               kubelet            Pulling image "registry.k8s.io/busybox"
    Normal   Pulled     52s               kubelet            Successfully pulled image "registry.k8s.io/busybox" in 3.292188263s
    Normal   Created    52s               kubelet            Created container basic-container
    Normal   Started    52s               kubelet            Started container basic-container
    Warning  Unhealthy  5s (x3 over 25s)  kubelet            Liveness probe failed: cat: can't open '/tmp/healthy': No such file or directory
    Normal   Killing    5s                kubelet            Container basic-container failed liveness probe, will be restarted

From the output, you can see that 5 seconds ago, the liveness probe failed, which prompted the kubectl to restart the container. Now, check the pod's status:

kubectl get pod liveness-demo

Expected output:

NAME            READY   STATUS    RESTARTS     AGE
liveness-demo   1/1     Running   1 (4s ago)   85s

Notice that the RESTARTS field shows 1 which confirms that the container was restarted.

Configuring a Readiness Probe

Configuring the readiness probe is similar to configuring the liveness probe. You can create a similar probe for readiness check by adding the following in the configuration file under the pods.spec.containers field.

    readinessProbe:
      exec:
        command:
        - cat
        - /tmp/healthy
      initialDelaySeconds: 5
      periodSeconds: 10

Conclusion

This article explained Kubernetes' liveness and readiness probes, their use, and how to configure them. To learn more, see Configure Liveness, Readiness and Startup Probes at kubernetes.io.