How to Deploy Redis® on Vultr Kubernetes Engine (VKE)

Introduction

Redis® is an open-source in-memory data store that can work as a quick-response database. It's capable of handling millions of queries per second with high availability and scalability. A Redis® Cluster is a group of Redis® instances used to achieve high availability, improved fault tolerance, and horizontal scaling in data management systems.

A Redis® cluster uses a full mesh topology where every node interconnects with the main node using a TCP connection. In Kubernetes, pods interconnect to each other making it possible to deploy Redis for high availability within a cluster.

Benefits of deploying Redis® in a Kubernetes cluster include:

• High Availability
• Scalability
• Load Balancing and Service Discovery
• Simplified Deployment and Management
• Monitoring and Logging

This article explains how to deploy a Redis® Cluster on a Vultr Kubernetes Engine (VKE) cluster.

Prerequisites

Before you begin

• Deploy a Vultr Kubernetes Engine (VKE) cluster with at least three nodes
• Install Kubectl on your local machine to access the cluster

Create the Redis® Namespace

By default, Kubernetes adds all cluster components such as services, pods and ConfigMaps to the default namespace. By creating a separate namespace, you are able to manage pods and services more efficiently in the cluster. In this section, create the Redis® namespace as described in the steps below.

1. Create a new namespace for the Redis® cluster.

    $ kubectl create ns redis
   

2. Verify that the new namespace is available.

    $ kubectl get ns

   Output:

    NAME              STATUS   AGE
    default           Active   12m
    kube-node-lease   Active   12m
    kube-public       Active   12m
    kube-system       Active   12m
    redis             Active   7s

Define a Storage Class

A storage class is a Kubernetes resource that allows you to reserve disk space or attach volumes from a cloud provider to your cluster. By default, pods do not store the data permanently. When a pod gets deleted or restarts, data inside the pods is permanently lost.

In this section, mount a Vultr Block Storage instance to your Kubernetes cluster to store data permanently, and avoid any data loss as described below.

1. Using a text editor such as Nano, create a new storage class YAML file.

    $ nano sc.yaml
   

2. Add the following configurations to the file.

    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: redis-storage
    provisioner: block.csi.vultr.com
    volumeBindingMode: WaitForFirstConsumer
    allowVolumeExpansion: true
    reclaimPolicy: Delete

   Save and close the file.

3. Apply the storage class to your cluster.

    $ kubectl apply -f sc.yaml
   

4. Verify that the storage class is available.

    $ kubectl get sc

   Your output should look like the below:

    NAME                             PROVISIONER                    RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
    redis-storage                    block.csi.vultr.com            Delete          WaitForFirstConsumer   true                   6s
    vultr-block-storage (default)    block.csi.vultr.com            Delete          Immediate              true                   12m
    vultr-block-storage-hdd          block.csi.vultr.com            Delete          Immediate              true                   12m
    vultr-block-storage-hdd-retain   block.csi.vultr.com            Retain          Immediate              true                   12m
    vultr-block-storage-retain       block.csi.vultr.com            Retain          Immediate              true                   12m

   As displayed in the above output, a new storage class redis-storage is available and it's attached to a Vultr Block Storage volume.

Create a Persistent Volume

Data durability is essential for Redis® deployments. Persistent volumes store Redis® data on the cluster to achieve data durability in case a pod restarts. A Persistent volume requests a specified amount of storage from the Kubernetes cluster, and it's dynamically provisioned by a StorageClass.

In this section, create a persistent volume as described below.

1. Create a new manifest file to configure three persistent volumes using the vultr-block-storage provisioner.

    $ nano pv.yaml
   

2. Add the following configurations to the file.

    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: redis-pv1
    spec:
      storageClassName: vultr-block-storage
      capacity:
        storage: 10Gi
      accessModes:
        - ReadWriteOnce
      hostPath:
        path: "/storage/data1"
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: redis-pv2
    spec:
      storageClassName: vultr-block-storage
      capacity:
        storage: 10Gi
      accessModes:
        - ReadWriteOnce
      hostPath:
        path: "/storage/data2"
    ---
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: redis-pv3
    spec:
      storageClassName: vultr-block-storage
      capacity:
        storage: 10Gi
      accessModes:
        - ReadWriteOnce
      hostPath:
        path: "/storage/data3"

   Save and close the file.

   The above configuration creates three Persistent Volumes with a size of 10 GB using the vultr-block-storage provisioner. ReadWriteOnce means that the PVC is only mounted as read-write by a single node at a time.

3. Apply the configuration to the cluster

    $ kubectl apply -f pv.yaml
   

4. Verify the new persistent volumes

    $ kubectl get pv

   Your output should look like the one below:

    NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY    STATUS      CLAIM   STORAGECLASS         REASON   AGE
    redis-pv1   10Gi        RWO            Retain           Available           vultr-block-storage            6s
    redis-pv2   10Gi        RWO            Retain           Available           vultr-block-storage            5s
    redis-pv3   10Gi        RWO            Retain           Available           vultr-block-storage            4s

Create the ConfigMap

The ConfigMap is a key-value store in a Kubernetes cluster in which you can define the Redis® configuration information. In this section, create a ConfigMap for the Redis® cluster as described below.

1. Create a new ConfigMap YAML file

    $ nano redis-config.yaml
   

2. Add the following configurations to the file. Replace your_secure_password with a secure password for your cluster

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: redis-config
    data:
      redis.conf: |
        masterauth your_secure_password
        requirepass your_secure_password
        # Define the Redis listening interfaces
        bind 0.0.0.0
        # Enable or disable the protected mode
        protected-mode no
        # Define the Redis listening port
        port 6379
        tcp-backlog 511
        # Close the connection after a client is idle for N seconds (0 to disable)
        timeout 0
   
        tcp-keepalive 300
        daemonize no
        supervised no
        # Define Redis PID file
        pidfile "/var/run/redis_6379.pid"
        loglevel notice
        logfile ""
        databases 16
        always-show-logo yes
        save 900 1
        save 300 10
        save 60 10000
         
        stop-writes-on-bgsave-error yes
        rdbcompression yes
        rdbchecksum yes
        dbfilename "dump.rdb"
        rdb-del-sync-files no
        # Define the database storage location
        dir "/data"
        replica-serve-stale-data yes
        replica-read-only yes
        repl-diskless-sync no
        repl-diskless-sync-delay 5
        repl-diskless-load disabled
        repl-disable-tcp-nodelay no
        replica-priority 100
        acllog-max-len 128
        lazyfree-lazy-eviction no
        lazyfree-lazy-expire no
        lazyfree-lazy-server-del no
        replica-lazy-flush no
   
        lazyfree-lazy-user-del no
        appendonly yes
        appendfilename "appendonly.aof"
        appendfsync everysec
        no-appendfsync-on-rewrite no
        auto-aof-rewrite-percentage 100
        auto-aof-rewrite-min-size 64mb
   
        aof-load-truncated yes
        aof-use-rdb-preamble yes
        lua-time-limit 5000
        slowlog-log-slower-than 10000
        slowlog-max-len 128
        latency-monitor-threshold 0
        notify-keyspace-events ""
        hash-max-ziplist-entries 512
        hash-max-ziplist-value 64
        list-max-ziplist-size -2
        list-compress-depth 0
        set-max-intset-entries 512
        zset-max-ziplist-entries 128
        zset-max-ziplist-value 64
        hll-sparse-max-bytes 3000
        stream-node-max-bytes 4kb
        stream-node-max-entries 100
   
        activerehashing yes
        client-output-buffer-limit normal 0 0 0
        client-output-buffer-limit replica 256mb 64mb 60
        client-output-buffer-limit pubsub 32mb 8mb 60
        hz 10
        dynamic-hz yes
        aof-rewrite-incremental-fsync yes
        rdb-save-incremental-fsync yes
   
        jemalloc-bg-thread yes

   Save and close the file

   In the above configuration, the master and slavepasswords must be the same to establish a connection in the Redis® cluster

3. Apply your configuration to the Kubernetes cluster

    $ kubectl apply -n redis -f redis-config.yaml
   

4. Verify that the ConfigMap is available in the Redis® namespace

    $ kubectl get configmap -n redis

   Output:

    NAME               DATA   AGE 
    kube-root-ca.crt   1      110s
    redis-config       1      4s

   To view the full Redis® code for a ConfigMap, visit the GitHub repository to fork or download the file.

Scale Redis® in the Kubernetes Cluster using a StatefulSet

A StatefulSet deploys stateful applications and clustered applications that save data to persistent storage. It's suitable for deploying Redis® and other applications that require persistent identities and stable hostnames. In this section, create a StatefulSet as below.

1. Create a new StatefulSet YAML file to scale the Redis® cluster

    $ nano redis-statefulset.yaml
   

2. Add the following configurations to the file

    apiVersion: apps/v1
    kind: StatefulSet
    metadata:
      name: redis
    spec:
      serviceName: redis
      # Specify the number of Redis® replicas
      replicas: 3
      selector:
        matchLabels:
          app: redis
      template:
        metadata:
          labels:
            app: redis
        spec:
          initContainers:
          - name: config
            # Specify the Redis® docker image version
            image: redis:6.2.3-alpine
            command: [ "sh", "-c" ]
            args:
              - |
                # Copy the Redis® configuration file to each Redis® pod.
                cp /tmp/redis/redis.conf /etc/redis/redis.conf
        
                echo "finding master..."
                MASTER_FDQN=`hostname  -f | sed -e 's/redis-[0-9]\./redis-0./'`
                if [ "$(redis-cli -h sentinel -p 5000 ping)" != "PONG" ]; then
                  echo "master not found, defaulting to redis-0"
   
                  if [ "$(hostname)" == "redis-0" ]; then
                    echo "this is redis-0, not updating config..."
                  else
                    echo "updating redis.conf..."
                    echo "slaveof $MASTER_FDQN 6379" >> /etc/redis/redis.conf
                  fi
                else
                  echo "sentinel found, finding master"
                  MASTER="$(redis-cli -h sentinel -p 5000 sentinel get-master-addr-by-name mymaster | grep -E '(^redis-\d{1,})|([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})')"
                  echo "master found : $MASTER, updating redis.conf"
                  echo "slaveof $MASTER 6379" >> /etc/redis/redis.conf
                fi
            # Specify the Redis® volume name and mount path
            volumeMounts:
            - name: redis-config
              mountPath: /etc/redis/
            - name: config
              mountPath: /tmp/redis/
          # Specify the Redis® Docker image version, listening port, volume name, and mount path.
           containers:
          - name: redis
            image: redis:6.2.3-alpine
            command: ["redis-server"]
            args: ["/etc/redis/redis.conf"]
            ports:
            - containerPort: 6379
              name: redis
            volumeMounts:
            - name: data
              mountPath: /data
            - name: redis-config
              mountPath: /etc/redis/
          volumes:
          - name: redis-config
            emptyDir: {}
          - name: config
            configMap:
              name: redis-config
      # Specify the Vultr storage class and requested storage space from the Kubernetes cluster.
       volumeClaimTemplates:
      - metadata:
          name: data
        spec:
          accessModes: [ "ReadWriteOnce" ]
          storageClassName: "vultr-block-storage"
          resources:
            requests:
              storage: 500Mi

   Save and close the file

3. Apply the above StatefulSet configuration to deploy the Redis® cluster

    $ kubectl apply -n redis -f redis-statefulset.yaml
   

4. Verify the list of running pods in the cluster

    $ kubectl get pods -n redis

   Output:

    NAME      READY   STATUS    RESTARTS   AGE
    redis-0   1/1     Running   0          31s
    redis-1   1/1     Running   0          25s
    redis-2   1/1     Running   0          20s

   As displayed in the above output, all Redis® cluster pods are available and running.

Create a Headless Service Resource

To access Redis® internally on your cluster, create a headless service object in the Kubernetes cluster to access the application internally as described below.

1. Create a new headless service resource file

    $ nano redis-service.yaml
   

2. Add the following configurations to the file

    apiVersion: v1
    kind: Service
    metadata:
      name: redis
    spec:
      clusterIP: None
      ports:
      - port: 6379
        targetPort: 6379
        name: redis
      selector:
        app: redis

   Save and close the file

3. Apply the service resource to the Kubernetes cluster

    $ kubectl apply -n redis -f redis-service.yaml
   

4. Verify that the Redis® service is running

    $ kubectl get service -n redis

   Output:

    NAME    TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
    redis   ClusterIP   None         <none>        6379/TCP   8s

Verify the Redis® Master Slave Replication

You have deployed the Redis cluster with three pods named redis-0, redis-1, and redis-2. The pod redis-0 acts as a master while other pods work as slaves in the cluster.

1. View the master pod redis-0 logs.

    $ kubectl -n redis logs redis-0

   Output:

    1:C 17 Jul 2023 15:17:12.968 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
    1:C 17 Jul 2023 15:17:12.981 # Redis version=6.2.3, bits=64, commit=00000000, modified=0, pid=1, just started
    1:C 17 Jul 2023 15:17:12.981 # Configuration loaded
    1:M 17 Jul 2023 15:17:12.982 * monotonic clock: POSIX clock_gettime
                    _._                                                  
               _.-``__ ''-._                                             
          _.-``    `.  `_.  ''-._           Redis 6.2.3 (00000000/0) 64 bit
      .-`` .-```.  ```\/    _.,_ ''-._                                  
     (    '      ,       .-`  | `,    )     Running in standalone mode
     |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
     |    `-._   `._    /     _.-'    |     PID: 1
      `-._    `-._  `-./  _.-'    _.-'                                   
     |`-._`-._    `-.__.-'    _.-'_.-'|                                            |    `-._`-._        _.-'_.-'    |           https://redis.io       
      `-._    `-._`-.__.-'_.-'    _.-'                                   
     |`-._`-._    `-.__.-'    _.-'_.-'|                                  
     |    `-._`-._        _.-'_.-'    |                                  
      `-._    `-._`-.__.-'_.-'    _.-'                                   
          `-._    `-.__.-'    _.-'                                       
              `-._        _.-'                                           
                  `-.__.-'                                               
   
    1:M 17 Jul 2023 15:17:12.989 # Server initialized
    1:M 17 Jul 2023 15:17:12.989 * Ready to accept connections
    1:M 17 Jul 2023 15:17:57.464 * Replica 10.244.9.4:6379 asks for synchronization
    1:M 17 Jul 2023 15:17:57.465 * Full resync requested by replica 10.244.9.4:6379
    1:M 17 Jul 2023 15:17:57.465 * Replication backlog created, my new replication IDs are 'b70dca12298759349d656cfe77273767af7384af' and  '0000000000000000000000000000000000000000'
    1:M 17 Jul 2023 15:17:57.465 * Starting BGSAVE for SYNC with target: disk
    12:C 17 Jul 2023 15:17:58.034 * DB saved on disk
    12:C 17 Jul 2023 15:17:58.037 * RDB: 0 MB of memory used by copy-on-write
    1:M 17 Jul 2023 15:17:58.073 * Background saving terminated with success
    1:M 17 Jul 2023 15:17:58.073 * Synchronization with replica 10.244.85.131:6379 succeeded
   

2. To view detailed information about the Redis® master node, use the describe command as below

    $ kubectl -n redis describe pod redis-0

   Output:

          /etc/redis/ from redis-config (rw)
          /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-5czkj (ro)
    Conditions:
      Type              Status
      Initialized       True 
      Ready             True 
      ContainersReady   True 
      PodScheduled      True 
    Volumes:
      data:
        Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
        ClaimName:  data-redis-0
        ReadOnly:   false
      redis-config:
        Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
        Medium:     
        SizeLimit:  <unset>
      config:
        Type:      ConfigMap (a volume populated by a ConfigMap)
        Name:      redis-config
        Optional:  false
      kube-api-access-5czkj:
        Type:                    Projected (a volume that contains injected data from multiple sources)
        TokenExpirationSeconds:  3607
        ConfigMapName:           kube-root-ca.crt
        ConfigMapOptional:       <nil>
        DownwardAPI:             true
    QoS Class:                   BestEffort
    Node-Selectors:              <none>
    Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                                  node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
   

3. Connect to the redis-0 pod to fetch the replication information

    $ kubectl -n redis exec -it redis-0 -- sh
   

4. Log in to the Redis® shell

    # redis-cli 
   

5. Authenticate with the replica using your master password

    > auth your_secure_password
   

6. Verify the available replication information

    > info replication

   Your output should look like the one below:

     # Replication
     role:master
     connected_slaves:2
     slave0:ip=10.244.9.4,port=6379,state=online,offset=112,lag=0
     slave1:ip=10.244.85.131,port=6379,state=online,offset=112,lag=0
     master_failover_state:no-failover
     master_replid:b70dca12298759349d656cfe77273767af7384af
     master_replid2:0000000000000000000000000000000000000000
     master_repl_offset:112
     second_repl_offset:-1
     repl_backlog_active:1
     repl_backlog_size:1048576
     repl_backlog_first_byte_offset:1
     repl_backlog_histlen:112
   

7. Exit the Redis® replica instance

     $ exit

Test Replication Across the Redis® Cluster Nodes

To test the Redis® replication process, write sample data on the master pod and verify that the same data replicates on the slave pods

1. Connect to the master pod redis-0

    $ kubectl -n redis exec -it redis-0 -- sh
   

2. Log in to Redis® using Redis® CLI

    # redis-cli 
   

3. Enter the master password to gain full access to the Redis® cluster

    127.0.0.1:6379> auth your_secure_password
   

4. Add some data to the master node

    > SET test1 june
    > SET test2 july
    > SET test3 august
   

5. Verify the added data

    > KEYS *

   Output:

    1) "test3"
    2) "test2"
    3) "test1"
   

6. Connect to the slave pod redis-1

    $ kubectl -n redis exec -it redis-1 -- sh
   

7. Log in to the Redis® shell

    # redis-cli
   

8. Authenticate using the slave password you created earlier

    > auth your_secure_password
   

9. Verify that data successfully replicates from the master node

    > KEYS *

   Your output should look like the one below.

    1) "test1"
    2) "test2"
    3) "test3"

Conclusion

In this article, you have deployed a Redis® cluster to a Vultr Kubernetes Engine (VKE) cluster. You have added a key-value store on the master node and verified the replicated data on the slave node. For more information about Redis® cluster, visit the official documentation.