How to Deploy Redis on Vultr Kubernetes Engine (VKE)

Updated on October 6, 2023
How to Deploy Redis on Vultr Kubernetes Engine (VKE) header image

Introduction

Redis is an open-source in-memory data store that can work as a quick-response database. It's capable of handling millions of queries per second with high availability and scalability. A Redis Cluster is a group of Redis instances used to achieve high availability, improved fault tolerance, and horizontal scaling in data management systems.

A Redis cluster uses a full mesh topology where every node interconnects with the main node using a TCP connection. In Kubernetes, pods interconnect to each other making it possible to deploy Redis for high availability within a cluster.

Benefits of deploying Redis in a Kubernetes cluster include:

  • High Availability
  • Scalability
  • Load Balancing and Service Discovery
  • Simplified Deployment and Management
  • Monitoring and Logging

This article explains how to deploy a Redis Cluster on a Vultr Kubernetes Engine (VKE) cluster.

Prerequisites

Before you begin

Create the Redis Namespace

By default, Kubernetes adds all cluster components such as services, pods and ConfigMaps to the default namespace. By creating a separate namespace, you are able to manage pods and services more efficiently in the cluster. In this section, create the Redis namespace as described in the steps below.

  1. Create a new namespace for the Redis cluster.

     $ kubectl create ns redis
  2. Verify that the new namespace is available.

     $ kubectl get ns

    Output:

     NAME              STATUS   AGE
     default           Active   12m
     kube-node-lease   Active   12m
     kube-public       Active   12m
     kube-system       Active   12m
     redis             Active   7s

Define a Storage Class

A storage class is a Kubernetes resource that allows you to reserve disk space or attach volumes from a cloud provider to your cluster. By default, pods do not store the data permanently. When a pod gets deleted or restarts, data inside the pods is permanently lost.

In this section, mount a Vultr Block Storage instance to your Kubernetes cluster to store data permanently, and avoid any data loss as described below.

  1. Using a text editor such as Nano, create a new storage class YAML file.

     $ nano sc.yaml
  2. Add the following configurations to the file.

     apiVersion: storage.k8s.io/v1
     kind: StorageClass
     metadata:
       name: redis-storage
     provisioner: block.csi.vultr.com
     volumeBindingMode: WaitForFirstConsumer
     allowVolumeExpansion: true
     reclaimPolicy: Delete

    Save and close the file.

  3. Apply the storage class to your cluster.

     $ kubectl apply -f sc.yaml
  4. Verify that the storage class is available.

     $ kubectl get sc

    Your output should look like the below:

     NAME                             PROVISIONER                    RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
     redis-storage                    block.csi.vultr.com            Delete          WaitForFirstConsumer   true                   6s
     vultr-block-storage (default)    block.csi.vultr.com            Delete          Immediate              true                   12m
     vultr-block-storage-hdd          block.csi.vultr.com            Delete          Immediate              true                   12m
     vultr-block-storage-hdd-retain   block.csi.vultr.com            Retain          Immediate              true                   12m
     vultr-block-storage-retain       block.csi.vultr.com            Retain          Immediate              true                   12m

    As displayed in the above output, a new storage class redis-storage is available and it's attached to a Vultr Block Storage volume.

Create a Persistent Volume

Data durability is essential for Redis deployments. Persistent volumes store Redis data on the cluster to achieve data durability in case a pod restarts. A Persistent volume requests a specified amount of storage from the Kubernetes cluster, and it's dynamically provisioned by a StorageClass.

In this section, create a persistent volume as described below.

  1. Create a new manifest file to configure three persistent volumes using the vultr-block-storage provisioner.

     $ nano pv.yaml
  2. Add the following configurations to the file.

     apiVersion: v1
     kind: PersistentVolume
     metadata:
       name: redis-pv1
     spec:
       storageClassName: vultr-block-storage
       capacity:
         storage: 10Gi
       accessModes:
         - ReadWriteOnce
       hostPath:
         path: "/storage/data1"
     ---
     apiVersion: v1
     kind: PersistentVolume
     metadata:
       name: redis-pv2
     spec:
       storageClassName: vultr-block-storage
       capacity:
         storage: 10Gi
       accessModes:
         - ReadWriteOnce
       hostPath:
         path: "/storage/data2"
     ---
     apiVersion: v1
     kind: PersistentVolume
     metadata:
       name: redis-pv3
     spec:
       storageClassName: vultr-block-storage
       capacity:
         storage: 10Gi
       accessModes:
         - ReadWriteOnce
       hostPath:
         path: "/storage/data3"

    Save and close the file.

    The above configuration creates three Persistent Volumes with a size of 10 GB using the vultr-block-storage provisioner. ReadWriteOnce means that the PVC is only mounted as read-write by a single node at a time.

  3. Apply the configuration to the cluster

     $ kubectl apply -f pv.yaml
  4. Verify the new persistent volumes

     $ kubectl get pv

    Your output should look like the one below:

     NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY    STATUS      CLAIM   STORAGECLASS         REASON   AGE
     redis-pv1   10Gi        RWO            Retain           Available           vultr-block-storage            6s
     redis-pv2   10Gi        RWO            Retain           Available           vultr-block-storage            5s
     redis-pv3   10Gi        RWO            Retain           Available           vultr-block-storage            4s

Create the ConfigMap

The ConfigMap is a key-value store in a Kubernetes cluster in which you can define the Redis configuration information. In this section, create a ConfigMap for the Redis cluster as described below.

  1. Create a new ConfigMap YAML file

     $ nano redis-config.yaml
  2. Add the following configurations to the file. Replace your_secure_password with a secure password for your cluster

     apiVersion: v1
     kind: ConfigMap
     metadata:
       name: redis-config
     data:
       redis.conf: |
         masterauth your_secure_password
         requirepass your_secure_password
         # Define the Redis listening interfaces
         bind 0.0.0.0
         # Enable or disable the protected mode
         protected-mode no
         # Define the Redis listening port
         port 6379
         tcp-backlog 511
         # Close the connection after a client is idle for N seconds (0 to disable)
         timeout 0
    
         tcp-keepalive 300
         daemonize no
         supervised no
         # Define Redis PID file
         pidfile "/var/run/redis_6379.pid"
         loglevel notice
         logfile ""
         databases 16
         always-show-logo yes
         save 900 1
         save 300 10
         save 60 10000
    
         stop-writes-on-bgsave-error yes
         rdbcompression yes
         rdbchecksum yes
         dbfilename "dump.rdb"
         rdb-del-sync-files no
         # Define the database storage location
         dir "/data"
         replica-serve-stale-data yes
         replica-read-only yes
         repl-diskless-sync no
         repl-diskless-sync-delay 5
         repl-diskless-load disabled
         repl-disable-tcp-nodelay no
         replica-priority 100
         acllog-max-len 128
         lazyfree-lazy-eviction no
         lazyfree-lazy-expire no
         lazyfree-lazy-server-del no
         replica-lazy-flush no
    
         lazyfree-lazy-user-del no
         appendonly yes
         appendfilename "appendonly.aof"
         appendfsync everysec
         no-appendfsync-on-rewrite no
         auto-aof-rewrite-percentage 100
         auto-aof-rewrite-min-size 64mb
    
         aof-load-truncated yes
         aof-use-rdb-preamble yes
         lua-time-limit 5000
         slowlog-log-slower-than 10000
         slowlog-max-len 128
         latency-monitor-threshold 0
         notify-keyspace-events ""
         hash-max-ziplist-entries 512
         hash-max-ziplist-value 64
         list-max-ziplist-size -2
         list-compress-depth 0
         set-max-intset-entries 512
         zset-max-ziplist-entries 128
         zset-max-ziplist-value 64
         hll-sparse-max-bytes 3000
         stream-node-max-bytes 4kb
         stream-node-max-entries 100
    
         activerehashing yes
         client-output-buffer-limit normal 0 0 0
         client-output-buffer-limit replica 256mb 64mb 60
         client-output-buffer-limit pubsub 32mb 8mb 60
         hz 10
         dynamic-hz yes
         aof-rewrite-incremental-fsync yes
         rdb-save-incremental-fsync yes
    
         jemalloc-bg-thread yes

    Save and close the file

    In the above configuration, the master and slavepasswords must be the same to establish a connection in the Redis cluster

  3. Apply your configuration to the Kubernetes cluster

     $ kubectl apply -n redis -f redis-config.yaml
  4. Verify that the ConfigMap is available in the Redis namespace

     $ kubectl get configmap -n redis

    Output:

     NAME               DATA   AGE 
     kube-root-ca.crt   1      110s
     redis-config       1      4s

    To view the full Redis code for a ConfigMap, visit the GitHub repository to fork or download the file.

Scale Redis in the Kubernetes Cluster using a StatefulSet

A StatefulSet deploys stateful applications and clustered applications that save data to persistent storage. It's suitable for deploying Redis and other applications that require persistent identities and stable hostnames. In this section, create a StatefulSet as below.

  1. Create a new StatefulSet YAML file to scale the Redis cluster

     $ nano redis-statefulset.yaml
  2. Add the following configurations to the file

     apiVersion: apps/v1
     kind: StatefulSet
     metadata:
       name: redis
     spec:
       serviceName: redis
       # Specify the number of Redis replicas
       replicas: 3
       selector:
         matchLabels:
           app: redis
       template:
         metadata:
           labels:
             app: redis
         spec:
           initContainers:
           - name: config
             # Specify the Redis docker image version
             image: redis:6.2.3-alpine
             command: [ "sh", "-c" ]
             args:
               - |
                 # Copy the Redis configuration file to each Redis pod.
                 cp /tmp/redis/redis.conf /etc/redis/redis.conf
    
                 echo "finding master..."
                 MASTER_FDQN=`hostname  -f | sed -e 's/redis-[0-9]\./redis-0./'`
                 if [ "$(redis-cli -h sentinel -p 5000 ping)" != "PONG" ]; then
                   echo "master not found, defaulting to redis-0"
    
                   if [ "$(hostname)" == "redis-0" ]; then
                     echo "this is redis-0, not updating config..."
                   else
                     echo "updating redis.conf..."
                     echo "slaveof $MASTER_FDQN 6379" >> /etc/redis/redis.conf
                   fi
                 else
                   echo "sentinel found, finding master"
                   MASTER="$(redis-cli -h sentinel -p 5000 sentinel get-master-addr-by-name mymaster | grep -E '(^redis-\d{1,})|([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})')"
                   echo "master found : $MASTER, updating redis.conf"
                   echo "slaveof $MASTER 6379" >> /etc/redis/redis.conf
                 fi
             # Specify the Redis volume name and mount path
             volumeMounts:
             - name: redis-config
               mountPath: /etc/redis/
             - name: config
               mountPath: /tmp/redis/
           # Specify the Redis Docker image version, listening port, volume name, and mount path.
            containers:
           - name: redis
             image: redis:6.2.3-alpine
             command: ["redis-server"]
             args: ["/etc/redis/redis.conf"]
             ports:
             - containerPort: 6379
               name: redis
             volumeMounts:
             - name: data
               mountPath: /data
             - name: redis-config
               mountPath: /etc/redis/
           volumes:
           - name: redis-config
             emptyDir: {}
           - name: config
             configMap:
               name: redis-config
       # Specify the Vultr storage class and requested storage space from the Kubernetes cluster.
        volumeClaimTemplates:
       - metadata:
           name: data
         spec:
           accessModes: [ "ReadWriteOnce" ]
           storageClassName: "vultr-block-storage"
           resources:
             requests:
               storage: 500Mi

    Save and close the file

  3. Apply the above StatefulSet configuration to deploy the Redis cluster

     $ kubectl apply -n redis -f redis-statefulset.yaml
  4. Verify the list of running pods in the cluster

     $ kubectl get pods -n redis

    Output:

     NAME      READY   STATUS    RESTARTS   AGE
     redis-0   1/1     Running   0          31s
     redis-1   1/1     Running   0          25s
     redis-2   1/1     Running   0          20s

    As displayed in the above output, all Redis cluster pods are available and running.

Create a Headless Service Resource

To access Redis internally on your cluster, create a headless service object in the Kubernetes cluster to access the application internally as described below.

  1. Create a new headless service resource file

     $ nano redis-service.yaml
  2. Add the following configurations to the file

     apiVersion: v1
     kind: Service
     metadata:
       name: redis
     spec:
       clusterIP: None
       ports:
       - port: 6379
         targetPort: 6379
         name: redis
       selector:
         app: redis

    Save and close the file

  3. Apply the service resource to the Kubernetes cluster

     $ kubectl apply -n redis -f redis-service.yaml
  4. Verify that the Redis service is running

     $ kubectl get service -n redis

    Output:

     NAME    TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)    AGE
     redis   ClusterIP   None         <none>        6379/TCP   8s

Verify the Redis Master Slave Replication

You have deployed the Redis cluster with three pods named redis-0, redis-1, and redis-2. The pod redis-0 acts as a master while other pods work as slaves in the cluster.

  1. View the master pod redis-0 logs.

     $ kubectl -n redis logs redis-0

    Output:

     1:C 17 Jul 2023 15:17:12.968 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
     1:C 17 Jul 2023 15:17:12.981 # Redis version=6.2.3, bits=64, commit=00000000, modified=0, pid=1, just started
     1:C 17 Jul 2023 15:17:12.981 # Configuration loaded
     1:M 17 Jul 2023 15:17:12.982 * monotonic clock: POSIX clock_gettime
                     _._                                                  
                _.-``__ ''-._                                             
           _.-``    `.  `_.  ''-._           Redis 6.2.3 (00000000/0) 64 bit
       .-`` .-```.  ```\/    _.,_ ''-._                                  
      (    '      ,       .-`  | `,    )     Running in standalone mode
      |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
      |    `-._   `._    /     _.-'    |     PID: 1
       `-._    `-._  `-./  _.-'    _.-'                                   
      |`-._`-._    `-.__.-'    _.-'_.-'|                                            |    `-._`-._        _.-'_.-'    |           https://redis.io       
       `-._    `-._`-.__.-'_.-'    _.-'                                   
      |`-._`-._    `-.__.-'    _.-'_.-'|                                  
      |    `-._`-._        _.-'_.-'    |                                  
       `-._    `-._`-.__.-'_.-'    _.-'                                   
           `-._    `-.__.-'    _.-'                                       
               `-._        _.-'                                           
                   `-.__.-'                                               
    
     1:M 17 Jul 2023 15:17:12.989 # Server initialized
     1:M 17 Jul 2023 15:17:12.989 * Ready to accept connections
     1:M 17 Jul 2023 15:17:57.464 * Replica 10.244.9.4:6379 asks for synchronization
     1:M 17 Jul 2023 15:17:57.465 * Full resync requested by replica 10.244.9.4:6379
     1:M 17 Jul 2023 15:17:57.465 * Replication backlog created, my new replication IDs are 'b70dca12298759349d656cfe77273767af7384af' and  '0000000000000000000000000000000000000000'
     1:M 17 Jul 2023 15:17:57.465 * Starting BGSAVE for SYNC with target: disk
     12:C 17 Jul 2023 15:17:58.034 * DB saved on disk
     12:C 17 Jul 2023 15:17:58.037 * RDB: 0 MB of memory used by copy-on-write
     1:M 17 Jul 2023 15:17:58.073 * Background saving terminated with success
     1:M 17 Jul 2023 15:17:58.073 * Synchronization with replica 10.244.85.131:6379 succeeded
  2. To view detailed information about the Redis master node, use the describe command as below

     $ kubectl -n redis describe pod redis-0

    Output:

           /etc/redis/ from redis-config (rw)
           /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-5czkj (ro)
     Conditions:
       Type              Status
       Initialized       True 
       Ready             True 
       ContainersReady   True 
       PodScheduled      True 
     Volumes:
       data:
         Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
         ClaimName:  data-redis-0
         ReadOnly:   false
       redis-config:
         Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
         Medium:     
         SizeLimit:  <unset>
       config:
         Type:      ConfigMap (a volume populated by a ConfigMap)
         Name:      redis-config
         Optional:  false
       kube-api-access-5czkj:
         Type:                    Projected (a volume that contains injected data from multiple sources)
         TokenExpirationSeconds:  3607
         ConfigMapName:           kube-root-ca.crt
         ConfigMapOptional:       <nil>
         DownwardAPI:             true
     QoS Class:                   BestEffort
     Node-Selectors:              <none>
     Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                                   node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
  3. Connect to the redis-0 pod to fetch the replication information

     $ kubectl -n redis exec -it redis-0 -- sh
  4. Log in to the Redis shell

     # redis-cli 
  5. Authenticate with the replica using your master password

     > auth your_secure_password
  6. Verify the available replication information

     > info replication

    Your output should look like the one below:

      # Replication
      role:master
      connected_slaves:2
      slave0:ip=10.244.9.4,port=6379,state=online,offset=112,lag=0
      slave1:ip=10.244.85.131,port=6379,state=online,offset=112,lag=0
      master_failover_state:no-failover
      master_replid:b70dca12298759349d656cfe77273767af7384af
      master_replid2:0000000000000000000000000000000000000000
      master_repl_offset:112
      second_repl_offset:-1
      repl_backlog_active:1
      repl_backlog_size:1048576
      repl_backlog_first_byte_offset:1
      repl_backlog_histlen:112
  7. Exit the Redis replica instance

      $ exit

Test Replication Across the Redis Cluster Nodes

To test the Redis replication process, write sample data on the master pod and verify that the same data replicates on the slave pods

  1. Connect to the master pod redis-0

     $ kubectl -n redis exec -it redis-0 -- sh
  2. Log in to Redis using Redis CLI

     # redis-cli 
  3. Enter the master password to gain full access to the Redis cluster

     127.0.0.1:6379> auth your_secure_password
  4. Add some data to the master node

     > SET test1 june
     > SET test2 july
     > SET test3 august
  5. Verify the added data

     > KEYS *

    Output:

     1) "test3"
     2) "test2"
     3) "test1"
  6. Connect to the slave pod redis-1

     $ kubectl -n redis exec -it redis-1 -- sh
  7. Log in to the Redis shell

     # redis-cli
  8. Authenticate using the slave password you created earlier

     > auth your_secure_password
  9. Verify that data successfully replicates from the master node

     > KEYS *

    Your output should look like the one below.

     1) "test1"
     2) "test2"
     3) "test3"

Conclusion

In this article, you have deployed a Redis cluster to a Vultr Kubernetes Engine (VKE) cluster. You have added a key-value store on the master node and verified the replicated data on the slave node. For more information about the Redis cluster, visit the official documentation.