How to Use Lua Scripting with Vultr Managed Database for Caching

Lua Scripting Vultr Managed Database Caching

Published on July 31, 2023•Updated on June 20, 2024

How to Use Lua Scripting with Vultr Managed Database for Caching header image

Introduction

Redis® is an open-source, in-memory data structure store. In addition to its core data types, Redis® provides the ability to execute server-side programs similar to stored procedures in relational databases. However, in Redis®, scripts are executed by an embedded engine, which only supports the Lua interpreter. Lua is a scripting language that supports multiple programming paradigms, including procedural, functional, and object orientation. As part of a Lua script, you can write logic that uses Redis® commands to read and write data.

Benefits of Lua scripting with Redis® include:

Efficiency: Lua scripting allows you to execute logic closer to the data. By executing the logic in a Redis® server, you can reduce latency and save network bandwidth in return.
Atomicity: A Lua script is executed atomically in Redis® because the entire server is blocked during the duration of script execution. Although this is a powerful feature, it requires you to be careful about the correctness and execution time of Lua scripts.
Flexibility: Lua is a full-fledged programming language, with it, you can implement server-side features by combining Redis® commands. This allows you to build capabilities specific to a set of requirements that may not be natively supported by Redis®.

This article explains how to implement Lua scripting using a Vultr Managed Database for Caching.

Prerequisites

Before you begin:

Deploy a Vultr Managed Database for Caching.
- When deployed and ready, copy the database connection string to connect to the Redis® database. For example:
```
    rediss://default:[DATABASE_PASSWORD]@[DATABASE_HOST]:[DATABASE_PORT]
```
Deploy a Ubuntu 22.04 management server.
Using SSH, access the server as a non-root sudo user and install:
- The latest stable GO programming language.
```
  $ sudo snap install go --classic
```
- The Redis® CLI tool.
```
  $ sudo apt-get install redis
```

Lua Scripting Commands

Redis® supports Lua scripting usage via the EVAL, EVALSHA, SCRIPT LOAD, SCRIPT EXISTS, SCRIPT FLUSH, and SCRIPT KILL commands. In this section, try the commands on your database using the Redis CLI tool.

To get started, use redis-cli to access your Vultr Managed Database for Caching using the connection string as below.

$ redis-cli -u rediss://default:[DATABASE_PASSWORD]@[DATABASE_HOST]:[DATABASE_PORT]

Replace DATABASE_PASSWORD, DATABASE_HOST, and DATABASE_PORT with your actual Redis® database values.

EVAL

EVAL runs a Lua script on the server. For example, to increment the count of orders for a user, write a script in Lua and run it using EVAL as below.

> EVAL "return redis.call('incr', KEYS[1])" 1 user:123:orders

Output:

(integer) 1

The command increments the value stored in user:123:orders where 1 is the number of keys that the Lua script takes.

SCRIPT LOAD

SCRIPT LOAD adds a script into Redis® memory and returns its SHA1 hash for use with EVALSHA. For example:

> SCRIPT LOAD "return redis.call('incr', KEYS[1])"

Your output should look like the one below:

2bab3b661081db58bd2341920e0ba7cf5dc77b25

As displayed in the output, the command loads the script and returns a SHA1 hash. Later, you will use the hash with EVALSHA to run the script.

EVALSHA

To run a cached script on the server using its SHA1 hash generated by the SCRIPT LOAD command, use the syntax below.

> EVALSHA 2bab3b661081db58bd2341920e0ba7cf5dc77b25 1 user:123:orders

Output:

(integer) 2

The command runs the previously loaded script using its SHA1 hash.

SCRIPT EXISTS

SCRIPT EXISTS checks if a script exists in the Redis® cache as below:

> SCRIPT EXISTS 2bab3b661081db58bd2341920e0ba7cf5dc77b25

Output:

1) (integer) 1

The command returns 1 if the script exists in the cache and 0 if otherwise.

SCRIPT FLUSH

SCRIPT FLUSH removes all scripts from the Redis® script cache.

> SCRIPT FLUSH

Output:

OK

SCRIPT KILL

SCRIPT KILL kills the Lua script in execution, assuming no write operation is performed by the script.

> SCRIPT KILL

Depending on the script state, your output may look like the one below:

(error) NOTBUSY No scripts in execution right now.

The command stops the Lua script from execution if no write operations are performed yet.

Initialize the Project

In this section, execute Lua scripts in a Go application with the help of the go-redis client as described in the following steps.

Exit the Redis® shell.
```
 > EXIT
```
Create a directory to store your Lua scripts. For example redis-lua-scripting.
```
 $ mkdir redis-lua-scripting
```
Switch to the directory.
```
 $ cd redis-lua-scripting
```
Create a new Go module.
```
 $ go mod init redis-lua-scripting
```
The above command creates a new go.mod file
Create a new file main.go.
```
 $ touch main.go
```

Use `Eval` and `EvalRO` Functions

The Eval function makes it possible to invoke a server-side Lua script. It has a variant, EvalRO that prohibits commands which mutate data from executing in Redis®. When a script is executed, it's cached in Redis®, and can be uniquely identified using its SHA1 hash. The EvalSha function loads a script from the Redis® cache its SHA1 hash and executes it as described in this section.

Using a text editor such as Nano, edit the main.go file.
```
 $ nano main.go
```

Add the following code to the file.

 package main

 import (
    "context"
    "crypto/tls"
    "fmt"
    "log"
    "os"

    "github.com/go-redis/redis/v9"
 )

 var client *redis.Client

 const luaScript = `
    local key = KEYS[1]
    local value = ARGV[1] + ARGV[2]
    redis.call("SET", key, value)
    return value
    `

 func init() {
    redisURL := os.Getenv("REDIS_URL")
    if redisURL == "" {
        log.Fatal("missing environment variable REDIS_URL")
    }

    opt, err := redis.ParseURL(redisURL)
    if err != nil {
        log.Fatal("invalid redis url", err)
    }

    opt.TLSConfig = &tls.Config{}

    client = redis.NewClient(opt)

    _, err = client.Ping(context.Background()).Result()

    if err != nil {
        log.Fatal("ping failed. could not connect", err)
    }
 }

 func main() {
    script := redis.NewScript(luaScript)

    result, err := script.Eval(context.Background(), client, []string{"sum"}, "10", "20").Int()
    if err != nil {
        fmt.Println("lua scrip eval failed", err)
    }
    fmt.Println("lua script eval sum result", result)

    result, err = script.EvalSha(context.Background(), client, []string{"sum"}, "10", "20").Int()
    if err != nil {
        fmt.Println("lua scrip eval failed", err)
    }
    fmt.Println("lua script eval sha sum result", result)

    _, err = script.EvalRO(context.Background(), client, []string{"sum"}, "40", "21").Int()
    if err != nil {
        fmt.Println("lua scrip eval read_only failed", err)
    }
 }

Save and close the file.

To run the program, fetch the Go module dependencies.
```
 $ go get
```
To use your Redis® connection in the file, add your Vultr Managed Database for Caching connection string to the REDIS_URL variable.
```
 $ export REDIS_URL=rediss://default:[DATABASE_PASSWORD]@[DATABASE_HOST]:[DATABASE_PORT]
```
Replace DATABASE_PASSWORD, DATABASE_HOST, and DATABASE_PORT with your actual database values.

Run the program.

 $ go run main.go

Your output should look like the one below:

 lua script eval sum result 30
 lua script eval sha sum result 30
 lua scrip eval read_only failed ERR Write commands are not allowed from read-only scripts. script: abf78cadfdce42a3df3df54bf8545340bccd5289, on @user_script:4.

In this section, you imported the required packages and defined the Lua script. Then, the init function reads the Vultr Managed Database for Caching URL from the REDIS_URL environment variable and fails if it's missing. The program creates a new redis.Client instance and verifies connectivity to Vultr Managed Database for Caching by using the Ping utility. If connectivity fails, the program exists with an error message.

In the Go file, a Lua script is used to calculate the sum of two numbers and store it in another key. The Eval function executes the script, when successful, the result is printed. The EvalSha function also works as expected, and the EvalRO function fails with an error.

How to Use `Run` and `RunRO` Functions

The Run function provides a convenient way of executing Lua scripts because it optimistically uses EvalSha and retries the execution using Eval if the script does not exist. Similar to the EvalRO function, RunRO uses EvalSha_RO to run the script and retries using Eval_RO when required. In this section, use the Run or RunRO functions to run Lua scripts as described below.

Back up the original main.go file.
```
 $ mv main.go main.ORIG
```
Create a new main.go file.
```
 $ nano main.go
```

Add the following contents to the file.

 package main

 import (
    "context"
    "crypto/tls"
    "fmt"
    "log"
    "os"

    "github.com/go-redis/redis/v9"
 )

 var client *redis.Client

 const luaScript = `
    local key = KEYS[1]
    local value = ARGV[1] + ARGV[2]
    redis.call("SET", key, value)
    return value
    `

 func init() {
    redisURL := os.Getenv("REDIS_URL")
    if redisURL == "" {
        log.Fatal("missing environment variable REDIS_URL")
    }

    opt, err := redis.ParseURL(redisURL)
    if err != nil {
        log.Fatal("invalid redis url", err)
    }

    opt.TLSConfig = &tls.Config{}

    client = redis.NewClient(opt)

    _, err = client.Ping(context.Background()).Result()

    if err != nil {
        log.Fatal("ping failed. could not connect", err)
    }
 }

 func main() {
    script := redis.NewScript(luaScript)

    result, err := script.Run(context.Background(), client, []string{"sum_run"}, "10", "20").Int()
    if err != nil {
        fmt.Println("lua script run failed", err)
    }
    fmt.Println("lua script run sum result", result)

    _, err = script.RunRO(context.Background(), client, []string{"sum_run"}, "40", "21").Int()
    if err != nil {
        fmt.Println("lua script run read_only failed", err)
    }
 }

Save and close the file.

Run the program.

 $ go run main.go

Your output should look like the one below:

 lua script run sum result 30
 lua scrip run read_only failed ERR Write commands are not allowed from read-only scripts. script: abf78cadfdce42a3df3df54bf8545340bccd5289, on @user_script:4.

In this section, you imported required packages to the file and used the Run function to execute the Lua script, when successful, the result is printed, and as expected, the RunRO function fails with an error.

Implement Increment Functionality with a Lua Script

Redis® has an in-built INCRBY command that increments a counter atomically. In this section, implement the functionality using a Lua script and examine its atomicity characteristics.

Back up the main.go file.
```
 $ mv main.go main.ORIG.1
```
Create a new main.go file.
```
 $ nano main.go
```

Add the following contents to the file.

 package main

 import (
   "context"
   "crypto/tls"
   "fmt"
   "log"
   "os"
   "sync"

   "github.com/go-redis/redis/v9"
 )

 var client *redis.Client

 const counterName = "test_counter"

 func init() {
   redisURL := os.Getenv("REDIS_URL")
   if redisURL == "" {
       log.Fatal("missing environment variable REDIS_URL")
   }

   opt, err := redis.ParseURL(redisURL)
   if err != nil {
       log.Fatal("invalid redis url", err)
   }

   opt.TLSConfig = &tls.Config{}

   client = redis.NewClient(opt)

   _, err = client.Ping(context.Background()).Result()

   if err != nil {
       log.Fatal("ping failed. could not connect", err)
   }
 }

 func main() {
   var wg sync.WaitGroup
   wg.Add(5)

   for i := 1; i <= 5; i++ {
       go func() {
           LuaIncrBy(client, counterName, 2)
           wg.Done()
       }()
   }

   fmt.Println("waiting for operations to finish")
   wg.Wait()
   fmt.Println("all operations finished")

   result := client.Get(context.Background(), counterName).Val()
   fmt.Println("final result", result)
 }

 func LuaIncrBy(c *redis.Client, key string, counter int) int {
   incrByScript := redis.NewScript(`
               local key = KEYS[1]
               local counter = ARGV[1]

               local value = redis.call("GET", key)
               if not value then
               value = 0
               end

               value = value + counter
               redis.call("SET", key, value)

               return value
           `)

   k := []string{key}

   val, err := incrByScript.Run(context.Background(), c, k, counter).Int()

   if err != nil {
       log.Fatal("lua script execution failed", err)
   }

   return val
 }

Save and close the file.

Elements of the Lua script include:

key and counter are local variables retrieved from the script key and argument, respectively.
If the key is not found, its value is set to 0.
The value is incremented by the provided counter and the updated value is SET.
The main function:
- A sync.WaitGroup object is defined and initialized to 5.
- goroutine spawns five instances, each of which invokes the LuaIncrBy operation concurrently. This verifies that the Lua script execution is indeed atomic.
- Waits for all the instances of goroutine to finish.
- When all five operations are complete, the program prints the final value (result of increment) and exits.

Run the program.

 $ go run main.go

Your output should look like the one below:

 waiting for operations to finish
 all operations finished
 final result 10

In this section, you imported necessary packages, and used the LuaIncrBy function which provides the ability to increment a given key with the specified counter. Within the file, a Lua script is defined, and it’s executed using the Run function by passing in the key and counter value. When successfully executed, the result is returned, or the program exits with an error.

Conclusion

In this article, you have implemented Lua scripts to execute server-side logic using a Vultr Managed Database for Caching. You ran Lua scripting commands with the Redis® CLI and several programs that use Lua scripts in Go. For more information about Lua Scripting, visit the Redis® Lua documentation.

How to Use Lua Scripting with Vultr Managed Database for Caching

Introduction

Prerequisites

Lua Scripting Commands

EVAL

SCRIPT LOAD

EVALSHA

SCRIPT EXISTS

SCRIPT FLUSH

SCRIPT KILL

Initialize the Project

Use `Eval` and `EvalRO` Functions

How to Use `Run` and `RunRO` Functions

Implement Increment Functionality with a Lua Script

Conclusion

Products

Features

Marketplace

Resources

Company

How to Use Lua Scripting with Vultr Managed Database for Caching

Introduction

Prerequisites

Lua Scripting Commands

EVAL

SCRIPT LOAD

EVALSHA

SCRIPT EXISTS

SCRIPT FLUSH

SCRIPT KILL

Initialize the Project

Use Eval and EvalRO Functions

How to Use Run and RunRO Functions

Implement Increment Functionality with a Lua Script

Conclusion

Use `Eval` and `EvalRO` Functions

How to Use `Run` and `RunRO` Functions