How to Build a Whisper.cpp Container Image

Introduction

Whisper.cpp is a high-performance and lightweight inference of the OpenAI Whisper automatic speech recognition (ASR) model. It offers plain C/C++ implementations without dependency packages and performs speech recognition with support for both CPU and GPU-based systems.

This article explains how to build a Whisper.cpp container image and publish it to a Vultr Container Registry.

Prerequisites

Before you begin:

• Deploy an instance using Vultr's GPU Marketplace App

• Access the server using SSH.

• Start the Docker service.

  console Copy

  $ sudo systemctl start docker
  

• Add the non-root user to the Docker group. For example, linuxuser.

  console Copy

  $ sudo usermod -aG docker linuxuser
  

• Switch to the user:

  console Copy

  $ su - linuxuser
  

Set Up the Server

1. Create a new directory to store your Whisper C++ project files.

   console Copy

   $ mkdir whisper-project
   

2. Switch to the directory.

   console Copy

   $ cd whisper-project
   

3. Clone the Whisper.cpp project repository using Git.

   console Copy

   $ git clone https://github.com/ggerganov/whisper.cpp
   

4. List all directory files and verify that a new whisper.cpp directory is available.

   console Copy

   $ ls
   

   Output:

   whisper.cpp

5. Switch to the project directory.

   console Copy

   $ cd whisper.cpp
   

6. List all hidden files and verify that a .devops directory is available.

   console Copy

   $ ls -a
   

   Output:

   .  .devops         cmake           examples        ggml-backend.h       ggml-cuda     ggml-impl.h       ggml-metal.m      ggml-quants.c  ggml-vulkan.h    .gitmodules  models  .......    

   The .devops directory includes the following Dockerfile resources:

   • main.Dockerfile: Contains the build context for CPU systems.
   • main-cuda.Dockerfile: Contains the build context for GPU systems.

   Use the above resources in the next sections to build a CPU or GPU system container image.

Build a Whisper.cpp Container Image for CPU Systems

Follow the steps below to build a Whisper.cpp container image using the main.Dockerfile that contains all necessary dependencies for CPU-based systems.

1. Copy the main.Dockerfile from the .devops directory to the main whisper.cpp project directory.

   console Copy

   $ cp .devops/main.Dockerfile .
   

2. Build a new container image using main.Dockerfile with all files in the project directory. Replace whisper-image with your desired image name.

   console Copy

   $ docker build -f main.Dockerfile -t whisper-image .
   

   Wait for the build process to complete. When successful, your output should look like the one below.

   [+] Building 117.7s (12/12) FINISHED                                                                                                                                                                                 docker:default
    => [internal] load build definition from main.Dockerfile                                                                                                                                                                      0.0s
    => => transferring dockerfile: 446B                                                                                                                                                                                           0.0s
    => [internal] load metadata for docker.io/library/ubuntu:22.04                                                                                                                                                                3.1s
    => [internal] load .dockerignore          

3. View all Docker images on the server and verify that the new whisper-image is available.

   console Copy

   $ docker images
   

   Output:

   REPOSITORY      TAG       IMAGE ID       CREATED         SIZE                                                                                                                                                                       
   whisper-image   latest    d54bd7163db5   4 minutes ago   608MB    

Build a Whisper.cpp Container Image for GPU Systems

The Whisper.cpp main-cuda.Dockerfile resource contains the build context for GPU systems compatible with the latest NVIDIA CUDA drivers. Follow the steps below to build a Whisper.cpp container image compatible with NVIDIA GPU systems.

1. Copy main-cuda.Dockerfile to the main project directory.

   console Copy

   $ cp .devops/main-cuda.Dockerfile .
   

2. Build a new container image whisper-gpu-image using the main-cuda.Dockerfile with all files in the working project directory.

   console Copy

   $ docker build -f main-cuda.Dockerfile -t whisper-gpu-image .
   

3. View all Docker images on the server and verify that the new Whisper.cpp GPU image is available.

   console Copy

   $ docker images
   

   Output:

   REPOSITORY          TAG       IMAGE ID       CREATED          SIZE
   whisper-gpu-image   latest    47be5b1566fd   31 seconds ago   2.76GB

   To run the whisper.cpp GPU container image, verify that your target host includes the minimum or higher CUDA version referenced by the ARG CUDA_VERSION= directive within the main-cuda.Dockerfile. Run the following command to verify the target CUDA version.

   console Copy

   $ cat main-cuda.Dockerfile | grep CUDA_VERSION=
   

   Output:

   ARG CUDA_VERSION=12.3.1

Upload the Whisper.cpp Container Image to the Vultr Container Registry

1. Open the Vultr Customer Portal.

2. Click Products and select Container Registry on the main navigation menu.

   

3. Click your target Vultr Container Registry to open the management panel and view the registry access credentials.

4. Copy the Registry URL value, Username, and API Key to use when accessing the registry.

   

5. Switch to your server terminal session and log in to your Vultr Container Registry. Replace exampleregistry, exampleuser, registry-password with your actual registry details.

   console Copy

   $ docker login https://sjc.vultrcr.com/exampleregistry -u exampleuser -p registry-password
   

6. Tag the whisper.cpp container image with your desired Vultr Container Registry tag. For example, sjc.vultrcr.com/exampleregistry/whisper-gpu-image.

   console Copy

   $ docker tag whisper-gpu-image sjc.vultrcr.com/exampleregistry/whisper-gpu-image
   

7. View all Docker images on the server and verify that the new tagged image is available.

   console Copy

   $ docker images
   

   Output:

   REPOSITORY                                  TAG       IMAGE ID       CREATED          SIZE
   whisper-gpu-image                           latest    47be5b1566fd   6 minutes ago    2.76GB
   sjc.vultrcr.com/exampleregistry/whisper-gpu-image   latest    47be5b1566fd   6 minutes ago    2.76GB

8. Push the tagged image to your Vultr Container Registry.

   console Copy

   $ docker push sjc.vultrcr.com/exampleregistry/whisper-gpu-image
   

   When successful, your output should be similar to the one below:

   The push refers to repository [sjc.vultrcr.com/exampleregistry/whisper-gpu-image]
   a42c530a140b: Pushed 
   445c20a8fa7a: Pushed 
   8ceb9643fb36: Pushed 
   latest: digest: sha256:70645eed0095268b6c11b051083cb83c41c1991dc088b0d0485ccc50aa4cf4af size: 2838

9. Open your Vultr Container Registry management panel and click Repositories on the top navigation bar to verify that the new repository is available.