How to Build a Whisper.cpp Container Image
Introduction
Whisper.cpp is a high-performance and lightweight inference of the OpenAI Whisper automatic speech recognition (ASR) model. It offers plain C/C++ implementations without dependency packages and performs speech recognition with support for both CPU and GPU-based systems.
This article explains how to build a Whisper.cpp container image and publish it to a Vultr Container Registry.
Prerequisites
Before you begin:
Deploy an instance using Vultr's GPU Marketplace App
Access the server using SSH.
Start the Docker service.
CONSOLEAdd the non-root user to the Docker group. For example,
linuxuser.CONSOLESwitch to the user:
CONSOLE
Set Up the Server
Create a new directory to store your Whisper C++ project files.
CONSOLESwitch to the directory.
CONSOLEClone the Whisper.cpp project repository using Git.
CONSOLEList all directory files and verify that a new
whisper.cppdirectory is available.CONSOLEOutput:
Switch to the project directory.
CONSOLEList all hidden files and verify that a
.devopsdirectory is available.CONSOLEOutput:
The
.devopsdirectory includes the following Dockerfile resources:main.Dockerfile: Contains the build context for CPU systems.main-cuda.Dockerfile: Contains the build context for GPU systems.
Use the above resources in the next sections to build a CPU or GPU system container image.
Build a Whisper.cpp Container Image for CPU Systems
Follow the steps below to build a Whisper.cpp container image using the main.Dockerfile that contains all necessary dependencies for CPU-based systems.
Copy the
main.Dockerfilefrom the.devopsdirectory to the mainwhisper.cppproject directory.CONSOLEBuild a new container image using
main.Dockerfilewith all files in the project directory. Replacewhisper-imagewith your desired image name.CONSOLEWait for the build process to complete. When successful, your output should look like the one below.
View all Docker images on the server and verify that the new
whisper-imageis available.CONSOLEOutput:
Build a Whisper.cpp Container Image for GPU Systems
The Whisper.cpp main-cuda.Dockerfile resource contains the build context for GPU systems compatible with the latest NVIDIA CUDA drivers. Follow the steps below to build a Whisper.cpp container image compatible with NVIDIA GPU systems.
Copy
main-cuda.Dockerfileto the main project directory.CONSOLEBuild a new container image
whisper-gpu-imageusing themain-cuda.Dockerfilewith all files in the working project directory.CONSOLEView all Docker images on the server and verify that the new Whisper.cpp GPU image is available.
CONSOLEOutput:
To run the whisper.cpp GPU container image, verify that your target host includes the minimum or higher CUDA version referenced by the
ARG CUDA_VERSION=directive within themain-cuda.Dockerfile. Run the following command to verify the target CUDA version.CONSOLEOutput:
Upload the Whisper.cpp Container Image to the Vultr Container Registry
Open the Vultr Console.
Click Products and select Container Registry on the main navigation menu.
Click your target Vultr Container Registry to open the management panel and view the registry access credentials.
Copy the Registry URL value, Username, and API Key to use when accessing the registry.
Switch to your server terminal session and log in to your Vultr Container Registry. Replace
exampleregistry,exampleuser,registry-passwordwith your actual registry details.CONSOLETag the whisper.cpp container image with your desired Vultr Container Registry tag. For example,
sjc.vultrcr.com/exampleregistry/whisper-gpu-image.CONSOLEView all Docker images on the server and verify that the new tagged image is available.
CONSOLEOutput:
Push the tagged image to your Vultr Container Registry.
CONSOLEWhen successful, your output should be similar to the one below:
Open your Vultr Container Registry management panel and click Repositories on the top navigation bar to verify that the new repository is available.
0 Comments
Be the first to comment and share your perspective with the community.