How to Deploy SUSE AI on a Vultr Cloud GPU

SUSE AI is an enterprise platform that runs large language models (LLMs), inference engines, and retrieval-augmented generation (RAG) pipelines on Kubernetes. It bundles SUSE Rancher Prime as the management plane, SUSE Storage for persistent volumes, SUSE Security for runtime protection, SUSE Observability for cluster telemetry, and a curated AI library that includes Ollama, vLLM, Open WebUI, and vector databases such as OpenSearch and Milvus.

This guide explains how to deploy the SUSE AI stack on a Vultr GPU server running Ubuntu 24.04, using Terraform and Ansible. A single variable selects whether Terraform provisions a Vultr Bare Metal server or a Vultr Cloud GPU instance. Terraform provisions the GPU server, the SSH keypair, and renders the Ansible inventory file. Ansible installs Rancher Kubernetes Engine 2 (RKE2) Prime as the Kubernetes distribution, deploys Rancher Prime with Let's Encrypt TLS, configures SUSE Storage, applies the NVIDIA GPU Operator, deploys SUSE Observability with a generated values file, and runs Ollama and Open WebUI as the inference backend and chat interface.

Prerequisites

Before you begin, you need to:

• Have access to a Vultr API key with permission to provision and manage resources.
• Have quota on your Vultr account for a Bare Metal GPU server or a Cloud GPU instance. Vultr Bare Metal GPU plans are not self-serve on every account. Contact Vultr support to request access before provisioning.
• Hold a SUSE Customer Center (SCC) account with active subscriptions for SUSE Rancher Prime, SUSE Storage, SUSE Security, SUSE Observability, and SUSE AI.
• Generate a SUSE Application Collection access token from apps.rancher.io.
• Retrieve the Prime Artifacts URL from the SCC partner documentation portal.
• Install Terraform, Ansible, and git on the workstation that runs the deployment.
• Open inbound TCP ports 22, 80, 443, 6443, and 9345 on the target server firewall.

Resource Requirements

Pick the server size based on the components you intend to deploy. The table lists the approximate CPU, memory, and storage each layer requires. The SUSE Observability rows follow the published profile requirements.

The trial and 10-nonha profiles request the same CPU and memory but differ in storage (154 GB versus 335 GB). Every non-HA profile, including 20-nonha, 50-nonha, and 100-nonha, runs on a single node. The HA profiles (150-ha and up) require a minimum of three nodes and fall outside the single-server deployment this guide builds. See the SUSE Observability sizing reference for the full profile table.

Pick a plan that meets the minimum for the deployment scope:

• Core stack with trial or 10-nonha Observability: 24 vCPUs, 50 GB RAM, 600 GB storage minimum.
• Core stack with 100-nonha Observability: 40 vCPUs, 80 GB RAM, 1 TB storage minimum.

Vultr offers several Bare Metal and Cloud GPU plans at different sizes. Choose any plan that meets the minimum above, using the List Bare Metal Plans and List GPU Plans API references to compare specs.

Clone the Repository

The Terraform and Ansible configuration lives in the suse-ai-on-vultr directory of the code-samples repository. Clone it onto the workstation that runs Terraform and Ansible.

1. Clone the code-samples repository.

   CONSOLE

   Copy

   $ git clone https://github.com/vultr-marketing/code-samples.git

   Explain Code

   Powered by Vultr AgentBeta

2. Navigate to the suse-ai-on-vultr directory.

   CONSOLE

   Copy

   $ cd code-samples/suse-ai-on-vultr

   Explain Code

   Powered by Vultr AgentBeta

3. Inspect the directory structure.

   CONSOLE

   Copy

   $ ls -la

   Explain Code

   Powered by Vultr AgentBeta

   The directory contains:

   • terraform/: Provisions the GPU server, the SSH keypair, the firewall, and renders the Ansible inventory.
   • ansible/: Holds the roles that install each layer of the SUSE stack.
   • Makefile: Wraps the Terraform and Ansible commands behind make targets.
   • .env.example: Template that documents every variable the pipeline expects.

Configure the Deployment Variables

The pipeline reads every secret and tunable from a single .env file at the repository root. The Makefile sources this file and re-exports the variables under the TF_VAR_ prefix that Terraform expects.

1. Copy the example file.

   CONSOLE

   Copy

   $ cp .env.example .env

   Explain Code

   Powered by Vultr AgentBeta

2. Open the copied file for editing.

   CONSOLE

   Copy

   $ nano .env

   Explain Code

   Powered by Vultr AgentBeta

3. Populate the deployment variables.

   INI

   Copy

   VULTR_API_KEY=YOUR_VULTR_API_KEY
   
   APPCO_USERNAME=YOUR_SCC_EMAIL
   APPCO_USER_TOKEN=YOUR_APPLICATION_COLLECTION_TOKEN
   SCC_REG_CODE=YOUR_RANCHER_PRIME_REG_CODE
   SUSE_OBSERVABILITY_REG_CODE=YOUR_OBSERVABILITY_REG_CODE
   PRIME_ARTIFACTS_URL=YOUR_PRIME_ARTIFACTS_URL
   LE_EMAIL=YOUR_EMAIL_ADDRESS
   HF_TOKEN=YOUR_HUGGINGFACE_TOKEN
   
   SERVER_TYPE=bare-metal
   TF_VAR_instance_label=suse-ai
   TF_VAR_region=atl
   TF_VAR_plan=vbm-112c-2048gb-8-a100-gpu
   TF_VAR_os_id=2284
   
   DEPLOY_OBSERVABILITY=true
   DEPLOY_GPU_OPERATOR=true
   DEPLOY_VLLM=false
   OBSERVABILITY_PROFILE=100-nonha

   Explain Code

   Powered by Vultr AgentBeta

   Save and close the file.

   Replace the placeholder values:

   • VULTR_API_KEY: The API key generated from the Vultr Console under Account, then API.
   • APPCO_USERNAME: The email registered with SCC.
   • APPCO_USER_TOKEN: The Application Collection access token from apps.rancher.io.
   • SCC_REG_CODE: The SCC registration code for the Rancher Prime subscription.
   • SUSE_OBSERVABILITY_REG_CODE: The SCC registration code for the SUSE Observability subscription.
   • PRIME_ARTIFACTS_URL: The Prime Artifacts URL retrieved from the SCC partner documentation portal.
   • LE_EMAIL: The contact email for the Let's Encrypt account.
   • HF_TOKEN: A HuggingFace access token with the license accepted for meta-llama/Llama-3.1-8B-Instruct, the model the Ansible pipeline deploys by default when DEPLOY_VLLM is true. The pipeline forwards this value to the vLLM deployment automatically.
   • SERVER_TYPE: The Vultr server type to provision. Set to bare-metal for a Bare Metal server or cloud for a Cloud GPU instance.
   • TF_VAR_instance_label: A label for the Vultr resources that Terraform creates.
   • TF_VAR_region: The Vultr region code that hosts the GPU plan (for example, atl, ewr, or fra).
   • TF_VAR_plan: The Vultr plan ID for the server. Use a vbm-* plan when SERVER_TYPE is bare-metal, or a vcg-* plan when SERVER_TYPE is cloud.
   • TF_VAR_os_id: The Vultr operating system ID. 2284 corresponds to Ubuntu 24.04 LTS.
   • DEPLOY_OBSERVABILITY: Set to true to deploy SUSE Observability and its agent.
   • DEPLOY_GPU_OPERATOR: Set to true to deploy the NVIDIA GPU Operator.
   • OBSERVABILITY_PROFILE: The SUSE Observability sizing profile (trial, 10-nonha, 100-nonha, or 150-ha). See Resource Requirements for the minimum vCPUs and RAM each profile needs.
   • DEPLOY_VLLM: Set to true to have the Ansible pipeline deploy vLLM automatically (requires a second GPU and a valid HF_TOKEN). Skip the manual vLLM steps in Install Optional AI Library Components when this is true.

Provision the GPU Server

The Terraform configuration generates an ED25519 keypair, uploads the public key to Vultr, provisions the server set by SERVER_TYPE (a Bare Metal server or a Cloud GPU instance), and writes the Ansible inventory file with the discovered public IP.

1. Initialize Terraform to download the Vultr, TLS, and Local providers.

   CONSOLE

   Copy

   $ make init

   Explain Code

   Powered by Vultr AgentBeta

2. Run a dry-run plan to preview the resources Terraform will create.

   CONSOLE

   Copy

   $ make plan

   Explain Code

   Powered by Vultr AgentBeta

   The plan output lists the server that SERVER_TYPE selects (Bare Metal or Cloud GPU), the SSH keypair, and the local files that Terraform writes.

3. Apply the plan to provision the server.

   CONSOLE

   Copy

   $ make apply

   Explain Code

   Powered by Vultr AgentBeta

   Provisioning a Bare Metal server takes approximately ten to fifteen minutes. A Cloud GPU instance takes a few minutes.

Run the Ansible Pipeline

The Ansible playbook contains roles that mirror the SUSE AI deployment phases. The first play waits for SSH connectivity, so the same command works whether the server finished booting or is still coming online. The roles run idempotently, so re-runs reconcile drift without reinstalling running components.

1. Run the full playbook against the provisioned server.

   CONSOLE

   Copy

   $ make ansible

   Explain Code

   Powered by Vultr AgentBeta

   The pipeline installs RKE2, Rancher Prime, SUSE Storage, SUSE Observability, the NVIDIA GPU Operator, and the Ollama and Open WebUI AI workloads. The full run takes approximately twenty to thirty minutes, depending on which components are enabled. The final play prints the access URLs and the auto-generated admin credentials.

2. Confirm that every pod in the cluster reached the Running or Completed state.

   CONSOLE

   Copy

   $ make verify

   Explain Code

   Powered by Vultr AgentBeta

   The task fails and lists any pod that is not Running or Completed. No output from that task confirms that every component started successfully.

3. Print the access URLs and credentials again without rerunning the deployment.

   CONSOLE

   Copy

   $ make credentials

   Explain Code

   Powered by Vultr AgentBeta

   This command re-runs only the final summary play and reprints the same SUSE AI DEPLOYMENT COMPLETE block that make ansible prints at the end.

Set Up Rancher Prime

Rancher Prime is the management plane for the cluster. It registers the SCC subscription, installs SUSE Security, and monitors every workload the pipeline deployed. The first visit to the Rancher UI presents a welcome screen that sets the admin password and the server URL.

1. On the workstation, open the URL below in a browser. Replace SERVER-IP with the public IP of the server.

   https://rancher-prime.SERVER-IP.sslip.io

2. On Welcome to Rancher, enter admin in the Bootstrap Password field.

3. Select Set a specific password to use, then enter and confirm a strong password in the New Password and Confirm New Password fields.

4. Confirm that the Server URL field reads https://rancher-prime.SERVER-IP.sslip.io.

5. Select the checkbox to accept the End User License Agreement & Terms & Conditions.

6. Click Continue.

Register Rancher Prime with SCC

SUSE Customer Center (SCC) tracks the subscriptions backing this deployment and unlocks support entitlements once Rancher Prime registers against it. The Helm install already staged the registration code in the UI during setup. Activate it now to complete the registration.

1. In the Rancher UI, click the hamburger menu, then navigate to Global Settings, then Registration.

2. Confirm that the Online Registration tab shows the pre-filled registration code.

3. Click Register.

   The banner changes to System registration successfully activated, and the Product Registration panel reports the status as Active for the SUSE Rancher product.

Install SUSE Security

SUSE Security (NeuVector) provides runtime protection for the cluster. It scans containers for vulnerabilities, segments network traffic, and detects active threats. Rancher installs it from the Apps catalog or through Cluster Tools. The NeuVector chart does not auto-detect the container runtime, so RKE2 and K3s clusters must set the container runtime to k3scontainerd in the chart values. See the SUSE Rancher Manager NeuVector overview for the runtime values per cluster type. The steps below use the Apps catalog.

1. In the Rancher UI, open the Cluster Explorer view for the local cluster.
2. Navigate to Apps, then Charts.
3. Search for NeuVector, the chart name for SUSE Security, and select it from the results.
4. Click Install this version.
5. On the metadata step, select Default in Install into Project, then click Next.
6. In the chart configuration, open the Container Runtime section and select k3scontainerd, the runtime RKE2 and K3s require.
7. Click Install. Rancher pins the namespace to cattle-neuvector-system and streams the Helm install logs in the panel at the bottom of the screen.

Access the NeuVector UI

NeuVector serves its UI on its own ingress.

1. SSH into the server.

   CONSOLE

   Copy

   $ make ssh

   Explain Code

   Powered by Vultr AgentBeta

2. On the server, wait for the NeuVector pods to reach the Running state.

   CONSOLE

   Copy

   $ kubectl get pods -n cattle-neuvector-system

   Explain Code

   Powered by Vultr AgentBeta

3. On the server, retrieve the bootstrap admin password.

   CONSOLE

   Copy

   $ kubectl get secret -n cattle-neuvector-system neuvector-bootstrap-secret \
       -o go-template='{{.data.bootstrapPassword|base64decode}}{{"\n"}}'

   Explain Code

   Powered by Vultr AgentBeta

   Copy the printed password for use when logging in below.

4. Exit the SSH session to return to the workstation.

   CONSOLE

   Copy

   $ exit

   Explain Code

   Powered by Vultr AgentBeta

5. From the workstation, in the code-samples/suse-ai-on-vultr directory, expose the NeuVector UI over HTTPS.

   CONSOLE

   Copy

   $ make neuvector-https

   Explain Code

   Powered by Vultr AgentBeta

6. On the workstation, open the URL below in a browser. Replace SERVER-IP with the public IP of the server.

   https://neuvector.SERVER-IP.sslip.io

7. Log in with the following credentials:

   • Username: admin
   • Password: the bootstrap password you copied earlier

Access Open WebUI

Open WebUI provides the chat interface and uses the GPU-accelerated Ollama runtime for inference. The Ansible pipeline pre-deploys the gemma:2b model, which appears in the model selector after the first login.

1. On the workstation, open the URL below in a browser. Replace SERVER-IP with the public IP of the server.

   https://webui.SERVER-IP.sslip.io

2. Click Get Started.

3. Enter your full name, email, and password, then click Create Admin Account. The first account you register becomes the Open WebUI admin.

4. Select the gemma:2b model from the model dropdown and send a message.

   Open WebUI routes the message to the Ollama service over the private Kubernetes cluster network. Ollama runs the gemma:2b model on the NVIDIA GPU and streams the response back to Open WebUI.

Deploy SUSE Observability

SUSE Observability ships the platform UI and a separate agent. The agent collects topology, metrics, and traces from the cluster and forwards them to the platform. Install the Kubernetes StackPack, generate a service token, then deploy the agent.

1. On the workstation, open the URL below and log in with the username admin and the password printed by the Ansible pipeline. Replace SERVER-IP with the public IP of the server.

   https://observability.SERVER-IP.sslip.io

2. Navigate to StackPacks, then Kubernetes.

3. Under Installed Instances, enter a name for your Kubernetes cluster in the Kubernetes Cluster Name field, then click Install. The StackPack reports that it is waiting for data from the agent.

4. On the waiting-for-data screen, click CREATE NEW SERVICE TOKEN and copy the token.

5. SSH into the server.

   CONSOLE

   Copy

   $ make ssh

   Explain Code

   Powered by Vultr AgentBeta

6. Export the service token, cluster name, and server IP.

   CONSOLE

   Copy

   $ export SERVICE_TOKEN=svctok-YOUR_GENERATED_TOKEN
   $ export CLUSTER_NAME=YOUR_CLUSTER_NAME
   $ export SERVER_IP=YOUR_SERVER_IP

   Explain Code

   Powered by Vultr AgentBeta

   Replace svctok-YOUR_GENERATED_TOKEN with the token you copied from the UI, YOUR_CLUSTER_NAME with the name you entered during StackPack installation, and YOUR_SERVER_IP with the public IP of the server.

7. Add the SUSE Observability Helm repository.

   CONSOLE

   Copy

   $ helm repo add suse-observability \
       https://charts.rancher.com/server-charts/prime/suse-observability

   Explain Code

   Powered by Vultr AgentBeta

8. Refresh the local Helm repository cache.

   CONSOLE

   Copy

   $ helm repo update

   Explain Code

   Powered by Vultr AgentBeta

9. Deploy the SUSE Observability agent, cluster agent, and checks agent.

   CONSOLE

   Copy

   $ helm upgrade --install \
       --namespace suse-observability \
       --create-namespace \
       --set-string 'stackstate.apiKey'=$SERVICE_TOKEN \
       --set-string 'stackstate.cluster.name'=$CLUSTER_NAME \
       --set-string 'stackstate.url'="https://observability.$SERVER_IP.sslip.io/receiver/stsAgent" \
       suse-observability-agent suse-observability/suse-observability-agent

   Explain Code

   Powered by Vultr AgentBeta

Patch the Node Agent Health Port

The node agent binds STS_HEALTH_PORT to 5555, which the NVIDIA Data Center GPU Manager (DCGM) host engine (nv-hostengine) already holds on GPU nodes. The conflict crashes the agent with bind: address already in use. Move the agent to port 5556.

1. Set the new health port on the node agent's environment.

   CONSOLE

   Copy

   $ kubectl set env ds/suse-observability-agent-node-agent \
       -n suse-observability -c node-agent STS_HEALTH_PORT=5556

   Explain Code

   Powered by Vultr AgentBeta

2. Patch the container port definition to match.

   CONSOLE

   Copy

   $ kubectl patch ds -n suse-observability suse-observability-agent-node-agent --type=json -p='[
       {"op":"replace","path":"/spec/template/spec/containers/0/ports/1/containerPort","value":5556}
     ]'

   Explain Code

   Powered by Vultr AgentBeta

3. Restart the DaemonSet to apply the change.

   CONSOLE

   Copy

   $ kubectl rollout restart ds -n suse-observability suse-observability-agent-node-agent

   Explain Code

   Powered by Vultr AgentBeta

   After the agent restarts and sends data, the Installed Instances entry transitions to the Active state, and the cluster topology renders in the UI.

Install Optional AI Library Components

The SUSE Application Collection ships three additional AI library charts beyond Ollama and Open WebUI. vLLM, mcpo, and PyTorch are each optional, install into the suse-ai namespace, and reuse the application-collection image pull secret from the Rancher install. Apply runtimeClassName: nvidia to any GPU-backed workload. Every command in each subsection below runs on the server.

Verify the Deployment

Confirm that every layer of the stack reports a healthy state.

1. SSH into the server.

   CONSOLE

   Copy

   $ make ssh

   Explain Code

   Powered by Vultr AgentBeta

2. List the pods in the AI namespace.

   CONSOLE

   Copy

   $ kubectl get pods -n suse-ai

   Explain Code

   Powered by Vultr AgentBeta

   The output displays the OpenSearch, Ollama, Open WebUI, and Redis pods in the Running state.

3. List the pods in the observability namespace.

   CONSOLE

   Copy

   $ kubectl get pods -n suse-observability

   Explain Code

   Powered by Vultr AgentBeta

   The output displays the platform and agent pods in the Running state.

4. Confirm that Ollama uses the GPU.

   CONSOLE

   Copy

   $ kubectl logs -n suse-ai deploy/ollama | grep -i "inference compute" | tail -1

   Explain Code

   Powered by Vultr AgentBeta

   The output reports library=CUDA with the A100 GPU description, confirming GPU-backed inference.

Conclusion

You have successfully deployed the SUSE AI stack on a Vultr Bare Metal or Cloud GPU server running Ubuntu 24.04 using Terraform and Ansible. The cluster runs RKE2 Prime as the Kubernetes distribution, Rancher Prime as the management plane, SUSE Storage for persistent volumes, the NVIDIA GPU Operator for accelerator scheduling, SUSE Observability for monitoring, SUSE Security for runtime protection, and the SUSE AI library components Ollama, Open WebUI, and OpenSearch for retrieval-augmented generation. For more information, visit the official SUSE AI documentation.