Deploy SUSE AI on a Vultr Bare Metal or Cloud GPU Server

How to Deploy SUSE AI on a Vultr Cloud GPU header image

SUSE AI is an enterprise platform that runs large language models (LLMs), inference engines, and retrieval-augmented generation (RAG) pipelines on Kubernetes. It bundles SUSE Rancher Prime as the management plane, SUSE Storage for persistent volumes, SUSE Security for runtime protection, SUSE Observability for cluster telemetry, and a curated AI library that includes Ollama, vLLM, Open WebUI, and vector databases such as OpenSearch and Milvus.

This guide explains how to deploy the SUSE AI stack on a Vultr GPU server running Ubuntu 24.04, using Terraform and Ansible. A single variable selects whether Terraform provisions a Vultr Bare Metal server or a Vultr Cloud GPU instance. Terraform provisions the GPU server, the SSH keypair, and renders the Ansible inventory file. Ansible installs Rancher Kubernetes Engine 2 (RKE2) Prime as the Kubernetes distribution, deploys Rancher Prime with Let's Encrypt TLS, configures SUSE Storage, applies the NVIDIA GPU Operator, deploys SUSE Observability with a generated values file, and runs Ollama and Open WebUI as the inference backend and chat interface.

Prerequisites

Before you begin, you need to:

Have access to a Vultr API key with permission to provision and manage resources.
Have quota on your Vultr account for a Bare Metal GPU server or a Cloud GPU instance. Vultr Bare Metal GPU plans are not self-serve on every account. Contact Vultr support to request access before provisioning.
Hold a SUSE Customer Center (SCC) account with active subscriptions for SUSE Rancher Prime, SUSE Storage, SUSE Security, SUSE Observability, and SUSE AI.
Generate a SUSE Application Collection access token from apps.rancher.io.
Retrieve the Prime Artifacts URL from the SCC partner documentation portal.
Install Terraform, Ansible, and git on the workstation that runs the deployment.
Open inbound TCP ports 22, 80, 443, 6443, and 9345 on the target server firewall.

Resource Requirements

Pick the server size based on the components you intend to deploy. The following figures reflect real CPU and RAM consumption from a live deployment, not the theoretical chart defaults.

Component	CPU req	RAM req	Storage
Core platform (RKE2, Rancher Prime, SUSE Storage, SUSE Security, NVIDIA GPU Operator, cert-manager)	7.3 cores	12.5 GB	125 GB
AI workloads (Ollama + 1x A100, Open WebUI, OpenSearch)	2.8 cores	15 GB	85 GB
SUSE Observability `trial` profile	7.2 cores	23.4 GB	163 GB
SUSE Observability `10-nonha` profile	7.2 cores	23.4 GB	163 GB
SUSE Observability `100-nonha` profile	24.1 cores	45.7 GB	562 GB

The trial and 10-nonha profiles share identical sizing. Larger non-HA profiles (20-nonha and up) and every HA profile (150-ha and up) require a multi-node cluster and fall outside the single-server deployment this guide builds. See the SUSE Observability sizing reference before choosing one of those profiles.

Pick a plan that meets the minimum for the deployment scope:

Core stack with trial or 10-nonha Observability: 24 vCPUs, 50 GB RAM, 600 GB storage minimum.
Core stack with 100-nonha Observability: 40 vCPUs, 80 GB RAM, 1 TB storage minimum.

Vultr offers several Bare Metal and Cloud GPU plans at different sizes. Choose any plan that meets the minimum above, using the List Bare Metal Plans and List GPU Plans API references to compare specs.

Clone the Repository

The Terraform and Ansible configuration lives in the suse-ai-on-vultr directory of the code-samples repository. Clone it onto the workstation that runs Terraform and Ansible.

Clone the code-samples repository.

                            console
                            
$ git clone https://github.com/vultr-marketing/code-samples.git

Navigate to the suse-ai-on-vultr directory.
console
```
$ cd code-samples/suse-ai-on-vultr
```
Inspect the directory structure.
console
```
$ ls -la
```
The directory contains:
- terraform/: Provisions the GPU server, the SSH keypair, the firewall, and renders the Ansible inventory.
- ansible/: Holds the roles that install each layer of the SUSE stack.
- Makefile: Wraps the Terraform and Ansible commands behind make targets.
- .env.example: Template that documents every variable the pipeline expects.

Configure the Deployment Variables

The pipeline reads every secret and tunable from a single .env file at the repository root. The Makefile sources this file and re-exports the variables under the TF_VAR_ prefix that Terraform expects.

Copy the example file.
console
```
$ cp .env.example .env
```
Open the copied file for editing.
console
```
$ nano .env
```
Populate the deployment variables.
ini
```
VULTR_API_KEY=YOUR_VULTR_API_KEY

APPCO_USERNAME=YOUR_SCC_EMAIL
APPCO_USER_TOKEN=YOUR_APPLICATION_COLLECTION_TOKEN
SCC_REG_CODE=YOUR_RANCHER_PRIME_REG_CODE
SUSE_OBSERVABILITY_REG_CODE=YOUR_OBSERVABILITY_REG_CODE
PRIME_ARTIFACTS_URL=YOUR_PRIME_ARTIFACTS_URL
LE_EMAIL=YOUR_EMAIL_ADDRESS
HF_TOKEN=YOUR_HUGGINGFACE_TOKEN

SERVER_TYPE=bare-metal
TF_VAR_instance_label=suse-ai
TF_VAR_region=atl
TF_VAR_plan=vbm-112c-2048gb-8-a100-gpu
TF_VAR_os_id=2284

DEPLOY_OBSERVABILITY=true
DEPLOY_GPU_OPERATOR=true
DEPLOY_VLLM=false
OBSERVABILITY_PROFILE=100-nonha
```
Save and close the file.

Replace the placeholder values:
- VULTR_API_KEY: The API key generated from the Vultr Console under Account, then API.
- APPCO_USERNAME: The email registered with SCC.
- APPCO_USER_TOKEN: The Application Collection access token from apps.rancher.io.
- SCC_REG_CODE: The SCC registration code for the Rancher Prime subscription.
- SUSE_OBSERVABILITY_REG_CODE: The SCC registration code for the SUSE Observability subscription.
- PRIME_ARTIFACTS_URL: The Prime Artifacts URL retrieved from the SCC partner documentation portal.
- LE_EMAIL: The contact email for the Let's Encrypt account.
- HF_TOKEN: A HuggingFace access token with the license accepted for meta-llama/Llama-3.1-8B-Instruct, the model the Ansible pipeline deploys by default when DEPLOY_VLLM is true. The pipeline forwards this value to the vLLM deployment automatically.
- SERVER_TYPE: The Vultr server type to provision. Set to bare-metal for a Bare Metal server or cloud for a Cloud GPU instance.
- TF_VAR_instance_label: A label for the Vultr resources that Terraform creates.
- TF_VAR_region: The Vultr region code that hosts the GPU plan (for example, atl, ewr, or fra).
- TF_VAR_plan: The Vultr plan ID for the server. Use a vbm-* plan when SERVER_TYPE is bare-metal, or a vcg-* plan when SERVER_TYPE is cloud.
- TF_VAR_os_id: The Vultr operating system ID. 2284 corresponds to Ubuntu 24.04 LTS.
- DEPLOY_OBSERVABILITY: Set to true to deploy SUSE Observability and its agent.
- DEPLOY_GPU_OPERATOR: Set to true to deploy the NVIDIA GPU Operator.
- OBSERVABILITY_PROFILE: The SUSE Observability sizing profile (trial, 10-nonha, 100-nonha, or 150-ha). See Resource Requirements for the minimum vCPUs and RAM each profile needs.
- DEPLOY_VLLM: Set to true to have the Ansible pipeline deploy vLLM automatically (requires a second GPU and a valid HF_TOKEN). Skip the manual vLLM steps in Install Optional AI Library Components when this is true.

Provision the GPU Server

The Terraform configuration generates an ED25519 keypair, uploads the public key to Vultr, provisions the server set by SERVER_TYPE (a Bare Metal server or a Cloud GPU instance), and writes the Ansible inventory file with the discovered public IP.

Initialize Terraform to download the Vultr, TLS, and Local providers.
console
```
$ make init
```
Run a dry-run plan to preview the resources Terraform will create.
console
```
$ make plan
```
The plan output lists the server that SERVER_TYPE selects (Bare Metal or Cloud GPU), the SSH keypair, and the local files that Terraform writes.
Apply the plan to provision the server.
console
```
$ make apply
```
Provisioning a Bare Metal server takes approximately ten to fifteen minutes. A Cloud GPU instance takes a few minutes.

Run the Ansible Pipeline

The Ansible playbook contains roles that mirror the SUSE AI deployment phases. The first play waits for SSH connectivity, so the same command works whether the server finished booting or is still coming online. The roles run idempotently, so re-runs reconcile drift without reinstalling running components.

Run the full playbook against the provisioned server.
console
```
$ make ansible
```
The pipeline installs RKE2, Rancher Prime, SUSE Storage, SUSE Observability, the NVIDIA GPU Operator, and the Ollama and Open WebUI AI workloads. The full run takes approximately twenty to thirty minutes, depending on which components are enabled. The final play prints the access URLs and the auto-generated admin credentials.
Confirm that every pod in the cluster reached the Running or Completed state.
console
```
$ make verify
```
The task fails and lists any pod that is not Running or Completed. No output from that task confirms that every component started successfully.
Print the access URLs and credentials again without rerunning the deployment.
console
```
$ make credentials
```
This command re-runs only the final summary play and reprints the same SUSE AI DEPLOYMENT COMPLETE block that make ansible prints at the end.

Set Up Rancher Prime

Rancher Prime is the management plane for the cluster. It registers the SCC subscription, installs SUSE Security, and monitors every workload the pipeline deployed. The first visit to the Rancher UI presents a welcome screen that sets the admin password and the server URL.

On the workstation, open the URL below in a browser. Replace SERVER-IP with the public IP of the server.
```
https://rancher-prime.SERVER-IP.sslip.io
```
On Welcome to Rancher, enter admin in the Bootstrap Password field.
Select Set a specific password to use, then enter and confirm a strong password in the New Password and Confirm New Password fields.
Confirm that the Server URL field reads https://rancher-prime.SERVER-IP.sslip.io.
Select the checkbox to accept the End User License Agreement & Terms & Conditions.
Click Continue.

Register Rancher Prime with SCC

SUSE Customer Center (SCC) tracks the subscriptions backing this deployment and unlocks support entitlements once Rancher Prime registers against it. The Helm install already staged the registration code in the UI during setup. Activate it now to complete the registration.

In the Rancher UI, click the hamburger menu, then navigate to Global Settings, then Registration.
Confirm that the Online Registration tab shows the pre-filled registration code.
Click Register.

The banner changes to System registration successfully activated, and the Product Registration panel reports the status as Active for the SUSE Rancher product.

Install SUSE Security

SUSE Security (NeuVector) provides runtime protection for the cluster. It scans containers for vulnerabilities, segments network traffic, and detects active threats. Rancher's Apps catalog is the only installation path for SUSE Security. Recent releases auto-detect the container runtime, so the default chart values work on RKE2.

In the Rancher UI, open the Cluster Explorer view for the local cluster.
Navigate to Apps, then Charts.
Search for NeuVector, the chart name for SUSE Security, and select it from the results.
Click Install this version.
On the metadata step, select Default in Install into Project, then click Next.
Click Install to accept the default values. Rancher pins the namespace to cattle-neuvector-system and streams the Helm install logs in the panel at the bottom of the screen.

Access the NeuVector UI

NeuVector serves its UI on its own ingress.

SSH into the server.
console
```
$ make ssh
```
On the server, wait for the NeuVector pods to reach the Running state.
console
```
$ kubectl get pods -n cattle-neuvector-system
```

On the server, retrieve the bootstrap admin password.

                            console
                            
                        
$ kubectl get secret -n cattle-neuvector-system neuvector-bootstrap-secret \
    -o go-template='{{.data.bootstrapPassword|base64decode}}{{"\n"}}'

Copy the printed password for use when logging in below.

Exit the SSH session to return to the workstation.
console
```
$ exit
```
From the workstation, in the code-samples/suse-ai-on-vultr directory, expose the NeuVector UI over HTTPS.
console
```
$ make neuvector-https
```
On the workstation, open the URL below in a browser. Replace SERVER-IP with the public IP of the server.
```
https://neuvector.SERVER-IP.sslip.io
```
Log in with the following credentials:
- Username: admin
- Password: the bootstrap password you copied earlier

Access Open WebUI

Open WebUI provides the chat interface and uses the GPU-accelerated Ollama runtime for inference. The Ansible pipeline pre-deploys the gemma:2b model, which appears in the model selector after the first login.

On the workstation, open the URL below in a browser. Replace SERVER-IP with the public IP of the server.
```
https://webui.SERVER-IP.sslip.io
```
Click Get Started.
Enter your full name, email, and password, then click Create Admin Account. The first account you register becomes the Open WebUI admin.
Select the gemma:2b model from the model dropdown and send a message.

Open WebUI routes the message to the Ollama service over the private Kubernetes cluster network. Ollama runs the gemma:2b model on the NVIDIA GPU and streams the response back to Open WebUI.

Deploy SUSE Observability

SUSE Observability ships the platform UI and a separate agent. The agent collects topology, metrics, and traces from the cluster and forwards them to the platform. Install the Kubernetes StackPack, generate a service token, then deploy the agent.

On the workstation, open the URL below and log in with the username admin and the password printed by the Ansible pipeline. Replace SERVER-IP with the public IP of the server.
```
https://observability.SERVER-IP.sslip.io
```
Navigate to StackPacks, then Kubernetes.
Under Installed Instances, enter a name for your Kubernetes cluster in the Kubernetes Cluster Name field, then click Install. The StackPack reports that it is waiting for data from the agent.
On the waiting-for-data screen, click CREATE NEW SERVICE TOKEN and copy the token.
SSH into the server.
console
```
$ make ssh
```
Export the service token, cluster name, and server IP.
console
```
$ export SERVICE_TOKEN=svctok-YOUR_GENERATED_TOKEN
$ export CLUSTER_NAME=YOUR_CLUSTER_NAME
$ export SERVER_IP=YOUR_SERVER_IP
```
Replace svctok-YOUR_GENERATED_TOKEN with the token you copied from the UI, YOUR_CLUSTER_NAME with the name you entered during StackPack installation, and YOUR_SERVER_IP with the public IP of the server.

Add the SUSE Observability Helm repository.

                            console
                            
                        
$ helm repo add suse-observability \
    https://charts.rancher.com/server-charts/prime/suse-observability

Refresh the local Helm repository cache.
console
```
$ helm repo update
```

Deploy the SUSE Observability agent, cluster agent, and checks agent.

                            console
                            
                        
$ helm upgrade --install \
    --namespace suse-observability \
    --create-namespace \
    --set-string 'stackstate.apiKey'=$SERVICE_TOKEN \
    --set-string 'stackstate.cluster.name'=$CLUSTER_NAME \
    --set-string 'stackstate.url'="https://observability.$SERVER_IP.sslip.io/receiver/stsAgent" \
    suse-observability-agent suse-observability/suse-observability-agent

Patch the Node Agent Health Port

The node agent binds STS_HEALTH_PORT to 5555, which the NVIDIA Data Center GPU Manager (DCGM) host engine (nv-hostengine) already holds on GPU nodes. The conflict crashes the agent with bind: address already in use. Move the agent to port 5556.

Set the new health port on the node agent's environment.

                            console
                            
                        
$ kubectl set env ds/suse-observability-agent-node-agent \
    -n suse-observability -c node-agent STS_HEALTH_PORT=5556

Patch the container and host port definitions to match.

                            console
                            
                        
$ kubectl patch ds -n suse-observability suse-observability-agent-node-agent --type=json -p='[
    {"op":"replace","path":"/spec/template/spec/containers/0/ports/1/containerPort","value":5556},
    {"op":"replace","path":"/spec/template/spec/containers/0/ports/1/hostPort","value":5556}
  ]'

Restart the DaemonSet to apply the change.
console
```
$ kubectl rollout restart ds -n suse-observability suse-observability-agent-node-agent
```
After the agent restarts and sends data, the Installed Instances entry transitions to the Active state, and the cluster topology renders in the UI.

Install Optional AI Library Components

The SUSE Application Collection ships three additional AI library charts beyond Ollama and Open WebUI. vLLM, mcpo, and PyTorch are each optional, install into the suse-ai namespace, and reuse the application-collection image pull secret from the Rancher install. Apply runtimeClassName: nvidia to any GPU-backed workload. Every command in each subsection below runs on the server.

Note

RKE2 does not promote the NVIDIA runtime to the containerd default even when the NVIDIA GPU Operator sets CONTAINERD_SET_AS_DEFAULT. GPU workloads must therefore set runtimeClassName: nvidia explicitly. The Ollama and vLLM roles already include this value. Without it, the pod schedules on the GPU but the NVIDIA devices never mount, and inference silently falls back to the CPU.

vLLM: High-Throughput Inference Engine

vLLM serves a single model with higher throughput than Ollama. It claims a dedicated GPU, so deploy it only when an accelerator is free. The SUSE Application Collection serving image at tag 0.13.0 is ARM64-only, so pull the upstream vllm/vllm-openai image for x86_64 hosts. The chart also deploys a router whose image lives in the private registry, so the router needs the pull secret too.

Note

Setting DEPLOY_VLLM=true in .env already deploys vLLM through the Ansible pipeline. Follow the steps below only when DEPLOY_VLLM was false during make ansible and a manual deployment with a different model is needed.

SSH into the server.
console
```
$ make ssh
```

Create the vLLM values file.

                            console
                            
                        
$ cat > vllm_values.yaml <<'EOF'
servingEngineSpec:
  runtimeClassName: nvidia
  modelSpec:
    - name: "phi3"
      registry: "docker.io"
      repository: "vllm/vllm-openai"
      tag: "v0.6.6"
      modelURL: "MODEL_ID"
      replicaCount: 1
      requestCPU: 6
      requestMemory: "16Gi"
      requestGPU: 1
      storageClass: longhorn
      pvcStorage: "50Gi"
      pvcAccessMode:
        - ReadWriteOnce
      vllmConfig:
        maxModelLen: 4096
        dtype: "bfloat16"
        extraArgs: ["--gpu-memory-utilization", "0.8"]
routerSpec:
  imagePullSecrets:
    - name: application-collection
EOF

Replace MODEL_ID with the HuggingFace model identifier to serve, for example microsoft/Phi-3-mini-4k-instruct for an ungated model, or meta-llama/Llama-3.1-8B-Instruct for a gated model. For a gated model, add a hf_token: "YOUR_HUGGINGFACE_TOKEN" line under the model spec with a token that has accepted the model license.

Install vLLM.

                            console
                            
                        
$ helm upgrade --install vllm \
    oci://dp.apps.rancher.io/charts/vllm \
    -n suse-ai \
    -f vllm_values.yaml \
    --timeout 20m

Wait for both the router and model pods to reach the Running state.

                            console
                            
                        
$ kubectl wait pod -n suse-ai -l app.kubernetes.io/part-of=vllm \
    --for=condition=Ready --timeout=600s

List the models the router serves. Copy the model name from the output for use in the next step.

                            console
                            
                        
$ kubectl run vllm-list --rm --attach --restart=Never \
    -n suse-ai --image=curlimages/curl:latest --command -- \
    curl -s http://vllm-router-service.suse-ai.svc.cluster.local:80/v1/models

Send a completion request and stream the response. Replace MODEL_ID with the model name copied above.

                            console
                            
                        
$ kubectl run vllm-test --rm --attach --restart=Never \
    -n suse-ai --image=curlimages/curl:latest --command -- \
    curl -s -X POST http://vllm-router-service.suse-ai.svc.cluster.local:80/v1/completions \
    -H "Content-Type: application/json" \
    -d '{"model":"MODEL_ID","prompt":"The capital of France is","max_tokens":10}'

The command prints the completion JSON inline and the pod deletes itself on exit.

Connect vLLM to Open WebUI through Admin Panel, then Settings, then Connections, using the URL http://vllm-router-service.suse-ai.svc.cluster.local:80/v1 and the API key dummy.

mcpo: MCP-to-OpenAPI Proxy for Open WebUI Tools

mcpo exposes Model Context Protocol (MCP) servers as OpenAPI tools that Open WebUI can call.

SSH into the server.
console
```
$ make ssh
```

Create the mcpo values file.

                            console
                            
                        
$ cat > mcpo_values.yaml <<'EOF'
global:
  imagePullSecrets:
    - application-collection
config:
  mcpServers:
    memory:
      command: npx
      args:
        - -y
        - "@modelcontextprotocol/server-memory"
EOF

Install mcpo.

                            console
                            
                        
$ helm upgrade --install mcpo \
    oci://dp.apps.rancher.io/charts/open-webui-mcpo \
    -n suse-ai \
    -f mcpo_values.yaml

Confirm the mcpo pod reaches the Running state.
console
```
$ kubectl get pods -n suse-ai | grep mcpo
```

PyTorch: GPU-Enabled Development Environment

PyTorch provides a CUDA-enabled container for training and experimentation.

SSH into the server.
console
```
$ make ssh
```

Create the PyTorch values file.

                            console
                            
                        
$ cat > pytorch_values.yaml <<'EOF'
global:
  imagePullSecrets:
    - application-collection
runtimeClassName: nvidia
image:
  registry: dp.apps.rancher.io
  repository: containers/pytorch
  tag: "2.7.0-nvidia"
  pullPolicy: IfNotPresent
persistence:
  enabled: true
  storageClass: longhorn
gpu:
  enabled: true
  type: 'nvidia'
  number: 1
EOF

Install PyTorch.

                            console
                            
                        
$ helm upgrade --install pytorch \
    oci://dp.apps.rancher.io/charts/pytorch \
    -n suse-ai \
    -f pytorch_values.yaml

Verify the Deployment

Confirm that every layer of the stack reports a healthy state.

SSH into the server.
console
```
$ make ssh
```
List the pods in the AI namespace.
console
```
$ kubectl get pods -n suse-ai
```
The output displays the OpenSearch, Ollama, Open WebUI, and Redis pods in the Running state.
List the pods in the observability namespace.
console
```
$ kubectl get pods -n suse-observability
```
The output displays the platform and agent pods in the Running state.
Confirm that Ollama uses the GPU.
console
```
$ kubectl logs -n suse-ai deploy/ollama | grep -i "inference compute" | tail -1
```
The output reports library=CUDA with the A100 GPU description, confirming GPU-backed inference.

Conclusion

You have successfully deployed the SUSE AI stack on a Vultr Bare Metal or Cloud GPU server running Ubuntu 24.04 using Terraform and Ansible. The cluster runs RKE2 Prime as the Kubernetes distribution, Rancher Prime as the management plane, SUSE Storage for persistent volumes, the NVIDIA GPU Operator for accelerator scheduling, SUSE Observability for monitoring, SUSE Security for runtime protection, and the SUSE AI library components Ollama, Open WebUI, and OpenSearch for retrieval-augmented generation. For more information, visit the official SUSE AI documentation.

Tags:

Vultr Cloud GPU

How to Deploy SUSE AI on a Vultr Cloud GPU

How to Deploy SUSE AI on a Vultr Cloud GPU

Prerequisites

Resource Requirements

Clone the Repository

Configure the Deployment Variables

Provision the GPU Server

Run the Ansible Pipeline

Set Up Rancher Prime

Register Rancher Prime with SCC

Install SUSE Security

Access the NeuVector UI

Access Open WebUI

Deploy SUSE Observability

Patch the Node Agent Health Port

Install Optional AI Library Components

Verify the Deployment

Conclusion

Comments

Products

Features

Solutions

Marketplace

Resources

Company

Tech Talks

Vultr Blogs