How to Deploy Portkey - Open-Source AI Gateway for LLM Routing

Portkey is an open-source AI gateway that unifies access to 250+ Large Language Models (LLMs) across dozens of providers behind a single, OpenAI-compatible API. Instead of integrating with each provider's SDK, applications send requests through Portkey, which handles routing, load balancing, and automatic fallback to keep AI-powered systems reliable.

This article explains how to deploy Portkey on a Linux server using Docker Compose and secure it with Traefik as a reverse proxy for TLS termination. It also covers protecting access with a combination of Basic Authentication for the web console and an IP allowlist for the API endpoint.

Prerequisites

Before you begin, you need to:

• Have access to a Linux-based server (with at least 2 CPU cores and 4 GB of RAM) as a non-root user with sudo privileges.
• Install Docker and Docker Compose.
• Create a DNS A record, such as portkey.example.com, pointing to your server's IP address.
• Have an API key from an LLM provider, used to test the gateway after deployment.

Set Up the Directory Structure and Environment Variables

Portkey's gateway container is stateless and does not require persistent storage of its own. The project directory contains the Docker Compose configuration and environment variables used to configure and run the gateway.

1. Create the project directory.

   CONSOLE

   Copy

   $ mkdir ~/portkey

   Explain Code

   Powered by Vultr AgentBeta

2. Navigate to the project directory.

   CONSOLE

   Copy

   $ cd ~/portkey

   Explain Code

   Powered by Vultr AgentBeta

3. Find your server's public IP address, which is used later to restrict API access.

   CONSOLE

   Copy

   $ curl -4 ifconfig.me

   Explain Code

   Powered by Vultr AgentBeta

   Copy this value for use when configuring the .env file.

4. Create an environment file to store configuration variables.

   CONSOLE

   Copy

   $ nano .env

   Explain Code

   Powered by Vultr AgentBeta

5. Add the following configuration.

   INI

   Copy

   DOMAIN=portkey.example.com
   LETSENCRYPT_EMAIL=admin@example.com
   SERVER_IP=YOUR_SERVER_IP/32

   Explain Code

   Powered by Vultr AgentBeta
   • Replace portkey.example.com with your domain name.
   • Replace admin@example.com with your email address.
   • Replace YOUR_SERVER_IP with your server's public IP address obtained in the earlier step.

   Save and close the file.

6. Install the htpasswd utility, which is used to generate credentials for the web console.

   CONSOLE

   Copy

   $ sudo apt install -y apache2-utils

   Explain Code

   Powered by Vultr AgentBeta

7. Generate a username and password for accessing the Portkey console. Replace portkeyadmin with your preferred username.

   CONSOLE

   Copy

   $ htpasswd -c ~/portkey/.htpasswd portkeyadmin

   Explain Code

   Powered by Vultr AgentBeta

   Enter and confirm a password when prompted. This creates a .htpasswd file containing the hashed credentials.

Deploy Portkey with Docker Compose

This section deploys Portkey and Traefik together, with Traefik handling HTTPS certificate issuance through Let's Encrypt. The self-hosted Portkey gateway has no built-in authentication of its own, so this guide adds Basic Authentication and an IP allowlist at the Traefik level to restrict access to the console and the API.

1. Create the Docker Compose manifest file.

   CONSOLE

   Copy

   $ nano docker-compose.yml

   Explain Code

   Powered by Vultr AgentBeta

2. Add the following configuration.

   YAML

   Copy

   services:
     traefik:
       image: traefik:v3.7.11
       container_name: traefik
       restart: unless-stopped
       environment:
         DOCKER_API_VERSION: "1.44"
       command:
         - "--providers.docker=true"
         - "--providers.docker.exposedbydefault=false"
         - "--entrypoints.web.address=:80"
         - "--entrypoints.websecure.address=:443"
         - "--entrypoints.web.http.redirections.entrypoint.to=websecure"
         - "--entrypoints.web.http.redirections.entrypoint.scheme=https"
         - "--certificatesresolvers.le.acme.httpchallenge=true"
         - "--certificatesresolvers.le.acme.httpchallenge.entrypoint=web"
         - "--certificatesresolvers.le.acme.email=${LETSENCRYPT_EMAIL}"
         - "--certificatesresolvers.le.acme.storage=/letsencrypt/acme.json"
       ports:
         - "80:80"
         - "443:443"
       volumes:
         - /var/run/docker.sock:/var/run/docker.sock:ro
         - ./letsencrypt:/letsencrypt
         - ./.htpasswd:/.htpasswd:ro
   
     portkey:
       image: portkeyai/gateway:1.15.2
       container_name: portkey
       restart: unless-stopped
       labels:
         - "traefik.enable=true"
         - "traefik.http.services.portkey.loadbalancer.server.port=8787"
         - "traefik.http.middlewares.portkey-auth.basicauth.usersfile=/.htpasswd"
         - "traefik.http.middlewares.portkey-ipallow.ipallowlist.sourcerange=${SERVER_IP}"
         - "traefik.http.routers.portkey-console.rule=Host(`${DOMAIN}`) && PathPrefix(`/public`)"
         - "traefik.http.routers.portkey-console.entrypoints=websecure"
         - "traefik.http.routers.portkey-console.tls=true"
         - "traefik.http.routers.portkey-console.tls.certresolver=le"
         - "traefik.http.routers.portkey-console.middlewares=portkey-auth"
         - "traefik.http.routers.portkey-console.service=portkey"
         - "traefik.http.routers.portkey-console.priority=10"
         - "traefik.http.routers.portkey-api.rule=Host(`${DOMAIN}`)"
         - "traefik.http.routers.portkey-api.entrypoints=websecure"
         - "traefik.http.routers.portkey-api.tls=true"
         - "traefik.http.routers.portkey-api.tls.certresolver=le"
         - "traefik.http.routers.portkey-api.middlewares=portkey-ipallow"
         - "traefik.http.routers.portkey-api.service=portkey"
         - "traefik.http.routers.portkey-api.priority=1"
         - "traefik.http.routers.portkey-logs.rule=Host(`${DOMAIN}`) && Path(`/log/stream`)"
         - "traefik.http.routers.portkey-logs.entrypoints=websecure"
         - "traefik.http.routers.portkey-logs.tls=true"
         - "traefik.http.routers.portkey-logs.tls.certresolver=le"
         - "traefik.http.routers.portkey-logs.middlewares=portkey-auth"
         - "traefik.http.routers.portkey-logs.service=portkey"
         - "traefik.http.routers.portkey-logs.priority=10"

   Explain Code

   Powered by Vultr AgentBeta

   Save and close the file.

   In the above manifest:

   • traefik: Serves as the reverse proxy and TLS termination point. It listens on ports 80 and 443, automatically redirects HTTP to HTTPS, and provisions Let's Encrypt certificates.
   • portkey: Runs the official portkeyai/gateway image on internal port 8787. Separate Traefik routers apply Basic Authentication to the /public console and an IP allowlist to the API, avoiding conflicts with provider Authorization headers. The stateless gateway requires no environment variables, volumes, or database.

3. Start the services in detached mode.

   CONSOLE

   Copy

   $ docker compose up -d

   Explain Code

   Powered by Vultr AgentBeta

4. Verify that all containers are running.

   CONSOLE

   Copy

   $ docker compose ps

   Explain Code

   Powered by Vultr AgentBeta

   The output displays two running containers with Traefik listening on ports 80 and 443.

Access the Portkey Gateway Console

Traefik routes all HTTPS traffic on the configured domain to the Portkey container, applying Basic Authentication to the console and an IP allowlist to the API.

1. Confirm that the API responds from your server. Replace portkey.example.com with your configured domain.

   CONSOLE

   Copy

   $ curl -I https://portkey.example.com/v1/chat/completions

   Explain Code

   Powered by Vultr AgentBeta

   A response other than 403 confirms that the request reached the gateway and was not blocked by the IP allowlist. A 403 response means the server's IP does not match the value configured in SERVER_IP in the .env file.

2. Open a web browser and navigate to https://portkey.example.com/public/, replacing portkey.example.com with your configured domain.

3. A browser prompt requests a username and password. Enter the credentials created earlier with htpasswd.

   

   The Portkey Console displays a test request panel and options to configure routing rules such as load balancing, fallbacks, and retries.

Test the Portkey Gateway Console

Validate the deployment by sending a live chat completion request through the gateway using an LLM provider API key.

1. Send a test chat completion request through the gateway. Replace portkey.example.com with your configured domain, your_provider with your LLM provider, and your_api_key and your_model with your provider's key and model name.

   CONSOLE

   Copy

   $ curl https://portkey.example.com/v1/chat/completions -H "x-portkey-provider: your_provider" -H "Authorization: Bearer your_api_key" -H "Content-Type: application/json" -d '{"messages": [{"role": "user", "content": "Say this is a test."}], "model": "your_model"}'

   Explain Code

   Powered by Vultr AgentBeta

   A successful response returns a chat.completion object containing the model's reply, confirming that the gateway routed the request to the provider and back.

2. Open the Portkey Console at https://portkey.example.com/public/, replacing portkey.example.com with your configured domain and review the request under the Logs section, confirming the gateway recorded the completed request.

   

Conclusion

You have deployed Portkey on a Linux server using Docker Compose, with Traefik as a reverse proxy for TLS termination. The deployment routes requests to language model providers through a single, OpenAI-compatible endpoint, with the web console protected by Basic Authentication and the API restricted through an IP allowlist. For more information, see the official Portkey documentation.