How to Deploy Jina Serve – Open-Source Neural Search and AI Serving Framework

5 min readUpdated 11 September 2026

Jina Serve is an open-source framework for building neural search and multimodal AI applications. It provides a cloud-native architecture for managing AI workloads including high-latency inference, dynamic batching, async streaming, and microservice orchestration. Jina handles the infrastructure layer so you can focus on building AI pipelines rather than managing service communication.

This article explains how to deploy Jina Serve on a Linux server using Docker Compose. It covers directory setup, environment configuration, Traefik reverse proxy integration for automatic HTTPS, and demonstrates the deployment by running a sample service and sending test queries.

Prerequisites

Before you begin, you need to:

Have access to a Linux-based server (with at least 2 CPU cores and 4 GB of RAM) as a non-root user with sudo privileges.
Install Docker and Docker Compose.
Create a DNS A record pointing to your server's IP address (for example, jina.example.com).

Set Up the Directory Structure, Configuration, and Environment Variables

Jina Serve runs as a containerized Flow that automatically wires the gateway and executors together. The project directory stores the Flow configuration, executor code, and environment variables.

Create the project directory structure for Jina Serve.
CONSOLE
$ mkdir -p ~/jina-serve/executor
Powered by Vultr AgentBeta
This directory stores the Docker Compose configuration, environment variables, and executor code.
Navigate to the project directory.
CONSOLE
$ cd ~/jina-serve
Powered by Vultr AgentBeta

Create a simple text processing executor.

CONSOLE

$ nano executor/executor.py

Add the following configuration:

PYTHON
from jina import Executor, requests
from docarray import BaseDoc, DocList


class TextDoc(BaseDoc):
    text: str = ""


class TextProcessor(Executor):
    @requests(on="/index")
    def index(self, docs: DocList[TextDoc], **kwargs) -> DocList[TextDoc]:
        for doc in docs:
            if doc.text:
                doc.text = doc.text.upper()
        return docs

    @requests(on="/search")
    def search(self, docs: DocList[TextDoc], **kwargs) -> DocList[TextDoc]:
        return docs
Powered by Vultr AgentBeta

Save and close the file.

Create the executor requirements file.
CONSOLE
$ nano executor/requirements.txt
Powered by Vultr AgentBeta
Add the following dependencies:
TEXT
jina==3.34.0 docarray>=0.40.0
Powered by Vultr AgentBeta
Save and close the file.
Create the executor configuration file.
CONSOLE
$ nano executor/config.yml
Powered by Vultr AgentBeta
Add the following configuration:
YAML
jtype: TextProcessor py_modules: - executor.py
Powered by Vultr AgentBeta
Save and close the file.
Create the Flow configuration file. This defines the processing pipeline and wires the executor to the gateway automatically.
CONSOLE
$ nano flow.yml
Powered by Vultr AgentBeta
Add the following configuration:
YAML
jtype: Flow with: protocol: http port: 8080 executors: - name: textprocessor uses: executor/config.yml
Powered by Vultr AgentBeta
Save and close the file.
Create the Dockerfile to build the Jina Flow container.
CONSOLE
$ nano Dockerfile
Powered by Vultr AgentBeta
Add the following configuration:
DOCKERFILE
FROM jinaai/jina:3.34.0-py39-standard WORKDIR /app COPY executor/requirements.txt /app/executor/requirements.txt RUN pip install --no-cache-dir -r /app/executor/requirements.txt COPY executor /app/executor COPY flow.yml /app/flow.yml ENTRYPOINT ["jina", "flow", "--uses", "flow.yml"]
Powered by Vultr AgentBeta
Save and close the file.
Create the environment configuration file.
CONSOLE
$ nano .env
Powered by Vultr AgentBeta
Add the following configuration:
INI
DOMAIN=jina.example.com LETSENCRYPT_EMAIL=admin@example.com JINA_LOG_LEVEL=INFO
Powered by Vultr AgentBeta
Replace:
- jina.example.com with your registered domain name.
- admin@example.com with your email address for Let's Encrypt notifications.
Save and close the file.

Deploy with Docker Compose

The deployment stack consists of two services: Traefik handles reverse proxy and automatic TLS certificate management via Let's Encrypt, and the Jina Flow runs the gateway and executor in a single container with automatic wiring.

Create the Docker Compose configuration file.

CONSOLE

$ nano docker-compose.yml

Add the following configuration:

YAML
services:
  traefik:
    image: traefik:v3.6
    container_name: traefik
    command:
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--entrypoints.web.http.redirections.entrypoint.to=websecure"
      - "--entrypoints.web.http.redirections.entrypoint.scheme=https"
      - "--certificatesresolvers.letsencrypt.acme.httpchallenge=true"
      - "--certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web"
      - "--certificatesresolvers.letsencrypt.acme.email=${LETSENCRYPT_EMAIL}"
      - "--certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json"
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "./letsencrypt:/letsencrypt"
    restart: unless-stopped

  jina:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: jina-flow
    environment:
      - JINA_LOG_LEVEL=${JINA_LOG_LEVEL}
    expose:
      - "8080"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.jina.rule=Host(`${DOMAIN}`)"
      - "traefik.http.routers.jina.entrypoints=websecure"
      - "traefik.http.routers.jina.tls.certresolver=letsencrypt"
      - "traefik.http.services.jina.loadbalancer.server.port=8080"
    restart: unless-stopped
Powered by Vultr AgentBeta

Save and close the file.

In the above manifest:

services: Launches two containers managed by Docker Compose:
- traefik: Functions as the reverse proxy and TLS termination point.
- jina: Runs the Jina Flow which automatically wires the gateway and TextProcessor executor together.
build (jina): Builds the container from the root Dockerfile which packages the Flow configuration and executor code.
expose (jina): Makes port 8080 available internally for Traefik routing without exposing it to the host network.
volumes: The ./letsencrypt bind mount stores TLS certificates persistently across container restarts.
labels (jina): Instructs Traefik to route HTTPS traffic for the configured domain to the Jina Flow on port 8080.
restart: unless-stopped: Configures containers to restart automatically after crashes or server reboots, unless explicitly stopped.

Build and start the containers in detached mode.
CONSOLE
$ docker compose up -d --build
Powered by Vultr AgentBeta
Verify that all containers are running.
CONSOLE
$ docker compose ps
Powered by Vultr AgentBeta
The output displays two running containers: the Jina Flow and Traefik with ports 80 and 443 exposed.
View the logs for the services.
CONSOLE
$ docker compose logs
Powered by Vultr AgentBeta
For more information on managing a Docker Compose stack, see the How to Use Docker Compose article.

Access and Configure Jina Serve

The Jina Flow exposes HTTP endpoints for health checks and document processing operations. Traefik routes HTTPS traffic to the Flow and provisions TLS certificates from Let's Encrypt.

Run the following command to verify that the gateway is accessible. Replace jina.example.com with your configured domain name.
CONSOLE
$ curl https://jina.example.com/status
Powered by Vultr AgentBeta
Output:
{"jina":{"jina":"3.34.0","docarray":"0.40.1" ...
Open your web browser and navigate to https://jina.example.com/status, replacing jina.example.com with your configured domain name.
The page displays the status JSON, confirming HTTPS routing through Traefik is working correctly.

Test the Deployment

Jina processes documents through the executor pipeline defined in the Flow configuration. Send test queries to verify the deployment is working correctly.

Send a test document to the /index endpoint. Replace jina.example.com with your configured domain name.
CONSOLE
$ curl -X POST https://jina.example.com/index \ -H "Content-Type: application/json" \ -d '{"data": [{"text": "hello world"}]}'
Powered by Vultr AgentBeta
Output:
{"data":[{"id":null,"text":"HELLO WORLD"}],"parameters":{},"header":{"request_id":"26d9746d60cf4dcfb48a22cc4060359c","target_executor":""}}
The executor converts the input text to uppercase, confirming the pipeline is functional.
Send a test document to the /search endpoint.
CONSOLE
$ curl -X POST https://jina.example.com/search \ -H "Content-Type: application/json" \ -d '{"data": [{"text": "test message"}]}'
Powered by Vultr AgentBeta
Output:
{"data":[{"id":null,"text":"test message"}],"parameters":{},"header":{"request_id":"bbed5b9d577749858df3025494c066bc","target_executor":""}}
The search endpoint returns the document unchanged, demonstrating multiple endpoints on a single executor.
Test with multiple documents in a single request.
CONSOLE
$ curl -X POST https://jina.example.com/index \ -H "Content-Type: application/json" \ -d '{"data": [{"text": "first"}, {"text": "second"}, {"text": "third"}]}'
Powered by Vultr AgentBeta
Output:
{"data":[{"id":null,"text":"FIRST"},{"id":null,"text":"SECOND"},{"id":null,"text":"THIRD"}],"parameters":{},"header":{"request_id":"cd2888dd18db407ea9c0a71260593aff","target_executor":""}}
Jina processes all documents in a single batch, demonstrating the framework's batch processing capability.

Conclusion

You have deployed Jina Serve on your server using Docker Compose with Traefik for automatic HTTPS. The Flow configuration automatically wires the gateway and executor together, eliminating manual topology management. Jina supports building more complex pipelines by adding additional executors, enabling GPU acceleration, and integrating with vector databases for neural search applications. For advanced configuration, executor development, and Flow orchestration, visit the official Jina documentation.

Tags:Open Source Jina Serve Inference AI & Machine Learning

Set Up the Directory Structure, Configuration, and Environment Variables

Jina Serve runs as a containerized Flow that automatically wires the gateway and executors together. The project directory stores the Flow configuration, executor code, and environment variables.

Create the project directory structure for Jina Serve.

CONSOLE

$ mkdir -p ~/jina-serve/executor

This directory stores the Docker Compose configuration, environment variables, and executor code.

Navigate to the project directory.

CONSOLE

$ cd ~/jina-serve

Create a simple text processing executor.

CONSOLE

$ nano executor/executor.py

Add the following configuration:

PYTHON
from jina import Executor, requests
from docarray import BaseDoc, DocList


class TextDoc(BaseDoc):
    text: str = ""


class TextProcessor(Executor):
    @requests(on="/index")
    def index(self, docs: DocList[TextDoc], **kwargs) -> DocList[TextDoc]:
        for doc in docs:
            if doc.text:
                doc.text = doc.text.upper()
        return docs

    @requests(on="/search")
    def search(self, docs: DocList[TextDoc], **kwargs) -> DocList[TextDoc]:
        return docs
Powered by Vultr AgentBeta

Save and close the file.

Create the executor requirements file.

CONSOLE

$ nano executor/requirements.txt

Add the following dependencies:

TEXT

jina==3.34.0
docarray>=0.40.0

Save and close the file.

Create the executor configuration file.

CONSOLE

$ nano executor/config.yml

Add the following configuration:

YAML
jtype: TextProcessor
py_modules:
  - executor.py
Powered by Vultr AgentBeta

Save and close the file.

Create the Flow configuration file. This defines the processing pipeline and wires the executor to the gateway automatically.

CONSOLE

$ nano flow.yml

Add the following configuration:

YAML
jtype: Flow
with:
  protocol: http
  port: 8080

executors:
  - name: textprocessor
    uses: executor/config.yml
Powered by Vultr AgentBeta

Save and close the file.

Create the Dockerfile to build the Jina Flow container.

CONSOLE

$ nano Dockerfile

Add the following configuration:

DOCKERFILE
FROM jinaai/jina:3.34.0-py39-standard

WORKDIR /app

COPY executor/requirements.txt /app/executor/requirements.txt
RUN pip install --no-cache-dir -r /app/executor/requirements.txt

COPY executor /app/executor
COPY flow.yml /app/flow.yml

ENTRYPOINT ["jina", "flow", "--uses", "flow.yml"]
Powered by Vultr AgentBeta

Save and close the file.

Create the environment configuration file.

CONSOLE

$ nano .env

Add the following configuration:

INI
DOMAIN=jina.example.com
LETSENCRYPT_EMAIL=admin@example.com
JINA_LOG_LEVEL=INFO
Powered by Vultr AgentBeta

Replace:

jina.example.com with your registered domain name.
admin@example.com with your email address for Let's Encrypt notifications.

Save and close the file.

Deploy with Docker Compose

Create the Docker Compose configuration file.

CONSOLE

$ nano docker-compose.yml

Add the following configuration:

YAML
services:
  traefik:
    image: traefik:v3.6
    container_name: traefik
    command:
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--entrypoints.web.http.redirections.entrypoint.to=websecure"
      - "--entrypoints.web.http.redirections.entrypoint.scheme=https"
      - "--certificatesresolvers.letsencrypt.acme.httpchallenge=true"
      - "--certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web"
      - "--certificatesresolvers.letsencrypt.acme.email=${LETSENCRYPT_EMAIL}"
      - "--certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json"
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "./letsencrypt:/letsencrypt"
    restart: unless-stopped

  jina:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: jina-flow
    environment:
      - JINA_LOG_LEVEL=${JINA_LOG_LEVEL}
    expose:
      - "8080"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.jina.rule=Host(`${DOMAIN}`)"
      - "traefik.http.routers.jina.entrypoints=websecure"
      - "traefik.http.routers.jina.tls.certresolver=letsencrypt"
      - "traefik.http.services.jina.loadbalancer.server.port=8080"
    restart: unless-stopped
Powered by Vultr AgentBeta

Save and close the file.

In the above manifest:

services: Launches two containers managed by Docker Compose:
- traefik: Functions as the reverse proxy and TLS termination point.
- jina: Runs the Jina Flow which automatically wires the gateway and TextProcessor executor together.
build (jina): Builds the container from the root Dockerfile which packages the Flow configuration and executor code.
expose (jina): Makes port 8080 available internally for Traefik routing without exposing it to the host network.
volumes: The ./letsencrypt bind mount stores TLS certificates persistently across container restarts.
labels (jina): Instructs Traefik to route HTTPS traffic for the configured domain to the Jina Flow on port 8080.
restart: unless-stopped: Configures containers to restart automatically after crashes or server reboots, unless explicitly stopped.

Build and start the containers in detached mode.

CONSOLE

$ docker compose up -d --build

Verify that all containers are running.

CONSOLE

$ docker compose ps

The output displays two running containers: the Jina Flow and Traefik with ports 80 and 443 exposed.

View the logs for the services.

CONSOLE

$ docker compose logs

For more information on managing a Docker Compose stack, see the How to Use Docker Compose article.

Access and Configure Jina Serve

The Jina Flow exposes HTTP endpoints for health checks and document processing operations. Traefik routes HTTPS traffic to the Flow and provisions TLS certificates from Let's Encrypt.

Run the following command to verify that the gateway is accessible. Replace jina.example.com with your configured domain name.

CONSOLE

$ curl https://jina.example.com/status

Output:

{"jina":{"jina":"3.34.0","docarray":"0.40.1"
...

Open your web browser and navigate to https://jina.example.com/status, replacing jina.example.com with your configured domain name.

The page displays the status JSON, confirming HTTPS routing through Traefik is working correctly.

Test the Deployment

Jina processes documents through the executor pipeline defined in the Flow configuration. Send test queries to verify the deployment is working correctly.

Send a test document to the /index endpoint. Replace jina.example.com with your configured domain name.

CONSOLE
$ curl -X POST https://jina.example.com/index \
  -H "Content-Type: application/json" \
  -d '{"data": [{"text": "hello world"}]}'
Powered by Vultr AgentBeta

Output:

{"data":[{"id":null,"text":"HELLO WORLD"}],"parameters":{},"header":{"request_id":"26d9746d60cf4dcfb48a22cc4060359c","target_executor":""}}

The executor converts the input text to uppercase, confirming the pipeline is functional.

Send a test document to the /search endpoint.

CONSOLE
$ curl -X POST https://jina.example.com/search \
  -H "Content-Type: application/json" \
  -d '{"data": [{"text": "test message"}]}'
Powered by Vultr AgentBeta

Output:

{"data":[{"id":null,"text":"test message"}],"parameters":{},"header":{"request_id":"bbed5b9d577749858df3025494c066bc","target_executor":""}}

The search endpoint returns the document unchanged, demonstrating multiple endpoints on a single executor.

Test with multiple documents in a single request.

CONSOLE
$ curl -X POST https://jina.example.com/index \
  -H "Content-Type: application/json" \
  -d '{"data": [{"text": "first"}, {"text": "second"}, {"text": "third"}]}'
Powered by Vultr AgentBeta

Output:

{"data":[{"id":null,"text":"FIRST"},{"id":null,"text":"SECOND"},{"id":null,"text":"THIRD"}],"parameters":{},"header":{"request_id":"cd2888dd18db407ea9c0a71260593aff","target_executor":""}}

Jina processes all documents in a single batch, demonstrating the framework's batch processing capability.

Conclusion

How to Deploy Jina Serve – Open-Source Neural Search and AI Serving Framework

Prerequisites

Set Up the Directory Structure, Configuration, and Environment Variables

Deploy with Docker Compose

Access and Configure Jina Serve

Test the Deployment

Conclusion

About the Author

Stella Oiro

More to Explore

Tech Talk: Running a Self-Improving AI Agent with Hermes on Vultr

Tech Talk: Implementing Fine-Grained Access Control with Vultr IAM

Tech Talk: Getting Started with Vultr IAM

Tech Talk: Simplifying Zero-Trust Networking with NetBird on Vultr Marketplace

0 Comments

How to Deploy Jina Serve – Open-Source Neural Search and AI Serving Framework

Prerequisites

Set Up the Directory Structure, Configuration, and Environment Variables

Deploy with Docker Compose

Access and Configure Jina Serve

Test the Deployment

Conclusion

About the Author

Stella Oiro

More to Explore

Tech Talk: Running a Self-Improving AI Agent with Hermes on Vultr

Tech Talk: Implementing Fine-Grained Access Control with Vultr IAM

Tech Talk: Getting Started with Vultr IAM

Tech Talk: Simplifying Zero-Trust Networking with NetBird on Vultr Marketplace

0 Comments