How to Deploy Jina Serve – Open-Source Neural Search and AI Serving Framework

Jina Serve is an open-source framework for building neural search and multimodal AI applications. It provides a cloud-native architecture for managing AI workloads including high-latency inference, dynamic batching, async streaming, and microservice orchestration. Jina handles the infrastructure layer so you can focus on building AI pipelines rather than managing service communication.

This article explains how to deploy Jina Serve on a Linux server using Docker Compose. It covers directory setup, environment configuration, Traefik reverse proxy integration for automatic HTTPS, and demonstrates the deployment by running a sample service and sending test queries.

Prerequisites

Before you begin, you need to:

Have access to a Linux-based server (with at least 2 CPU cores and 4 GB of RAM) as a non-root user with sudo privileges.
Install Docker and Docker Compose.
Create a DNS A record pointing to your server's IP address (for example, jina.example.com).

Set Up the Directory Structure, Configuration, and Environment Variables

Jina Serve runs as a containerized Flow that automatically wires the gateway and executors together. The project directory stores the Flow configuration, executor code, and environment variables.

Create the project directory structure for Jina Serve.
console
```
$ mkdir -p ~/jina-serve/executor
```
This directory stores the Docker Compose configuration, environment variables, and executor code.
Navigate to the project directory.
console
```
$ cd ~/jina-serve
```

Create a simple text processing executor.

console

$ nano executor/executor.py

Add the following configuration:

                            python
                            
                        
from jina import Executor, requests
from docarray import BaseDoc, DocList


class TextDoc(BaseDoc):
    text: str = ""


class TextProcessor(Executor):
    @requests(on="/index")
    def index(self, docs: DocList[TextDoc], **kwargs) -> DocList[TextDoc]:
        for doc in docs:
            if doc.text:
                doc.text = doc.text.upper()
        return docs

    @requests(on="/search")
    def search(self, docs: DocList[TextDoc], **kwargs) -> DocList[TextDoc]:
        return docs

Save and close the file.

Create the executor requirements file.
console
```
$ nano executor/requirements.txt
```
Add the following dependencies:
text
```
jina==3.34.0
docarray>=0.40.0
```
Save and close the file.
Create the executor configuration file.
console
```
$ nano executor/config.yml
```
Add the following configuration:
yaml
```
jtype: TextProcessor
py_modules:
  - executor.py
```
Save and close the file.
Create the Flow configuration file. This defines the processing pipeline and wires the executor to the gateway automatically.
console
```
$ nano flow.yml
```
Add the following configuration:
yaml
```
jtype: Flow
with:
  protocol: http
  port: 8080

executors:
  - name: textprocessor
    uses: executor/config.yml
```
Save and close the file.

Create the Dockerfile to build the Jina Flow container.

console

$ nano Dockerfile

Add the following configuration:

                            dockerfile
                            
FROM jinaai/jina:3.34.0-py39-standard

WORKDIR /app

COPY executor/requirements.txt /app/executor/requirements.txt
RUN pip install --no-cache-dir -r /app/executor/requirements.txt

COPY executor /app/executor
COPY flow.yml /app/flow.yml

ENTRYPOINT ["jina", "flow", "--uses", "flow.yml"]

Save and close the file.

Create the environment configuration file.
console
```
$ nano .env
```
Add the following configuration:
ini
```
DOMAIN=jina.example.com
LETSENCRYPT_EMAIL=admin@example.com
JINA_LOG_LEVEL=INFO
```
Replace:
- jina.example.com with your registered domain name.
- admin@example.com with your email address for Let's Encrypt notifications.
Save and close the file.

Deploy with Docker Compose

The deployment stack consists of two services: Traefik handles reverse proxy and automatic TLS certificate management via Let's Encrypt, and the Jina Flow runs the gateway and executor in a single container with automatic wiring.

Create the Docker Compose configuration file.

console

$ nano docker-compose.yml

Add the following configuration:

                            yaml
                            
                        
services:
  traefik:
    image: traefik:v3.6
    container_name: traefik
    command:
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--entrypoints.web.http.redirections.entrypoint.to=websecure"
      - "--entrypoints.web.http.redirections.entrypoint.scheme=https"
      - "--certificatesresolvers.letsencrypt.acme.httpchallenge=true"
      - "--certificatesresolvers.letsencrypt.acme.httpchallenge.entrypoint=web"
      - "--certificatesresolvers.letsencrypt.acme.email=${LETSENCRYPT_EMAIL}"
      - "--certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json"
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "./letsencrypt:/letsencrypt"
    restart: unless-stopped

  jina:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: jina-flow
    environment:
      - JINA_LOG_LEVEL=${JINA_LOG_LEVEL}
    expose:
      - "8080"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.jina.rule=Host(`${DOMAIN}`)"
      - "traefik.http.routers.jina.entrypoints=websecure"
      - "traefik.http.routers.jina.tls.certresolver=letsencrypt"
      - "traefik.http.services.jina.loadbalancer.server.port=8080"
    restart: unless-stopped

Save and close the file.

In the above manifest:

services: Launches two containers managed by Docker Compose:
- traefik: Functions as the reverse proxy and TLS termination point.
- jina: Runs the Jina Flow which automatically wires the gateway and TextProcessor executor together.
build (jina): Builds the container from the root Dockerfile which packages the Flow configuration and executor code.
expose (jina): Makes port 8080 available internally for Traefik routing without exposing it to the host network.
volumes: The ./letsencrypt bind mount stores TLS certificates persistently across container restarts.
labels (jina): Instructs Traefik to route HTTPS traffic for the configured domain to the Jina Flow on port 8080.
restart: unless-stopped: Configures containers to restart automatically after crashes or server reboots, unless explicitly stopped.

Build and start the containers in detached mode.
console
```
$ docker compose up -d --build
```
Verify that all containers are running.
console
```
$ docker compose ps
```
The output displays two running containers: the Jina Flow and Traefik with ports 80 and 443 exposed.
View the logs for the services.
console
```
$ docker compose logs
```
For more information on managing a Docker Compose stack, see the How to Use Docker Compose article.

Access and Configure Jina Serve

The Jina Flow exposes HTTP endpoints for health checks and document processing operations. Traefik routes HTTPS traffic to the Flow and provisions TLS certificates from Let's Encrypt.

Run the following command to verify that the gateway is accessible. Replace jina.example.com with your configured domain name.
console
```
$ curl https://jina.example.com/status
```
Output:
```
{"jina":{"jina":"3.34.0","docarray":"0.40.1"
...
```
Open your web browser and navigate to https://jina.example.com/status, replacing jina.example.com with your configured domain name.

The page displays the status JSON, confirming HTTPS routing through Traefik is working correctly.

Test the Deployment

Jina processes documents through the executor pipeline defined in the Flow configuration. Send test queries to verify the deployment is working correctly.

Send a test document to the /index endpoint. Replace jina.example.com with your configured domain name.

                            console
                            
                        
$ curl -X POST https://jina.example.com/index \
  -H "Content-Type: application/json" \
  -d '{"data": [{"text": "hello world"}]}'

Output:

{"data":[{"id":null,"text":"HELLO WORLD"}],"parameters":{},"header":{"request_id":"26d9746d60cf4dcfb48a22cc4060359c","target_executor":""}}

The executor converts the input text to uppercase, confirming the pipeline is functional.

Send a test document to the /search endpoint.

                            console
                            
                        
$ curl -X POST https://jina.example.com/search \
  -H "Content-Type: application/json" \
  -d '{"data": [{"text": "test message"}]}'

Output:

{"data":[{"id":null,"text":"test message"}],"parameters":{},"header":{"request_id":"bbed5b9d577749858df3025494c066bc","target_executor":""}}

The search endpoint returns the document unchanged, demonstrating multiple endpoints on a single executor.

Test with multiple documents in a single request.

                            console
                            
                        
$ curl -X POST https://jina.example.com/index \
  -H "Content-Type: application/json" \
  -d '{"data": [{"text": "first"}, {"text": "second"}, {"text": "third"}]}'

Output:

{"data":[{"id":null,"text":"FIRST"},{"id":null,"text":"SECOND"},{"id":null,"text":"THIRD"}],"parameters":{},"header":{"request_id":"cd2888dd18db407ea9c0a71260593aff","target_executor":""}}

Jina processes all documents in a single batch, demonstrating the framework's batch processing capability.

Conclusion

You have deployed Jina Serve on your server using Docker Compose with Traefik for automatic HTTPS. The Flow configuration automatically wires the gateway and executor together, eliminating manual topology management. Jina supports building more complex pipelines by adding additional executors, enabling GPU acceleration, and integrating with vector databases for neural search applications. For advanced configuration, executor development, and Flow orchestration, visit the official Jina documentation.

Tags:

Open Source

How to Deploy Jina Serve – Open-Source Neural Search and AI Serving Framework

How to Deploy Jina Serve – Open-Source Neural Search and AI Serving Framework

Prerequisites

Set Up the Directory Structure, Configuration, and Environment Variables

Deploy with Docker Compose

Access and Configure Jina Serve

Test the Deployment

Conclusion

Comments

Products

Features

Solutions

Marketplace

Resources

Company

Tech Talks

Vultr Blogs