How to Use Retrieval-Augmented Generation (RAG) with Vultr Serverless Inference

Retrieval-Augmented Generation (RAG) combines information retrieval with text generation. In Vultr Serverless Inference, RAG lets chat models use domain-specific knowledge from your vector store collections, which improves the contextual accuracy of responses and reduces hallucinations.

The RAG endpoint also supports tool calling. Models can invoke functions that you define during RAG-based interactions, such as performing calculations, fetching live data, or calling APIs based on the retrieved context.

Follow this guide to use RAG-based chat completion on Vultr Serverless Inference using the Vultr API.

Generate RAG-Based Chat Completions

Retrieve the subscription's API key and export it as the INFERENCE_API_KEY environment variable. See How to Retrieve the API Key for Vultr Serverless Inference.

1. Send a GET request to the List Collections endpoint and note the target collection's ID.

   CONSOLE

   Copy

   $ curl "https://api.vultrinference.com/v1/vector_store" \
       -X GET \
       -H "Authorization: Bearer ${INFERENCE_API_KEY}"

   Explain Code

   Powered by Vultr AgentBeta

2. Send a GET request to the List Models endpoint and note the preferred inference model's ID.

   CONSOLE

   Copy

   $ curl "https://api.vultrinference.com/v1/models" \
       -X GET \
       -H "Authorization: Bearer ${INFERENCE_API_KEY}"

   Explain Code

   Powered by Vultr AgentBeta

3. Send a POST request to the RAG Chat Completion endpoint to generate responses using Retrieval-Augmented Generation (RAG). Replace COLLECTION_ID with the collection ID, MODEL_ID with the model ID, and USER_INPUT with your message.

   CONSOLE

   Copy

   $ curl "https://api.vultrinference.com/v1/chat/completions/RAG" \
       -X POST \
       -H "Authorization: Bearer ${INFERENCE_API_KEY}" \
       -H "Content-Type: application/json" \
       -d '{
           "collection": "COLLECTION_ID",
           "model": "MODEL_ID",
           "messages": [
               {
                   "role": "user",
                   "content": "USER_INPUT"
               }
           ],
           "max_tokens": 512
       }'

   Explain Code

   Powered by Vultr AgentBeta

   Visit the RAG Chat Completion API page to view additional attributes you can apply for greater control when interacting with the preferred inference model.

Use Tool Calling with the RAG Endpoint

1. Send a POST request to the RAG Chat Completion endpoint to combine RAG retrieval and tool invocation. Define your tools in the tools parameter of the request body, and set tool_choice to auto, required, or none to control when the model triggers a tool call. Replace COLLECTION_ID with the collection ID and MODEL_ID with the model ID.

   CONSOLE

   Copy

   $ curl "https://api.vultrinference.com/v1/chat/completions/RAG" \
       -X POST \
       -H "Authorization: Bearer ${INFERENCE_API_KEY}" \
       -H "Content-Type: application/json" \
       -d '{
           "collection": "COLLECTION_ID",
           "model": "MODEL_ID",
           "messages": [
               { "role": "user", "content": "Ask a question that requires external data retrieval." }
           ],
           "tools": [
               {
                   "type": "function",
                   "function": {
                       "name": "function_name",
                       "description": "Describe the purpose of the function.",
                       "parameters": {
                           "type": "object",
                           "properties": {
                               "parameter_name": {
                                   "type": "string",
                                   "description": "Describe the expected input parameter."
                               }
                           },
                           "required": ["parameter_name"]
                       }
                   }
               }
           ],
           "tool_choice": "auto",
           "max_tokens": 512
       }'

   Explain Code

   Powered by Vultr AgentBeta

   The response includes a tool_calls array with the function name and its arguments.

   Run the function locally or through an API, then send its output back to the RAG endpoint in a follow-up request to generate a complete, context-aware response.

   For a complete implementation example, see How to Use Tool Calling with Vultr Serverless Inference.