Vultr Serverless Inference chat endpoint enables users to engage in chat conversations with Large Language Models (LLMs). This service allows for real-time interaction, leveraging advanced AI capabilities to facilitate dynamic and responsive communication. The endpoint also supports tool calling, letting models invoke defined functions (for example, fetch live data or call an API) during a conversation to produce data-driven responses. By integrating this endpoint, users can enhance their applications with sophisticated conversational AI, improving user experience and operational efficiency.
Follow this guide to utilize the chat endpoint on your Vultr account using the Vultr Console or API.
Navigate to Products, click Serverless, and then click Inference.
Click your target inference subscription to open its management page.
Open the Chat page.
Select a preferred model.
Provide Max Tokens value.
Send a message in the chat window.
Click History to view chat history.
Click New Conversation to create a chat window.
0 Comments
Be the first to comment and share your perspective with the community.