How to Install Cohere Toolkit on Ubuntu 24.04

Cohere Toolkit is an open-source, production-ready repository for developing generative AI applications. It provides UI components for knowledge assistants, supports models from Cohere Platform, Bedrock, and SageMaker, and connects to enterprise data sources. Install Cohere Toolkit to build and deploy Retrieval Augmented Generation (RAG) applications that are conversational, customizable, and grounded in domain-specific knowledge sources.

This article explains how to install Cohere Toolkit on Ubuntu 24.04.

Prerequisites

Before you begin, you need to:

• Have access to an Ubuntu 24.04 instance as a non-root sudo user.
• Install Docker and Docker Compose used internally by the Cohere toolkit.

Get a Cohere Toolkit API Key

1. Create a Cohere account.
2. Login to your Cohere account.
3. Select API Keys in the navigation margin.
4. Copy the default trial key from the Trial keys table and save it for later use.

Install Poetry

Poetry is a dependency management tool for Python you can use to create a dedicated environment for the Cohere Toolkit. Follow the steps to install Poetry and Python3.11 to use with the Cohere Toolkit on Ubuntu 24.04.

1. Add the deadsnakes PPA to your APT repository sources.

   CONSOLE

   Copy

   $ sudo add-apt-repository ppa:deadsnakes/ppa

   Explain Code

   Powered by Vultr AgentBeta

   The deadsnakes PPA provides access to multiple Python versions that may not be available in the default APT repository sources.

2. Update the APT package index to apply the repository changes.

   CONSOLE

   Copy

   $ sudo apt update

   Explain Code

   Powered by Vultr AgentBeta

3. Install Python 3.11.

   CONSOLE

   Copy

   $ sudo apt install python3.11

   Explain Code

   Powered by Vultr AgentBeta

   Cohere Toolkit requires Python 3.11 to run. If the build process displays warnings about an unsupported Python version, it automatically detects and uses Python 3.11 if it's available.

4. Install Poetry.

   CONSOLE

   Copy

   $ sudo apt install python3-poetry

   Explain Code

   Powered by Vultr AgentBeta

5. Verify the installed Poetry version.

   CONSOLE

   Copy

   $ poetry --version

   Explain Code

   Powered by Vultr AgentBeta

   Your output should be similar to the one below.

   Poetry (version 1.8.2)

6. Initialize a project and create a new pyproject.toml configuration.

   CONSOLE

   Copy

   $ poetry init

   Explain Code

   Powered by Vultr AgentBeta
   • Enter project details when prompted. Enter the compatible Python version as ^3.11 when prompted and press Enter after each option to save the project configuration.

   Your project configuration should be similar to the one below when successful.

   Package name [linuxuser]:
   Version [0.1.0]:
   Description []:
   Author [None, n to skip]:  n
   License []:
   Compatible Python versions [^3.12]: ^3.11
   Would you like to define your main dependencies interactively? (yes/no) [yes]
   Package to add or search for (leave blank to skip):
   Would you like to define your development dependencies interactively? (yes/no) [yes]
   Package to add or search for (leave blank to skip):
   
   Do you confirm generation? (yes/no) [yes]

7. Activate a new virtual environment using Python 3.11.

   CONSOLE

   Copy

   $ poetry env use 3.11

   Explain Code

   Powered by Vultr AgentBeta

   Your output should be similar to the one below:

   Creating virtualenv linuxuser-W6smXhkx-py3.11 in /home/linuxuser/.cache/pypoetry/virtualenvs
   Using virtualenv: /home/linuxuser/.cache/pypoetry/virtualenvs/linuxuser-W6smXhkx-py3.11

Install Cohere Toolkit

Cohere Toolkit includes two configuration templates, configuration.template.yaml and secrets.template.yaml you can use to create new configurations. Follow the steps below to clone the Cohere Toolkit GitHub repository and create a new configuration using a template.

1. Switch to your user's home directory.

   CONSOLE

   Copy

   $ cd

   Explain Code

   Powered by Vultr AgentBeta

2. Clone the Cohere Toolkit repository using Git.

   CONSOLE

   Copy

   $ git clone https://github.com/cohere-ai/cohere-toolkit.git

   Explain Code

   Powered by Vultr AgentBeta

3. Switch to the cohere-toolkit directory.

   CONSOLE

   Copy

   $ cd cohere-toolkit

   Explain Code

   Powered by Vultr AgentBeta

4. Copy the secrets.template.yaml file and save it as secrets.yaml.

   CONSOLE

   Copy

   $ cp src/backend/config/secrets.template.yaml src/backend/config/secrets.yaml 

   Explain Code

   Powered by Vultr AgentBeta

5. Open the secrets.yaml file using a text editor such as nano.

   CONSOLE

   Copy

   $ nano src/backend/config/secrets.yaml 

   Explain Code

   Powered by Vultr AgentBeta

6. Add your Cohere API key to the api_key field. Ensure there is a single space after : before the key. Replace hMtMgJGNbtymq-example with your actual key.

   INI

   Copy

   ...
   deployments:
    cohere_platform:
     api_key: hMtMgJGNbtymq-example
   ...

   Explain Code

   Powered by Vultr AgentBeta

   Save and close the file.

7. Copy the configuration.template.yaml file and save it as configuration.yaml.

   CONSOLE

   Copy

   $ cp src/backend/config/configuration.template.yaml src/backend/config/configuration.yaml 

   Explain Code

   Powered by Vultr AgentBeta

Configure Cohere Toolkit

Cohere Toolkit includes a Makefile to automate the compilation, setup, testing, and environment configurations. Follow the steps below to configure the Cohere Toolkit, including the backend and frontend.

1. Navigate to the cohere-toolkit directory.

   CONSOLE

   Copy

   $ cd ~/cohere-toolkit

   Explain Code

   Powered by Vultr AgentBeta

2. Run the first-run target using make.

   CONSOLE

   Copy

   $ sudo make first-run

   Explain Code

   Powered by Vultr AgentBeta

   The above command may terminate with an error due to psycopg2 (2.9.10) not supporting PEP 517 builds. The issue originates from the build backend and is not related to Poetry. Refer to common issue # 3 for more information.

   Your error should be similar to the one below.

   make[1]: *** [Makefile:104: setup] Error 1
   make[1]: Leaving directory '/home/linuxuser/cohere-toolkit'
   make: *** [Makefile:128: first-run] Error 2

3. Open the pyproject.toml file.

   CONSOLE

   Copy

   $ nano pyproject.toml

   Explain Code

   Powered by Vultr AgentBeta

4. Find the psycopg2 = "^2.9.9" field within the [tool.poetry.dependencies] section and comment it using the # symbol.

   ...
   psycopg2 = "^2.9.9"
   ...

   The updated section should look like the one below.

   ...
   [tool.poetry.dependencies]
   python = "~3.11"
   #psycopg2 = "^2.9.9"
   psycopg2-binary = "^2.9.9"
   ...

   • Comment the following psycopg2 = "^2.9.9" within the [tool.poetry.group.setup.dependencies] section, and add psycopg2-binary = "^2.9.9" on a new line.

     ...
     psycopg2 = "^2.9.9"
     ...

     The updated [tool.poetry.group.setup.dependencies] section should look like the one below.

     ...
     [tool.poetry.group.setup.dependencies]
     python = "~3.11"
     #psycopg2 = "^2.9.9"
     psycopg2-binary = "^2.9.9"
     ...

   Save and close the file.

5. Lock the new project dependencies.

   CONSOLE

   Copy

   $ poetry lock

   Explain Code

   Powered by Vultr AgentBeta

6. Run the first-run target again.

   CONSOLE

   Copy

   $ sudo make first-run

   Explain Code

   Powered by Vultr AgentBeta

   • Press Enter when prompted to accept the default public backend API hostname.

     Enter your public API Hostname or press enter for default [recommended]: http://localhost:8000

   • Press Enter to select the default dev build target.

     Select the build target:
     > dev
       prod

   • Press Enter to disable the Python Interpreter. Enabling it requires additional secrets.

     If you want to enable Python Interpreter, set up the following secrets. Otherwise, press enter.

   • Press Enter to disable Tavily Internet Search. Enabling it requires a TAVILY_API_KEY.

     If you want to enable Tavily Internet Search, set up the following secrets. Otherwise, press enter.
     Enter the value for TAVILY_API_KEY:

   • Select Cohere Platform as the deployment platform and press Enter to apply the changes.

     Select the model deployments you want to set up:
     > [ ] Azure
       [ ] Bedrock
       [X] Cohere Platform
       [ ] SageMaker

   • Press Enter to review the configuration variables and accept the project settings.

     Review your variables and select the ones you want to update, if any:
     > [ ] USE_COMMUNITY_FEATURES: True
       [ ] DATABASE_URL: postgresql+psycopg2://postgres:postgres@db:5432
       [ ] REDIS_URL: redis://:redis@redis:6379
       [ ] FRONTEND_HOSTNAME: http://localhost:4000
       [ ] PYTHON_INTERPRETER_URL: http://terrarium:8080
       [ ] SAGE_MAKER_REGION_NAME: us-west-2
       [ ] SAGE_MAKER_ENDPOINT_NAME: cohere-ai
       [ ] BEDROCK_REGION_NAME: us-west-2
       [ ] AZURE_CHAT_ENDPOINT_URL: https://cohere-ai.azurewebsites.net
       [ ] COHERE_API_KEY: hMtMTwIAgTjjtP1tHUVc
       [ ] NEXT_PUBLIC_API_HOSTNAME: http://localhost:8000
       [ ] BUILD_TARGET: dev
       [ ] TAVILY_API_KEY:

     The Cohere Toolkit also integrates with PostgreSQL and Redis databases. Your output should be similar to the one below when successful.

     backend-1    | INFO:     Started server process [9]
     backend-1    | INFO:     Waiting for application startup.
     backend-1    | INFO:     Application startup complete.

   • Cohere Toolkit setup completes with the following application endpoints.

     • Cohere Toolkit frontend: http://localhost:4000
     • Cohere Toolkit backend: http://localhost:8000
     • Cohere Platform model

7. Allow network connections ot the Cohere Toolkit backend port 8000 if you need to remotely access the application.

   CONSOLE

   Copy

   $ sudo ufw allow 8000/tcp

   Explain Code

   Powered by Vultr AgentBeta

   • Run the following command to install UFW if it's unavailable and allow SSH connections.

     CONSOLE

     Copy

     $ sudo apt install ufw -y && sudo ufw allow ssh

     Explain Code

     Powered by Vultr AgentBeta

8. Reload UFW to apply the firewall configuration changes.

   CONSOLE

   Copy

   $ sudo ufw reload

   Explain Code

   Powered by Vultr AgentBeta

Make Cohere API Calls

The Cohere Platform model uses two endpoints for API calls.

• Non-streaming: http://localhost:8000/v1/chat
• Streaming: http://localhost:8000/v1/chat-stream

Follow the steps below to make an API call to the Cohere backend.

• Send a request to the non-streaming endpoint using a prompt, instruction, or question. For example, run the following command to send a What is Vultr? prompt.

  CONSOLE

  Copy

  $ curl --location 'http://localhost:8000/v1/chat' \
  --header 'User-Id: user-id' \
  --header 'Content-Type: application/json' \
  --data '{
  "message": "What is Vultr?"
  }'

  Explain Code

  Powered by Vultr AgentBeta

  Your output should be similar to the one below:

  {"response_id":"33c86407-44e6-4632-b019-93f4c44a87eb","generation_id":"b9a6215b-4546-4194-9208-d4ea5d58134c","chat_history":[{"role":"USER","message":"What is Vultr?","tool_plan":null,"tool_results":null,"tool_calls":null},{"role":"CHATBOT","message":"Vultr is a cloud computing company that provides cloud infrastructure and services to businesses and individuals. It offers a range of products, including virtual private servers (VPS), cloud compute instances, block storage, and networking tools.","citations":[],"documents":[],"search_results":[],"search_queries":[],"conversation_id":"ba9dbf35-b188-45f1-81f8-5898ef3640f2","tool_calls":[],"error":null}

Conclusion

You have installed Cohere Toolkit on Ubuntu 24.04 and made an API call to the Cohere backend. You can use the toolkit to access Cohere's Command, Embed, and Rerank models and integrate them with third-party frameworks such as LangChain and LlamaIndex to develop applications using Large Language Models (LLMs). For more information and configuration options, visit the Cohere Toolkit documentation.