How to Install Cohere Toolkit on Ubuntu 24.04

Published on February 21, 2025•Updated on February 21, 2025

Cohere Toolkit is an open-source, production-ready repository for developing generative AI applications. It provides UI components for knowledge assistants, supports models from Cohere Platform, Bedrock, and SageMaker, and connects to enterprise data sources. Install Cohere Toolkit to build and deploy Retrieval Augmented Generation (RAG) applications that are conversational, customizable, and grounded in domain-specific knowledge sources.

This article explains how to install Cohere Toolkit on Ubuntu 24.04.

Prerequisites

Before you begin, you need to:

Have access to an Ubuntu 24.04 instance as a non-root sudo user.
Install Docker and Docker Compose used internally by the Cohere toolkit.

Get a Cohere Toolkit API Key

Create a Cohere account.
Login to your Cohere account.
Select API Keys in the navigation margin.
Copy the default trial key from the Trial keys table and save it for later use.

Install Poetry

Poetry is a dependency management tool for Python you can use to create a dedicated environment for the Cohere Toolkit. Follow the steps to install Poetry and Python3.11 to use with the Cohere Toolkit on Ubuntu 24.04.

Add the deadsnakes PPA to your APT repository sources.
console
```
$ sudo add-apt-repository ppa:deadsnakes/ppa
```
The deadsnakes PPA provides access to multiple Python versions that may not be available in the default APT repository sources.
Update the APT package index to apply the repository changes.
console
```
$ sudo apt update
```
Install Python 3.11.
console
```
$ sudo apt install python3.11
```
Cohere Toolkit requires Python 3.11 to run. If the build process displays warnings about an unsupported Python version, it automatically detects and uses Python 3.11 if it's available.
Install Poetry.
console
```
$ sudo apt install python3-poetry
```
Verify the installed Poetry version.
console
```
$ poetry --version
```
Your output should be similar to the one below.
```
Poetry (version 1.8.2)
```

Initialize a project and create a new pyproject.toml configuration.

console

$ poetry init

Enter project details when prompted. Enter the compatible Python version as ^3.11 when prompted and press Enter after each option to save the project configuration.

Your project configuration should be similar to the one below when successful.

Package name [linuxuser]:
Version [0.1.0]:
Description []:
Author [None, n to skip]:  n
License []:
Compatible Python versions [^3.12]: ^3.11
Would you like to define your main dependencies interactively? (yes/no) [yes]
Package to add or search for (leave blank to skip):
Would you like to define your development dependencies interactively? (yes/no) [yes]
Package to add or search for (leave blank to skip):

Do you confirm generation? (yes/no) [yes]

Activate a new virtual environment using Python 3.11.

                            console
                            
$ poetry env use 3.11

Your output should be similar to the one below:

Creating virtualenv linuxuser-W6smXhkx-py3.11 in /home/linuxuser/.cache/pypoetry/virtualenvs
Using virtualenv: /home/linuxuser/.cache/pypoetry/virtualenvs/linuxuser-W6smXhkx-py3.11

Install Cohere Toolkit

Cohere Toolkit includes two configuration templates, configuration.template.yaml and secrets.template.yaml you can use to create new configurations. Follow the steps below to clone the Cohere Toolkit GitHub repository and create a new configuration using a template.

Switch to your user's home directory.
console
```
$ cd
```

Clone the Cohere Toolkit repository using Git.

                            console
                            
$ git clone https://github.com/cohere-ai/cohere-toolkit.git

Switch to the cohere-toolkit directory.
console
```
$ cd cohere-toolkit
```

Copy the secrets.template.yaml file and save it as secrets.yaml.

                            console
                            
$ cp src/backend/config/secrets.template.yaml src/backend/config/secrets.yaml

Open the secrets.yaml file using a text editor such as nano.
console
```
$ nano src/backend/config/secrets.yaml
```
Add your Cohere API key to the api_key field. Ensure there is a single space after : before the key. Replace hMtMgJGNbtymq-example with your actual key.
ini
```
...
deployments:
 cohere_platform:
  api_key: hMtMgJGNbtymq-example
...
```
Save and close the file.

Copy the configuration.template.yaml file and save it as configuration.yaml.

                            console
                            
$ cp src/backend/config/configuration.template.yaml src/backend/config/configuration.yaml

Configure Cohere Toolkit

Cohere Toolkit includes a Makefile to automate the compilation, setup, testing, and environment configurations. Follow the steps below to configure the Cohere Toolkit, including the backend and frontend.

Navigate to the cohere-toolkit directory.
console
```
$ cd ~/cohere-toolkit
```
Run the first-run target using make.
console
```
$ sudo make first-run
```
The above command may terminate with an error due to psycopg2 (2.9.10) not supporting PEP 517 builds. The issue originates from the build backend and is not related to Poetry. Refer to common issue # 3 for more information.

Your error should be similar to the one below.
```
make[1]: *** [Makefile:104: setup] Error 1
make[1]: Leaving directory '/home/linuxuser/cohere-toolkit'
make: *** [Makefile:128: first-run] Error 2
```
Open the pyproject.toml file.
console
```
$ nano pyproject.toml
```
Find the psycopg2 = "^2.9.9" field within the [tool.poetry.dependencies] section and comment it using the # symbol.
```
...
psycopg2 = "^2.9.9"
...
```
The updated section should look like the one below.
```
...
[tool.poetry.dependencies]
python = "~3.11"
#psycopg2 = "^2.9.9"
psycopg2-binary = "^2.9.9"
...
```
- Comment the following psycopg2 = "^2.9.9" within the [tool.poetry.group.setup.dependencies] section, and add psycopg2-binary = "^2.9.9" on a new line.
```
...
psycopg2 = "^2.9.9"
...
```
  The updated [tool.poetry.group.setup.dependencies] section should look like the one below.
```
...
[tool.poetry.group.setup.dependencies]
python = "~3.11"
#psycopg2 = "^2.9.9"
psycopg2-binary = "^2.9.9"
...
```
Save and close the file.
Lock the new project dependencies.
console
```
$ poetry lock
```

Run the first-run target again.

                            console
                            
$ sudo make first-run

Press Enter when prompted to accept the default public backend API hostname.

Enter your public API Hostname or press enter for default [recommended]: http://localhost:8000

Press Enter to select the default dev build target.
```
Select the build target:
> dev
  prod
```

Press Enter to disable the Python Interpreter. Enabling it requires additional secrets.

If you want to enable Python Interpreter, set up the following secrets. Otherwise, press enter.

Press Enter to disable Tavily Internet Search. Enabling it requires a TAVILY_API_KEY.

If you want to enable Tavily Internet Search, set up the following secrets. Otherwise, press enter.
Enter the value for TAVILY_API_KEY:

Select Cohere Platform as the deployment platform and press Enter to apply the changes.

Select the model deployments you want to set up:
> [ ] Azure
  [ ] Bedrock
  [X] Cohere Platform
  [ ] SageMaker

Press Enter to review the configuration variables and accept the project settings.

Review your variables and select the ones you want to update, if any:
> [ ] USE_COMMUNITY_FEATURES: True
  [ ] DATABASE_URL: postgresql+psycopg2://postgres:postgres@db:5432
  [ ] REDIS_URL: redis://:redis@redis:6379
  [ ] FRONTEND_HOSTNAME: http://localhost:4000
  [ ] PYTHON_INTERPRETER_URL: http://terrarium:8080
  [ ] SAGE_MAKER_REGION_NAME: us-west-2
  [ ] SAGE_MAKER_ENDPOINT_NAME: cohere-ai
  [ ] BEDROCK_REGION_NAME: us-west-2
  [ ] AZURE_CHAT_ENDPOINT_URL: https://cohere-ai.azurewebsites.net
  [ ] COHERE_API_KEY: hMtMTwIAgTjjtP1tHUVc
  [ ] NEXT_PUBLIC_API_HOSTNAME: http://localhost:8000
  [ ] BUILD_TARGET: dev
  [ ] TAVILY_API_KEY:

The Cohere Toolkit also integrates with PostgreSQL and Redis databases. Your output should be similar to the one below when successful.

backend-1    | INFO:     Started server process [9]
backend-1    | INFO:     Waiting for application startup.
backend-1    | INFO:     Application startup complete.

Cohere Toolkit setup completes with the following application endpoints.
- Cohere Toolkit frontend: http://localhost:4000
- Cohere Toolkit backend: http://localhost:8000
- Cohere Platform model

Allow network connections ot the Cohere Toolkit backend port 8000 if you need to remotely access the application.
console
```
$ sudo ufw allow 8000/tcp
```
- Run the following command to install UFW if it's unavailable and allow SSH connections.
  console
  $ sudo apt install ufw -y && sudo ufw allow ssh
Reload UFW to apply the firewall configuration changes.
console
```
$ sudo ufw reload
```

Make Cohere API Calls

The Cohere Platform model uses two endpoints for API calls.

Non-streaming: http://localhost:8000/v1/chat
Streaming: http://localhost:8000/v1/chat-stream

Follow the steps below to make an API call to the Cohere backend.

Send a request to the non-streaming endpoint using a prompt, instruction, or question. For example, run the following command to send a What is Vultr? prompt.

                            console
                            
                        
$ curl --location 'http://localhost:8000/v1/chat' \
--header 'User-Id: user-id' \
--header 'Content-Type: application/json' \
--data '{
"message": "What is Vultr?"
}'

Your output should be similar to the one below:

{"response_id":"33c86407-44e6-4632-b019-93f4c44a87eb","generation_id":"b9a6215b-4546-4194-9208-d4ea5d58134c","chat_history":[{"role":"USER","message":"What is Vultr?","tool_plan":null,"tool_results":null,"tool_calls":null},{"role":"CHATBOT","message":"Vultr is a cloud computing company that provides cloud infrastructure and services to businesses and individuals. It offers a range of products, including virtual private servers (VPS), cloud compute instances, block storage, and networking tools.","citations":[],"documents":[],"search_results":[],"search_queries":[],"conversation_id":"ba9dbf35-b188-45f1-81f8-5898ef3640f2","tool_calls":[],"error":null}

Conclusion

You have installed Cohere Toolkit on Ubuntu 24.04 and made an API call to the Cohere backend. You can use the toolkit to access Cohere's Command, Embed, and Rerank models and integrate them with third-party frameworks such as LangChain and LlamaIndex to develop applications using Large Language Models (LLMs). For more information and configuration options, visit the Cohere Toolkit documentation.

How to Install Cohere Toolkit on Ubuntu 24.04

Prerequisites

Get a Cohere Toolkit API Key

Install Poetry

Install Cohere Toolkit

Configure Cohere Toolkit

Make Cohere API Calls

Conclusion

Products

Features

Solutions

Marketplace

Resources

Company

Tech Talks

Vultr Blogs