How to Deploy Large Language Models (LLMs) with Ollama
Ollama is an open-source platform for running large language models (LLMs) locally. Ollama supports most open-source Large Language models (LLMs) including Llama 3, DeepSeek R1, Mistral, Phi-4, and Gemma 2 models, you can run locally without an internet connection. It enhances security, privacy, and provides greater control over a model's performance, allowing you to customize it on your workstation.
Ollama includes a default models repository which lets you search and download LLMs to run on your workstation. It also supports OpenWebUI as a graphical web interface for running models without using the command line interface (CLI). It works on Linux, Windows, and macOS without any cloud-based APIs requirements to run models.
This article explains how to install Ollama and run large language models (LLMs) with all required dependencies on your workstation.
Download and Install Ollama
Ollama is compatible with Linux, macOS, and Windows. You can download and install Ollama using the official release package or script from the official website. Follow the steps below to download and install the latest Ollama version on your Linux, macOS, or Windows workstation.
Open a new terminal session.
Run the following command to download the
install.shOllama script and install it on your instance.CONSOLEVerify the installed Ollama version.
CONSOLEYour output should be similar to the one below.
List all available models.
CONSOLE
Manage the Ollama System Service on Linux
Ollama creates an ollama.service system service by default when installed, to manage the application. Follow the steps below to test the Ollama system service and enable it to automatically start at boot.
Test the Ollama service status and verify that it's available.
CONSOLEOutput:
Enable Ollama to automatically start at boot.
CONSOLERestart the Ollama service.
CONSOLE
Optional: Install AMD GPU ROCm drivers for Ollama
Download the latest Ollama archive with
rocmsupport from the official website.CONSOLEExtract all files from the archive to the
/usr/directory.CONSOLE
Download LLMs Using Ollama
Ollama downloads models using the ollama pull command that fetches the model to run on your system. You can download models from the Ollama models repository. Follow the steps below to download LLMs using Ollama to run on your workstation.
Run the Ollama with the
pulloption to download a model from the official repository.CONSOLEFor example, download Mistral using Ollama.
CONSOLEDownload the
Deepseek-R1-Distrill-Qwenmodel with 1.5B parameters using Ollama.CONSOLEDownload Llama 3.3 using Ollama.
CONSOLEThe Llama 3.3 model is
40GBin size. Verify that you have enough storage space to run this model before downloading it.List all locally available models using Ollama.
CONSOLEYour output should be similar to the one below.
Use Ollama to Run Models
You can run, pull and initialize large language models directly from the Ollama repository or locally from the available models using the ollama run command. Check the required system resources for each AI model before running it using Ollama. Follow the steps below to run models using Ollama and test the performance of each model on your workstation.
List all locally available models.
CONSOLERun a model directly using Ollama. For example, run the
qwen 2.5instruct model with1.5Bparameters.CONSOLEEnter a new prompt such as
Give me two lines of text about Vultr.Monitor the model output and results in your terminal.
Enter
/byeto exit Ollama.Run a model that's available on your workstation. For example, run the Deepseek R1 model you downloaded earlier.
CONSOLEEnter a prompt such as
Generate a recursive fractal pattern description using only mathematical notations and symbolic logic.Monitor the model output and results in your terminal.
Enter
/byeto exit Ollama.Different models excel at specific tasks and running them using Ollama allows you to test the performance and model capabilities. Use the
ollama runcommand to run locally available models, or pull the latest version from the Ollama models repository to run on your system.
Manage Available Models Using Ollama
Downloading multiple LLMs on your system is a complex task that requires continuous management using Ollama. Use the ollama list command to view all available and delete any versions you no longer need on your workstation. Follow the steps below to view and manage all available models using Ollama.
List all locally available models.
CONSOLEShow information about a specific model. For example, display the
Llama 3.3model information.CONSOLEOutput:
Stop a model that's actively running using Ollama.
CONSOLEFor example, stop the Deepseek R1 model.
CONSOLERemove a model from your workstation. For example, remove the
mistralmodel you pulled earlier.CONSOLEOutput:
List all locally available models and verify that the model is unavailable.
CONSOLE
Set Ollama Environment Variables
Ollama variables modify the application behavior and allow you to effectively run models on your workstation. The following are the most used Ollama variables.
OLLAMA_HOST: Specifies the Ollama server address.OLLAMA_GPU_OVERHEAD: Reserve a part of VRAM per GPU in bytes.OLLAMA_MODELS: Specifies a custom directory to store the model files on your system.OLLAMA_KEEP_ALIVE: Controls how long models can stay in memory.OLLAMA_DEBUG: Enables additional debug information while running LLMs.OLLAMA_FLASH_ATTENTION: Activates experimental optimizations for attention mechanismsOLLAMA_NOHISTORY: Disables readline history while running LLMs.OLLAMA_NOPRUNE: Disables pruning of model blobs at system boot or startup.OLLAMA_ORIGINS: Configures origin URLs with access to the Ollama server.
Follow the steps below to set Ollama environment variables based on your operating system.
Set Ollama Variables on Linux
Open the Ollama system service file.
CONSOLEAdd the
Environmentcommand after the[service]declaration and include an Ollama variable. For example, enterEnvironment="OLLAMA_DEBUG=1"to enable debugging.Add another Ollama environment variable such as
OLLAMA_HOST=0.0.0.0:11434to allow other hosts to connect to the Ollama server.INISave and close the file.
Setting the Ollama host address to
0.0.0.0allows other hosts to connect and access Ollama using the specified port.
Reload systemd to apply the service configuration changes.
CONSOLERestart Ollama to apply the environment variables.
CONSOLE
Set Ollama Variables on macOS
Quit Ollama if it's running in your terminal session.
Use the
launchctlcommand to set environment variables on Mac. For example, set theOLLAMA_HOSTvariable.CONSOLEStart Ollama to apply the variable changes.
CONSOLE
Set Ollama Variables on Windows
Quit Ollama if it's running.
Open the Windows search menu, search for
Environment Variables, and select Edit the System Variables from the search results.Click Environment Variables in the open System Properties window.
Click New to create a new variable.
Enter your Ollama variable in Variable name.
Enter a value in the Variable value.
Click OK to save the environment variable.
Click OK to close the Environment Variables prompt.
Click Apply to apply the variable changes and close the System Properties dialog.
Conclusion
You have installed Ollama and run large language models (LLMs) on your workstation. Ollama is an efficient platform for running LLMs locally or on a remote server. Use Ollama variables to create a dedicated workstation to run and access your models remotely from any device, including mobile devices. For more information and model examples, visit the Ollama GitHub repository.
0 Comments
Be the first to comment and share your perspective with the community.