How to Run DeepSeek-R1 and Llama 3 Locally on Linux with Ollama

Why Run Large Language Models Locally on Linux?

Running models like DeepSeek-R1 and Llama 3 on your local Linux machine gives you full data privacy, eliminates API token billing, and provides dependable offline inference. With modern runtimes like Ollama, the deployment process requires only a few terminal commands.

Prerequisites & System Requirements

Before installing Ollama, verify that your workstation meets the following hardware baseline:

• OS: Ubuntu 22.04 LTS or Ubuntu 24.04 LTS (Debian 12 also fully supported).
• Processor: Modern x86_64 CPU with AVX2 instruction support.
• Memory: Minimum 16GB RAM for 7B/8B models; 32GB RAM recommended for 14B models.
• GPU (Optional but Recommended): NVIDIA GPU with current proprietary drivers and CUDA support.

Step 1: Installing Ollama on Ubuntu Linux

Open your terminal and run the official Linux installation script:

curl -fsSL https://ollama.com/install.sh | sh

The installer automatically creates an unprivileged ollama service user, configures systemd integration, and starts the service listening on 127.0.0.1:11434. Verify the service status:

systemctl status ollama.service

Step 2: Pulling and Running DeepSeek-R1

DeepSeek-R1 offers specialized reasoning capabilities. To pull the 8-billion parameter quantized model and initiate an interactive terminal chat, run:

ollama run deepseek-r1:8b

Ollama downloads the model layers directly to /usr/share/ollama/.ollama/models. Once the download finishes, you can send prompts immediately through your terminal.

Step 3: Running Llama 3 for General Tasks

For standard coding, summarization, and writing tasks, Meta's Llama 3.2 provides excellent latency. In a new terminal window, execute:

ollama run llama3.2:3b

Step 4: Monitoring GPU VRAM Allocation

To confirm that model weights are offloaded to your dedicated GPU rather than system RAM, run nvidia-smi during generation:

watch -n 1 nvidia-smi

You should see the ollama_llama_server process consuming VRAM matching the model size (approximately 5.6GB for DeepSeek-R1 8B Q4_K_M).

Managing Models & Useful Commands

Here are the essential management commands for daily usage:

# List all locally downloaded models
ollama list

# Check active models loaded in memory
ollama ps

# Remove an unused model to free storage
ollama rm deepseek-r1:8b

With Ollama active on your Linux workstation, you now have a private AI engine ready for local script generation, document indexing, and developer workflows.