Ollama Docker Compose: Setup Guide with Open WebUI (2026)

Running Ollama with Docker Compose is the cleanest way to self-host a local AI stack: two containers, one file, and your data never leaves your machine. This guide pairs Ollama (v0.40.3 as of Oct 2026) with Open WebUI (v0.12.0) behind Traefik, with ready-to-use Compose files for CPU, NVIDIA GPU, and AMD ROCm setups.

You get three hardware variants of the same stack, a verify step after each stage, a rollback path for updates, and honest notes on the hardware you need. CPU-only is fine for tinkering. If you want to daily-drive bigger models, plan on a GPU box or a mini PC.

What is Ollama?

Ollama is the tool most people reach for when they want large language models running on their own hardware. It downloads and serves models behind a small REST API, and it keeps model files on disk instead of in someone else’s cloud. Library models come pre-quantized, so there is no quantize step in the download path. As of Oct 2026 the current release is v0.40.3.

• Models run on your machine, so prompts and documents stay private.
• ollama pull, ollama list, and ollama rm cover day-to-day model management.
• Everything the CLI does is also on the REST API at http://localhost:11434, which is what Open WebUI talks to.
• It runs on Linux, macOS, and Windows, with official Docker images (ollama/ollama, ~3.8 GB, ~185M pulls on Docker Hub).
• GPU support covers NVIDIA CUDA and AMD ROCm, plus a bundled Vulkan runtime for cards without official ROCm support.

The project is still moving fast. v0.30 rebuilt the engine on a new llama.cpp compatibility layer, and since v0.32 the ollama CLI can drop you into an interactive coding agent, with ollama launch driving external agents. For a Docker deployment none of that changes the compose file, but it is a good sign the base layer is not stagnant.

What is Open WebUI?

Ollama is happy on the command line. Most people are not. Open WebUI is the browser interface I put in front of it: a chat app that talks to Ollama and to other providers (OpenAI, Anthropic, anything OpenAI-compatible) from one screen. As of Oct 2026 the current release is v0.12.0.

• Point it at local Ollama and remote APIs side by side, and enable or disable each connection.
• The chat workspace has conversation history, prompt templates, export, and model switching per message.
• Users and auth cover local accounts, OAuth, and admin roles. The first account created becomes the admin.
• Since v0.9 there is a packaged desktop app for Mac/Windows/Linux if you want out of the browser.
• Since v0.10 you can share chats and knowledge per user or group.
• The v0.11 admin panel redesign is why older screenshots and menus in other guides look wrong now.
• Since v0.12 there is realtime voice on an OpenAI Realtime backend, plus personal MCP/tool-server integrations behind the ENABLE_DIRECT_INTEGRATIONS admin switch (off by default).

One detail that matters here: Open WebUI stores everything (users, chats, settings) in a SQLite database inside its data volume. That volume is the thing you back up. Ollama’s model blobs are disposable; webui.db is not.

How to set up Ollama and Open WebUI with Docker Compose

What we are building: one Compose file with two services. Open WebUI gets Traefik labels so it is reachable on your domain over HTTPS. Ollama stays internal. Neither container publishes a port to the host, so nothing is exposed to the internet except Traefik on 443.

Ollama and Open WebUI architecture on Docker Compose: browser to Traefik on traefik-net to the Open WebUI container on internal port 8080, which calls the Ollama container on internal port 11434; neither app container publishes ports

If you are interested to see some free cool open source self hosted apps you can check toolhunt.net self hosted section.

1. Prerequisites

Before you start, this must already exist on the host:

Hardware reality check, because this is where most local AI projects stall. A CPU-only VPS is for tinkering. An 8 vCPU / 16 GB box with no GPU will crawl on anything past the small models. If you want responsive daily use, get a GPU box, or run it at home on a mini PC: the mini PC as home server picks and the local LLM mini PC guide (for hardware that can hold a 70B model) both cover that. For rented hardware I use Hetzner and Hostinger; for home hardware, a compact mini PC like the ASUS DC510 is the right form factor.

VPS prices jumped across the board in 2026. If you are rethinking a rented box, see what changed and when a mini PC wins.

2. Docker Compose

One compose file, three hardware variants. Pick the one that matches your box. A few settings apply to all three:

• restart: unless-stopped on both services. The official snippets sometimes skip it on Ollama; add it unless you enjoy re-running docker compose up after every host reboot.
• OLLAMA_BASE_URL: http://ollama:11434 on the Open WebUI service. This is the single-backend variable.

If you are behind a corporate proxy, set HTTPS_PROXY on the ollama service only. The Ollama docs explicitly warn that HTTP_PROXY can break model pulls.

CPU only

services:
  open-webui:
    image: ghcr.io/open-webui/open-webui:main   # pin ghcr.io/open-webui/open-webui:v0.12.0 for shared/prod
    container_name: openwebui
    hostname: openwebui
    restart: unless-stopped
    networks:
      - traefik-net
    volumes:
      - ./open-webui-local:/app/backend/data
    environment:
      OLLAMA_BASE_URL: http://ollama:11434
      WEBUI_SECRET_KEY: change-me-openssl-rand-hex-32
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.openwebui.rule=Host(`openwebui.domain.com`)"
      - "traefik.http.routers.openwebui.entrypoints=https"
      - "traefik.http.services.openwebui.loadbalancer.server.port=8080"

  ollama:
    image: ollama/ollama:latest                 # or pin ollama/ollama:0.40.3
    container_name: ollama
    hostname: ollama
    restart: unless-stopped
    volumes:
      - ./ollama-local:/root/.ollama
    environment:
      OLLAMA_CONTEXT_LENGTH: "8192"             # small-RAM box; server default is VRAM-tiered (4k/32k/256k)

networks:
  traefik-net:
    external: true

What the Traefik labels do:

• traefik.enable=true: lets Traefik route to this container.
• traefik.http.routers.openwebui.rule=Host(`openwebui.domain.com`): matches traffic for that hostname. Replace with your own DNS name.
• traefik.http.routers.openwebui.entrypoints=https: serves it on the HTTPS entrypoint only.
• traefik.http.services.openwebui.loadbalancer.server.port=8080: Open WebUI listens on 8080 inside the container. This is still correct in v0.12.

No ports: section anywhere. That is deliberate. Ollama is only reachable as http://ollama:11434 from inside traefik-net.

On OLLAMA_CONTEXT_LENGTH: Ollama picks a default context by available VRAM (4k under 24 GiB, 32k for 24-48 GiB, 256k above). Setting 8192 is a sane floor for small boxes and keeps memory use predictable.

Docker Compose NVIDIA GPU

First install the NVIDIA Container Toolkit. This is what lets containers see the GPU at all. Without it, --gpus and the deploy block below do nothing.

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
 && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

# Configure NVIDIA Container Toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
  sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
sudo yum install -y nvidia-container-toolkit

# Configure NVIDIA Container Toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Smoke-test the GPU integration before touching Compose. This is verify step 1; if it fails, nothing after it will work.

docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

You want the usual GPU table (driver version, CUDA version, device list). If you get “could not select device driver”, the toolkit is not configured with Docker or the driver is missing.

Support floor as of Oct 2026: NVIDIA compute capability 5.0+ (Maxwell and newer), driver >= 550. Older cards in the 5.0-6.2 range need driver >= 570. RTX 50xx cards (compute capability 12.0) are supported.

The Compose file is the CPU one plus this block on the ollama service:

  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    hostname: ollama
    restart: unless-stopped
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              capabilities: ["gpu"]
              count: all
    networks:
      - traefik-net
    volumes:
      - ./ollama-local:/root/.ollama

This hands every visible GPU to the Ollama container. On a multi-GPU box, pin specific devices with CUDA_VISIBLE_DEVICES (UUIDs from nvidia-smi -L are the reliable handle). On Jetson hardware, pass JETSON_JETPACK=5 (or =6) to the container instead.

GPU acceleration versus CPU is a big difference on most models. Exact numbers depend on the model and quantization, so run ollama ps on your own hardware before you trust any figure you read online. CPU-only is for tinkering, not daily driving.

One clarification worth having in writing: ghcr.io/open-webui/open-webui:cuda only accelerates Open WebUI’s own local models (embeddings, reranking, Whisper). Models served by Ollama use the GPU only if the ollama container has GPU access. The deploy block above is what matters.

Known failure mode: after a Linux suspend/resume cycle Ollama silently drops to CPU. Fix is sudo rmmod nvidia_uvm && sudo modprobe nvidia_uvm, then restart the ollama container.

Docker Compose AMD GPU

Two paths here, and the right one depends on whether your card is in AMD’s official ROCm support list.

  ollama:
    image: ollama/ollama:rocm
    container_name: ollama
    hostname: ollama
    restart: unless-stopped
    devices:
      - /dev/kfd
      - /dev/dri
    networks:
      - traefik-net
    volumes:
      - ./ollama-local:/root/.ollama

  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    hostname: ollama
    restart: unless-stopped
    devices:
      - /dev/kfd
      - /dev/dri
    networks:
      - traefik-net
    volumes:
      - ./ollama-local:/root/.ollama

3. Start the Docker Compose file

docker compose up -d

Then confirm both containers are up:

docker compose ps
docker compose logs -f --tail=50

You want openwebui and ollama in Up state. The first boot of Open WebUI takes 30-60 seconds while it creates webui.db; you are ready when the login page renders in the browser. You can deploy the same file from Dockge if you prefer a UI; see the Dockge install guide.

If compose ps exits with a network error, check that the external network exists: docker network ls | grep traefik.

4. Access Open WebUI and create the admin account

Open the domain you set in the Traefik label (the example uses openwebui.domain.com). You will be prompted to create a user and password. The first account created becomes the admin, so do not hand this URL to anyone else before you have signed up.

Once the account exists, lock the door: set signups to closed by adding this to the open-webui service environment and recreating the container.

    environment:
      ENABLE_SIGNUP: false

docker compose up -d --force-recreate

That recreate matters on first boot only. See the gotcha below.

docker exec -it openwebui sqlite3 /app/backend/data/webui.db "UPDATE config SET data = json_set(data, '$.ENABLE_SIGNUP', json('false'));"

Cleaner option for a solo operator: skip the signup dance entirely. Set WEBUI_ADMIN_EMAIL and WEBUI_ADMIN_PASSWORD on the open-webui service before first boot. Open WebUI creates the admin account headlessly at startup and disables signup for you. No second recreate.

5. Pulling a model

With the admin account in place, pull a model. The menu moved in the v0.11 admin redesign, so older guides will send you to the wrong screen.

Current path (v0.12): Settings -> Admin -> Connections, find your instance under Manage Ollama API Connections, click Manage (the download icon), and download models from there. The same dialog has an Update All Models button and Create a model if you want a variant with a fixed system prompt.

Faster path: type the model name directly in the model selector at the top of the chat and accept the download prompt.

CLI alternative, which is what I use on servers:

docker exec -it ollama ollama pull qwen3:0.6b
docker exec -it ollama ollama list

Ollama also pulls GGUF straight from Hugging Face with the hf.co/{username}/{repository}:{quantization} pattern, for example hf.co/ggml-org/gemma-3-270m-it-GGUF:Q8_0. Useful when a model is not in the Ollama library yet.

Once a model answers in chat, the stack is serving. If you want to point a coding agent at it next, you can run the OpenClaw coding agent against your local Ollama models.

6. Recommended Ollama models

Picking a model is mostly a VRAM/RAM question. Download sizes track parameter count and quantization; what matters at runtime is whether the weights plus KV cache fit in memory. A few picks as of Oct 2026, with the hardware tier they belong on:

Two practical notes. First, always prefer a smaller quantization of a good model over a big model that swaps to disk; swapping destroys latency. Second, verify the exact tag names on ollama.com/library before you script pulls; tags change and I would rather you check than trust a list that ages. For a longer comparison of open weights, see our comparison of the best open source LLMs.

7. Verify the setup

This is the section most tutorials skip. Five minutes here saves an evening of guessing.

# GPU in use? (PROCESSOR should read 100% GPU; note the CONTEXT column)
docker exec -it ollama ollama ps

# API up inside the compose network?
docker exec -it ollama curl -s http://localhost:11434/api/version

# Same check from the Open WebUI container
docker exec -it openwebui curl -s http://ollama:11434/api/version

If curl is missing inside a container, the fallback is docker compose logs ollama and looking for the “listening on” line.

End-to-end, you can also expose your models to any app through an OpenAI-compatible gateway with LiteLLM on Docker Compose and point other tools at the gateway instead of at Ollama directly.

8. How to update Ollama and Open WebUI

Both projects release often. The update itself is two commands; the backup is the part people skip and regret.

# backup first
tar czf webui-backup-$(date +%F).tgz ./open-webui-local

# pull new images and recreate
docker compose pull && docker compose up -d

# then hard-refresh the browser (Ctrl+F5) so cached UI assets do not lie to you

After updating, hit /api/version again and check the version string in the logs. Do the same for Open WebUI in Admin -> Settings -> About.

Tag strategy:

For a homelab or a single user, keep :main / :latest and pull manually when you feel like it. Read the release notes first. For shared or production use, pin the tags (ghcr.io/open-webui/open-webui:v0.12.0, ollama/ollama:0.40.3 in the Compose file) and bump them deliberately. Never point a real data volume at :dev. A dev database migration can leave the volume unreadable by release builds; the docs say to use a separate volume entirely.

Rollback: untar the backup over ./open-webui-local, put the previous pinned tags back in the Compose file, docker compose up -d, then hard-refresh. Model blobs in ./ollama-local do not need backing up; delete and re-pull them if anything is corrupt. That is also why the backup tar stays small.

9. Troubleshooting common failures

A few shorter questions that come up:

Conclusions

That is the whole setup: Ollama and Open WebUI on Docker Compose, two containers behind Traefik, with CPU, NVIDIA, and AMD variants of the same file. Your prompts, chats, and model traffic stay on hardware you control. The verify step and the backup-before-update habit are the two things that make this stack boring to run instead of fragile.

GPU box for daily driving, CPU VPS for tinkering. Do not skip ollama ps after first boot and do not skip the webui.db tar before upgrades.

For more things to run next to it, see Best 100+ Docker Containers for Home Server and another AI workspace you can self-host with Docker. For the wider local AI and agent catalog on GitHub, see top AI GitHub repos.