Running OpenClaw with Ollama: Local Models Guide

I’ve been running OpenClaw with API providers for a while now, and the bills add up. Even cheap models like GLM-5 and MiniMax M2.5 still cost something, and every message goes through someone else’s server. So I set up Ollama on the same box and pointed OpenClaw at it. Zero cost per token, total privacy, and the latency is actually decent if you pick the right model for your hardware.

This guide covers how to get OpenClaw talking to Ollama, which models work well for different hardware tiers, and what to do when your machine can’t handle the bigger ones. Short answer on that last part: Nanbeige4.1-3B is shockingly good for a 3B model.

Why Run Local Models with OpenClaw

Three reasons to bother with this instead of just using an API:

The tradeoff is real: local models are slower and less capable than Claude Opus or GPT-5. But for everyday OpenClaw stuff like answering messages and running scheduled jobs, a decent local model handles it fine.

Hardware Requirements

This is the part most guides get wrong. They tell you the model size and forget about the actual experience. Here’s what I’ve found running different tiers.

VRAM Is What Matters

Models run in GPU VRAM when available. If the model doesn’t fit in VRAM, it spills into system RAM and gets much slower. CPU-only inference works but expect 5-10x slower responses.

Recommended Builds

Installing Ollama

One command:

curl -fsSL https://ollama.ai/install.sh | sh

On macOS, download from ollama.ai or use Homebrew:

brew install ollama

Check it’s running:

ollama --version
ollama list

If ollama list works, you’re set.

Picking the Right Model

Not every model works well with OpenClaw. You need tool calling support (so the agent can use its tools) and ideally good instruction following. Here are the models I’ve tested, ranked by hardware tier.

Best Models by Tier

* Community upload on Ollama — see Nanbeige section below.

My Picks

• 32GB Mac Mini or 16GB+ GPU: qwen2.5-coder:32b as primary, qwen3:8b as fallback
• 16GB RAM, no GPU: qwen3:8b as primary, qwen3:4b as fallback
• 8GB RAM: nanbeige4.1-3b or qwen3:4b

Pull your chosen model:

ollama pull qwen2.5-coder:32b
# or for lower-end hardware:
ollama pull qwen3:8b
# or for minimal hardware:
ollama pull tomng/nanbeige4.1

Nanbeige4.1-3B: The Budget Surprise

This model caught me off guard. Nanbeige4.1-3B is a 3B parameter model from Nanbeige Lab (a team at Kanzhun/BOSS Zhipin) that punches way above its weight class. Look at these numbers:

A 3B model beating Qwen3-32B on coding benchmarks. It scores 56.5 on BFCL-V4 (tool use), which matters because OpenClaw relies on tool calling. It can also handle over 500 rounds of tool invocations for deep-search tasks. I don’t know of another sub-4B model that can do that.

When to Use It

• Your machine has 8GB RAM and no GPU
• You want a backup model that uses minimal resources
• You’re running on a Raspberry Pi 5 or similar ARM board
• You need an emergency fallback when your main model is too slow

When NOT to Use It

• You have hardware for bigger models — 8B+ will still give better results on complex tasks
• You need long creative writing or nuanced conversation
• You’re doing heavy multi-file code refactoring

Installing Nanbeige4.1-3B

The model is available through community uploads on Ollama. The most popular version with tool support:

ollama pull tomng/nanbeige4.1

You can also import the GGUF from HuggingFace if you want a specific quantization:

# Download the Q4_K_M quantization (smallest useful size, ~2.3GB)
# From: huggingface.co/Edge-Quant/Nanbeige4.1-3B-Q4_K_M-GGUF

# Create a Modelfile
cat > Modelfile << 'EOF'
FROM ./Nanbeige4.1-3B-Q4_K_M.gguf
PARAMETER temperature 0.6
PARAMETER top_p 0.95
PARAMETER repeat_penalty 1.0
TEMPLATE """{{- if .System }}{{ .System }}{{ end }}
{{- range .Messages }}
{{- if eq .Role "user" }}
{{ .Content }}
{{- else if eq .Role "assistant" }}
{{ .Content }}
{{- end }}
{{- end }}"""
EOF

ollama create nanbeige4.1-3b -f Modelfile

Configuring OpenClaw for Ollama

Step 1: Enable Ollama

Set the environment variable that tells OpenClaw to look for Ollama:

export OLLAMA_API_KEY="ollama-local"

Or add it permanently to your OpenClaw environment:

# Add to ~/.openclaw/.env
echo 'OLLAMA_API_KEY=ollama-local' >> ~/.openclaw/.env

The value doesn’t matter (Ollama doesn’t check it), but OpenClaw needs it set to enable the Ollama provider.

Step 2: Verify Discovery

OpenClaw auto-discovers tool-capable Ollama models. Check what it found:

openclaw models list --local

You should see your pulled models listed. If a model doesn’t appear, it might not report tool support. You can still use it by adding explicit config (see Step 3).

Step 3: Set Your Model

For a straightforward setup with one model:

openclaw models set ollama/qwen2.5-coder:32b

Or edit your config file (~/.openclaw/openclaw.json):

{
  "agents": {
    "defaults": {
      "model": {
        "primary": "ollama/qwen2.5-coder:32b"
      }
    }
  }
}

Step 4: Set Up Fallbacks

Fallbacks matter more with local models because a single model might be too slow or run out of context. Configure a chain:

{
  "agents": {
    "defaults": {
      "model": {
        "primary": "ollama/qwen2.5-coder:32b",
        "fallbacks": [
          "ollama/qwen3:8b",
          "ollama/tomng/nanbeige4.1"
        ]
      }
    }
  }
}

If the 32B model chokes on a long context, OpenClaw falls through to the 8B, then to Nanbeige.

Full Config Example (Mid-Range Hardware)

Here’s a complete config for a 32GB Mac Mini:

{
  "agents": {
    "defaults": {
      "model": {
        "primary": "ollama/qwen2.5-coder:32b",
        "fallbacks": ["ollama/qwen3:8b"]
      },
      "imageModel": {
        "primary": "ollama/mistral-small3.2:24b"
      }
    }
  }
}

Full Config Example (Low-End Hardware)

For an 8GB RAM machine or Raspberry Pi:

{
  "agents": {
    "defaults": {
      "model": {
        "primary": "ollama/tomng/nanbeige4.1",
        "fallbacks": ["ollama/qwen3:1.7b"]
      }
    }
  }
}

Explicit Provider Config (Remote Ollama)

If Ollama runs on a different machine (say a GPU server on your LAN), you need explicit config instead of auto-discovery:

{
  "models": {
    "providers": {
      "ollama": {
        "baseUrl": "http://192.168.1.50:11434",
        "apiKey": "ollama-local",
        "api": "ollama",
        "models": [
          {
            "id": "qwen2.5-coder:32b",
            "name": "Qwen 2.5 Coder 32B",
            "reasoning": false,
            "input": ["text"],
            "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 },
            "contextWindow": 32768,
            "maxTokens": 8192
          }
        ]
      }
    }
  },
  "agents": {
    "defaults": {
      "model": {
        "primary": "ollama/qwen2.5-coder:32b"
      }
    }
  }
}

Hybrid Setup: Local + API Fallback

The setup I actually recommend is a hybrid. Local model as primary, cheap API as fallback for when the local model struggles. You get privacy and zero cost for 90% of messages, and API quality is there when you need it.

{
  "agents": {
    "defaults": {
      "model": {
        "primary": "ollama/qwen2.5-coder:32b",
        "fallbacks": [
          "ollama/qwen3:8b",
          "zai/glm-5"
        ]
      }
    }
  }
}

With this config, OpenClaw tries the local 32B first. If context is too long or the model errors, it falls to the local 8B. If that also fails, it hits the GLM-5 API (which is cheap). You can set up GLM-5 or MiniMax M2.5 as your API fallback following the best open source models guide.

Switching Models On the Fly

You don’t have to restart OpenClaw to change models. Use the /model command in chat:

/model                        # Show available models
/model list                   # Full list with providers
/model ollama/qwen3:8b        # Switch to a different model
/model status                 # Check current model + auth

Good for testing: pull a new model with ollama pull, and it shows up in /model list automatically (with auto-discovery enabled).

Performance Tuning

Context Window

Ollama reports the context window from the model metadata. You can override it in explicit config:

{
  "models": {
    "providers": {
      "ollama": {
        "models": [{
          "id": "qwen2.5-coder:32b",
          "contextWindow": 65536,
          "maxTokens": 16384
        }]
      }
    }
  }
}

Larger context windows use more VRAM. If you’re hitting memory limits, reduce contextWindow.

Running Multiple Models

Ollama can keep multiple models loaded if you have the VRAM. Set OLLAMA_NUM_PARALLEL to control concurrency:

OLLAMA_NUM_PARALLEL=2 ollama serve

With enough VRAM, you can run a coding model and a general model simultaneously, switching between them with /model in OpenClaw.

GPU Layers

If you have a GPU but not enough VRAM for the full model, Ollama automatically offloads some layers to CPU. You can control this:

OLLAMA_GPU_LAYERS=35 ollama serve

More layers on GPU = faster but more VRAM. Experiment to find your sweet spot.

Troubleshooting

ollama list
curl http://localhost:11434/api/tags

ollama serve

curl http://localhost:11434/api/tags

ollama pull tomng/nanbeige4.1

Security Considerations

Running models locally is inherently more private than API calls, but don’t skip the basics:

• Ollama has no auth: anyone on your network who can reach port 11434 can use your models. Bind to localhost or use a firewall.
• Model downloads: Ollama pulls models from ollama.com. If you’re security-conscious, verify model checksums or use airgapped installs.
• OpenClaw security: local models don’t change the OpenClaw threat model. Still follow the security hardening guide for gateway binding, sandbox mode, and skill vetting.

Related Guides

• OpenClaw Setup Guide — full installation on Hetzner VPS or Mac Mini
• Best Open Source Models for OpenClaw — API-based models (GLM-5, MiniMax M2.5) for when local isn’t enough
• DuckDuckGo Search for OpenClaw — free web search that pairs well with local models
• OpenClaw Security Guide — hardening your instance, especially relevant when running on a LAN
• Best OpenClaw Dashboards — dashboards that show model usage and cost (useful for tracking $0 local usage vs API fallback spend)
• OpenClaw Alternatives — other platforms with their own local model support

Local models with Ollama won’t replace Claude Opus for complex work. But they cover 80-90% of what I use OpenClaw for: answering messages, running cron jobs, lookups, automations. The hybrid setup (local primary + API fallback) means privacy and zero cost most of the time, with API quality when you actually need it.

Start with whatever model fits your hardware. If you’ve got a Mac Mini, go straight to qwen2.5-coder:32b. If you’re on a budget VPS with 8GB RAM, pull Nanbeige4.1-3B and be surprised.

For the rest of the OpenClaw stack: setup guide, API model recommendations, free web search, security hardening, dashboards, and alternative platforms.