OpenCode with Local Models: Pointing Your Coding Agent at Your Own Inference

export LOCAL_ENDPOINT=http://localhost:11434/v1

export OPENCODE_MODEL_NAME=llama3.1:8b

opencode

This points OpenCode at Ollama running on the default port. Replace the endpoint with your server:

| Server | Endpoint |

|--------|----------|

Method 2: Project Config (.opencode.json)

{

"provider": {

"local": {

"baseURL": "http://localhost:11434/v1",

"apiKey": "local"

}

},

"agents": {

"coder": {

"model": "local.llama3.1:8b",

"maxTokens": 4096

},

"task": {

"model": "local.llama3.1:8b",

"maxTokens": 4096

}

}

}

Method 3: CLI Flag

Override per-session:

opencode --model ollama/qwen3:8b

opencode --model local.codellama:13b

Useful for testing different models without changing config files.

What the Model Must Support

OpenCode is not just a chat wrapper. It is an agent. It calls tools: file reads, file writes, shell commands, search. This means your local model must support the OpenAI function calling / tools API format.

Models that work:

• Llama 3.1+ (8B, 70B) — native tool calling since Ollama 0.3+
• Qwen 2.5 / Qwen 3 — strong tool calling support
• Mistral / Mixtral — function calling supported
• DeepSeek Coder V2+ — tool calling supported
• Command R+ — native tool use

Models that do not work well:

• Older Llama 2 models — no native tool calling
• Phi-2 / Phi-3 (small variants) — unreliable function call formatting
• Any model without structured output capability

If the model cannot reliably produce valid JSON tool calls, OpenCode will error or hallucinate file edits. This is the primary failure mode with local models.

Context Window: The Hidden Constraint

OpenCode sends your codebase context (file contents, directory trees, previous edits) as part of each prompt. A typical agentic turn can consume 8,000-16,000 tokens of context before the model even starts reasoning.

Default Ollama context is 2048 tokens. This is not enough.

Set a larger context window:

export OLLAMA_NUM_CTX=32768

ollama serve

Or in your Modelfile:

PARAMETER num_ctx 32768

Without this, OpenCode will silently truncate context and produce nonsensical edits because the model cannot see the relevant code.

The localllm-engine Advantage

If you run localllm-engine (the router from devinfo.dev's stack), you get:

2. Automatic failover: Ollama down? Falls back to llama.cpp or cloud.

Configure OpenCode to use the engine:

{

"provider": {

"engine": {

"baseURL": "http://localhost:3001/v1",

"apiKey": "local"

}

},

"agents": {

"coder": {

"model": "engine.auto",

"maxTokens": 4096

}

}

}

Performance Expectations

Local inference is slower than cloud APIs. Set expectations:

| Model | Hardware | Tokens/sec | Usable for coding? |

|-------|----------|-----------|--------------------|

| Llama 3.1 8B Q4 | RTX 4090 | ~80 tok/s | Yes — responsive |

| Llama 3.1 8B Q4 | M2 Pro | ~40 tok/s | Yes — acceptable |

| Llama 3.1 8B Q4 | CPU only (16 core) | ~8 tok/s | Barely |

| Llama 3.1 70B Q4 | RTX 4090 | ~15 tok/s | Slow but functional |

| Qwen 2.5 Coder 32B Q4 | 2x RTX 4090 | ~25 tok/s | Good |

For coding agents, you want at minimum 20 tok/s for interactive use. Below that, the agent feels broken (long pauses between edits).

Known Issues

Model not discovered in listing:

Ollama models do not auto-populate in OpenCode's model selector. You must specify them explicitly in config or via CLI flag.

Tool calls malformed:

Some quantized models (especially Q2/Q3) produce invalid JSON in function calls. Use Q4_K_M or higher for reliable tool use.

Context overflow silent failure:

If your model's context window is exceeded, Ollama silently truncates. OpenCode does not warn you. Watch for edits that ignore recently-shown code — it means context was lost.

Streaming interruption:

Some proxy setups (nginx, reverse proxy) timeout on long-running streaming responses. Set proxy timeouts to at least 120s for agent workloads.

The Practical Setup

Minimal working configuration for a developer workstation: