OpenCode and Ollama: Why num_ctx Is 4096, and What a Local Coding Agent Actually Costs

OpenCode and Ollama: Why num_ctx Is 4096, and What a Local Coding Agent Actually Costs

In February we published a setup guide for running OpenCode against a local Ollama instance. Eight months later OpenCode has gone from 0.5 to 1.15, a major version jump, and the obvious thing to check was whether that guide still works. It does. The configuration file written in February is read without complaint by OpenCode 1.15.6, and all four local models show up.

That is the boring half of this article. The interesting half is everything the setup guide does not tell you, because it stops the moment the thing works. What nobody writes down is why your agent forgets the file it just read, why a request can come back completely empty while still burning tokens, and how long a trivial task really takes on a laptop. All three have the same root: local inference has a budget, and the defaults spend it for you.

The model says 40960. Ollama gives you 4096

This is the single most useful thing in this article, and it explains most of the confusion around local coding agents.

Ask Ollama what your model supports and it answers honestly:

ollama show qwen3:14b | grep -i "context length"
# context length      40960

Forty thousand tokens. Plenty for a coding agent. Except that number is the model's architectural ceiling, not what you get. Start the Ollama server and read its own startup log:

msg="vram-based default context" total_vram="17.8 GiB" default_num_ctx=4096

Ollama looks at the video memory it can actually use, 17.8 GiB of the 24 GB on this machine, and decides that 4096 tokens is what it will allocate. Not 40960. The model could hold ten times more; the runtime chose not to.

Four thousand tokens is roughly one medium source file plus the conversation around it. A coding agent reads a file, runs a command, reads the output, and that budget is gone. The symptom is not an error message. The agent simply loses the earlier part of the conversation and starts answering as if it had never seen your code. People reasonably conclude the model is bad. The model is fine and was never given room to work.

Raising num_ctx, and what it costs you

There are three ways to give the agent more room, and they differ in how permanent they are.

The first is per request, useful for testing but ignored by most tools since they do not expose it:

curl http://localhost:11434/api/generate -d '{
  "model": "qwen3:14b",
  "prompt": "hi",
  "options": { "num_ctx": 16384 }
}'

The second is global, applied to every model on the machine:

# macOS, for the Ollama app
launchctl setenv OLLAMA_CONTEXT_LENGTH 32768

# or when running the server yourself
OLLAMA_CONTEXT_LENGTH=32768 ollama serve

The third, and the one worth the trouble, is a model variant that carries its own context setting. Create a Modelfile, build it once, and every tool that asks for that model name gets the larger window without knowing anything about it:

# Modelfile
FROM qwen3:14b
PARAMETER num_ctx 32768

ollama create qwen3:14b-32k -f Modelfile
ollama show qwen3:14b-32k | grep num_ctx
# num_ctx    32768

This is the version worth having, because the setting travels with the model name. Point OpenCode at qwen3:14b-32k and the context is right whatever launched it.

The cost is memory, and it is not small. Context is allocated up front as a key value cache, so eight times the window means a large block of video memory reserved before a single token is generated. On a 24 GB machine running a 14B model that is affordable. Trying the same with a 30B model on the same hardware is where you start trading context against the model staying fully on the GPU, and a model that spills to CPU is not slightly slower, it is unusable for interactive work.

The empty answer that still costs tokens

This one cost me twenty minutes of confusion, which is a good sign it is worth writing down.

A plain request through the OpenAI compatible endpoint, asking for a four word answer, with a sensible cap on output:

curl http://localhost:11434/v1/chat/completions -d '{
  "model": "qwen3:14b-32k",
  "messages": [{ "role": "user", "content": "Reply with exactly: config works" }],
  "max_tokens": 40
}'

The response came back with HTTP 200, an empty content field, forty completion tokens consumed, and finish_reason: "length". Nothing was returned and the budget was spent.

The same request with the cap raised to 600 answers correctly: config works, twelve characters, ninety six completion tokens, finish_reason: "stop".

Twelve characters cost ninety six tokens because qwen3 is a reasoning model. It thinks first, and the thinking is counted. Ollama does not hide it: alongside content the response carries a reasoning field, and in the failed request that field held a half finished thought that was cut off before the model reached its actual answer. Around eighty four tokens of deliberation for a four word reply.

This matters beyond curiosity. February's guide recommended qwen2.5-coder, which does not reason and does not do this. If you have since moved to qwen3, as most people have, every output limit you set needs to accommodate the thinking as well as the answer. A limit that was generous before is now a limit that returns nothing while charging you for it.

What it actually costs: two minutes thirty four

Benchmarks of local models usually report tokens per second, which tells you almost nothing about using one. Here is a complete, ordinary task instead.

A two line Python file with an obvious bug, and one question:

def add(a, b):
    return a - b

opencode run --model ollama/qwen3:14b-32k \
  "Read calc.py and tell me in one sentence what bug it has."

It worked, and correctly. The agent searched for the file, read it, and answered that the function subtracts instead of adding. The full agent loop, tool calls included, runs against a local model exactly as advertised.

It took two minutes and thirty four seconds. To read two lines and write one sentence.

That is the number to sit with before planning your workflow around local inference. It is not a criticism of the tools, which did what they promised on hardware that costs a fraction of a GPU server. It is the actual shape of the trade. A hosted model answers that question in about two seconds. Local costs you roughly seventy times longer, and buys you code that never leaves the machine and a bill that never arrives.

So when is this worth it

The honest answer is narrower than the enthusiasm around local AI suggests, and wider than the sceptics allow.

The part that aged well

Worth ending on, because it is unusual. A configuration file written in February, for a tool that has since crossed a major version boundary, still works unchanged in October. Same schema, same npm package, same base URL with its mandatory /v1, same "tools": true. OpenCode 1.15.6 reads it and lists every model.

In a part of the industry that rewrites its interfaces every quarter, a config that survives eight months and a major release is worth noticing. The setup guide was not what needed updating. What needed writing was everything that happens after the setup works.

Published October 2026. This is an opinion piece and analysis, not a sponsored post. CodeHelper has no commercial relationship with the companies mentioned.