The Chat Template Is the Interface

The Chat Template Is the Interface

A language model is not trained on raw user messages. It is trained on formatted sequences: system blocks, user turns, assistant turns, each wrapped in model-specific control tokens.

Llama 3 uses <|start_header_id|>user<|end_header_id|>. Mistral uses [INST] and [/INST]. Qwen uses ChatML <|im_start|> and <|im_end|>. DeepSeek's template is non-linear — it strips trailing tokens from previous turns before rendering the next. They are not interchangeable.

The chat template is a Jinja2 program stored in the model's metadata as tokenizer.chat_template. When you load a GGUF file, that program is what sits between your structured API call and the raw token sequence the model actually sees.

Get it wrong and you have not misconfigured a parameter. You have broken the interface.

What "wrong" looks like in practice:

Hugging Face's documentation states it directly: "Using a format different from the format a model was trained with will usually cause severe, silent performance degradation." Silent is the operative word. The model still responds. Outputs still appear. Nothing throws an exception. The degradation is invisible unless you are specifically looking for it — and most people are not.

In Ollama, if you import a custom GGUF without specifying a TEMPLATE in the Modelfile, the default is {{ .Prompt }} — user input sent verbatim. Correct for a raw completion model. Wrong for every instruction-tuned model ever. The model sees your question without its expected framing tokens, and it responds as if the conversation structure it was trained on does not exist.

In llama.cpp, the --jinja flag enables Jinja2 template execution directly from GGUF metadata via llama_chat_apply_template(). Without it, the engine falls back to a hardcoded list of recognized templates. If your model's template is not on that list, you get the fallback. If the fallback does not match training, performance degrades.

Why engineers miss this:

They test with official models first. Official models in Ollama have correct templates packaged automatically. The bug only appears when importing a custom fine-tune, a LoRA fusion, or a quantization from an unfamiliar source.

At that point, the template embedded in the GGUF may differ from the canonical template in the model's tokenizer_config.json on Hugging Face. It may have been set incorrectly by the quantizer. It may have been lost in format conversion. The model card says nothing about it. The user sees degraded output and blames the quantization.

The fix is one step:

Run ollama show --modelfile and read the TEMPLATE field. Compare it to the training template in tokenizer_config.json on Hugging Face. They should be identical in structure. If they are not, override with the correct template in your Modelfile.

For llama.cpp server, pass --chat-template-file with the correct Jinja2 source, or use --jinja to enable native Jinja2 evaluation from GGUF metadata directly.

The chat template is not a configuration detail. It is the interface between your structured input and the model's learned behavior. Treat it like one.

References

1. Hugging Face, 2023. "Chat Templates: An End to the Silent Performance Killer." Hugging Face Blog. https://huggingface.co/blog/chat-templates

2. Hugging Face. "Chat templates." Hugging Face Transformers Documentation. https://huggingface.co/docs/transformers/en/chat_templating

3. Hugging Face. "Advanced Usage and Customizing Your Chat Templates." Hugging Face Transformers Documentation. https://huggingface.co/docs/transformers/main/en/chat_template_advanced

4. ggml-org. "Templates supported by llama_chat_apply_template." llama.cpp GitHub Wiki. https://github.com/ggml-org/llama.cpp/wiki/Templates-supported-by-llama_chat_apply_template

5. ochafik, 2024. "Add Jinja template support." llama.cpp Pull Request #11016. https://github.com/ggml-org/llama.cpp/pull/11016

6. ollama/ollama. "docs/template.mdx." Ollama GitHub repository. https://github.com/ollama/ollama/blob/e09b3f9f/docs/template.mdx

7. Hugging Face Forums, 2026. "What Is the Right Way to Configure GGUF Models? (Templates, Parameters, Model Creation)." https://discuss.huggingface.co/t/what-is-the-right-way-to-configure-gguf-models-templates-parameters-model-creation/175182

devinfo.dev. (2026). "The Chat Template Is the Interface." devinfo.dev:2026.0048. https://devinfo.dev/d/2026.0048