Module chat_template 

Chat-template rendering driven by the GGUF’s own tokenizer.chat_template Jinja2 string.

Why this exists

The previous implementation (ferrox-server’s chat_template.rs, and a near-identical copy in ferrox-cli’s run.rs) sniffed the template string for literal markers — <|im_start|>, <|start_header_id|>, <start_of_turn> — and picked one of six hand-written renderers. That has three failure modes, all of them silent:

1. Every unrecognised family renders as Plain. Mistral-Instruct’s real template is [INST] … [/INST], which matches no marker, so a Mistral checkpoint was served user: hi — a prompt shape it has never seen. Same for Phi-3/Phi-4 (<|user|>…<|end|> uses the <|user|> marker but not the </s>-terminated framing the GenericRoleMarkers renderer emits), Yi, and DeepSeek-R1.
2. The tool-calling half of every template is unreachable. No hand-written renderer ever consulted tools, so the <tool_call> / <|tool▁calls▁begin|> / Gemma <|tool> grammars a model was actually trained on could not be produced.
3. A recognised family is not the same as an implemented one. ChatTemplate::Gemma4 matched gemma-4’s <|turn> marker and then rendered a three-line approximation of an 18 KB template: no thinking-channel injection, no strip_thinking on replayed assistant turns, no multimodal placeholders, no tool blocks.

So this module evaluates the template instead of recognising it. ChatTemplate::from_gguf_metadata compiles the checkpoint’s own Jinja source with minijinja; the hand-written renderers survive only as BuiltinTemplate, used for checkpoints that ship no template at all (llama.cpp’s --jinja does the same thing, defaulting to ChatML) and for the server’s synthetic-weights demo path.

Failing loudly

A template that does not compile, or that uses a filter/test/function this evaluator does not provide, produces a TemplateError that propagates to the caller as a request failure. It does not fall back to a hand-written renderer: silently serving a Mistral checkpoint ChatML framing is exactly the class of bug this module exists to delete, and the repo’s rule is to land the refusal when the math is not there. The one thing that is a fallback is “the checkpoint carries no template”, which is a genuine absence rather than a guess.

Known Jinja constructs and how they are handled:

Construct	Status
{%- … -%} whitespace control	supported by minijinja
{% macro %} / recursive macros	supported
{% set ns = namespace(...) %} and loop-scope writes through it	supported
{% set x %}…{% endset %} block set	supported
loop.index0 / loop.last / loop.first	supported
raise_exception(msg)	provided here; aborts the render with the message
strftime_now(fmt)	provided here, UTC, subset of strftime (see [strftime_now])
dictsort, map, default, trim, reject, join, slicing	minijinja builtins
tojson	reimplemented here to match jinja2’s json.dumps(sort_keys=True) byte for byte
Python methods .get(), .split(), .strip()/.lstrip()/.rstrip()	provided here (see [python_method])
anything else	hard error, never silently empty

The one deliberate difference from jinja2: undefined variables are lenient (falsy in a condition, empty when printed) rather than StrictUndefined, because that is what HuggingFace’s apply_chat_template uses and templates rely on it — e.g. gemma-3 tests {%- if add_generation_prompt -%} without the caller having to define it.

Structs

ChatTemplate

A compiled chat template. Cheap to clone (Arc inside) because every *Loaded struct in ferrox-server carries one and hands it to each request.

RenderOptions

Everything a template can read besides messages.

Enums

BuiltinTemplate

Hand-written renderers, kept only for checkpoints that ship no tokenizer.chat_template at all.

TemplateError

Everything that can go wrong between a GGUF’s template string and a rendered prompt. Every variant is a refusal, not a fallback.