Skip to main content

Module chat_template

Module chat_template 

Source
Expand description

Render a model’s own chat template (the GGUF’s tokenizer.chat_template, Jinja) into a prompt. Pure, so it is tested without loading a model.

Using the model’s own template (instead of a generic one) is what makes the model answer in its trained format, and what lets callers pass template switches such as enable_thinking.

Structs§

TemplateError
A template that failed to compile or render.
TemplateVars
Special tokens some templates reference ({{ bos_token }}).

Functions§

merge_kwargs
Model defaults overlaid with the request’s own kwargs (request wins).
render_chat
Render messages with template, ending in the assistant turn. kwargs are extra template variables (e.g. enable_thinking).