Crate hf_chat_template 

Render Hugging Face chat_template (Jinja2) strings the way Python transformers.apply_chat_template does.

This crate emits a prompt string. Turning it into token IDs is the caller’s job (tokenizers, tiktoken-rs, …) — keeping that boundary is deliberate.

use hf_chat_template::ChatTemplate;
use minijinja::context;

let tmpl = ChatTemplate::from_str(
    "{% for m in messages %}<|{{ m.role }}|>{{ m.content }}\n{% endfor %}\
     {% if add_generation_prompt %}<|assistant|>{% endif %}",
).unwrap();

let out = tmpl.render_value(context! {
    messages => vec![
        context!{ role => "user", content => "hi" },
    ],
    add_generation_prompt => true,
}).unwrap();
assert_eq!(out, "<|user|>hi\n<|assistant|>");

Special tokens & BOS doubling

Templates that emit {{ bos_token }} expect you to pass bos_token in the context, and to set add_special_tokens = false at encode time so the tokenizer does not add BOS a second time. This crate renders exactly what the template says and never strips silently.

Re-exports

pub use minijinja;

Structs

ChatTemplate

A compiled Hugging Face chat template, ready to render.

ChatTemplateBuilder

Builder for ChatTemplate, exposing the knobs that affect rendering semantics.

FixedClock

A clock pinned to a fixed Unix timestamp — for deterministic golden tests.

LocalClock

Real wall-clock in the local timezone, matching Python’s datetime.now() (which is what transformers uses for strftime_now). Requires the strftime feature.

Message

A single chat message. role is typed; content is string-or-parts; all other keys (name, tool_call_id, …) flow through extra.

NamedTemplate

One entry of the list-of-named-templates form.

RenderInput

Everything a chat template renders from: the conversation plus optional tools/documents and the generation-prompt flag. Unmodeled keys land in extra and are passed to the Jinja context verbatim.

SystemClock

Real wall-clock (UTC). Dependency-free: reads SystemTime and does the calendar math here.

TokenizerConfig

A tolerant view over a parsed tokenizer_config.json.

Enums

ChatTemplateField

The chat_template field’s three historical shapes.

Content

Message content: a plain string, or a list of multimodal parts.

Error

Errors produced while compiling or rendering a chat template.

TokenField

A special-token field: either a bare string, or an AddedToken object with a content key.

Traits

Clock

Supplies “now”, formatted per a strftime-style format string. Injectable for determinism.