Module chat 

Chat templates, tool calling, and JSON-Schema-driven grammars.

This wraps llama.cpp’s common/chat.h — the layer its own server uses to turn an OpenAI-shaped request into a prompt, a GBNF grammar, and a parser that can read tool calls back out of whatever the model emits.

The value over hand-rolling it is ChatParams::grammar plus ChatParams::grammar_triggers. Naive grammar forcing breaks models that emit reasoning before a tool call: constrain from token zero and the model can never open its <think> block. A lazy grammar stays dormant until a trigger fires — usually the <tool_call> marker — so reasoning flows unconstrained and only the call itself is forced to be well-formed.

Example

let templates = ChatTemplates::from_model(model, None)?;

let applied = templates.apply(
    &ChatApplyParams::new(r#"[{"role":"user","content":"Weather in Tokyo?"}]"#)
        .with_tools(r#"[{"type":"function","function":{"name":"get_weather",
            "parameters":{"type":"object","properties":{"city":{"type":"string"}}}}}]"#)
        .with_tool_choice(ToolChoice::Required),
)?;

// Feed `applied.prompt` to the model, constrain with `applied.grammar`,
// then read the tool calls back out:
let msg = applied.parse("<tool_call>{\"name\":\"get_weather\"}</tool_call>", false)?;

Structs

ChatApplyParams

A request to render: messages, optional tools, and how to constrain output.

ChatParams

The rendered prompt and everything needed to constrain and parse generation.

ChatTemplates

The chat templates a model ships, ready to apply.

GrammarTrigger

A lazy-grammar trigger: what has to appear before the grammar engages.

Enums

ReasoningFormat

How reasoning (<think> blocks) should be handled when parsing output.

ToolChoice

What the model is allowed to do with the supplied tools.

Functions

format_name

Human-readable name of a common_chat_format discriminant.

json_schema_to_grammar

Convert a JSON Schema into a GBNF grammar.

parse_messages_oaicompat

Validate and normalise an OpenAI-shaped messages array.

parse_tools_oaicompat

Validate and normalise an OpenAI-shaped tools array into a JSON array of {"name", "description", "parameters"}.

Type Aliases

ChatError

Errors from the chat layer.