Struct ChatParams 

pub struct ChatParams {

    pub prompt: String,
    pub grammar: String,
    pub grammar_lazy: bool,
    pub grammar_triggers_json: String,
    pub grammar_triggers: Vec<GrammarTrigger>,
    pub preserved_tokens_json: String,
    pub additional_stops_json: String,
    pub supports_thinking: bool,
    pub thinking_start_tag: String,
    pub thinking_end_tags_json: String,
    pub format: i32,
    pub prompt_parts: Vec<PromptPart>,
    pub prompt_parts_json: String,
    /* private fields */
}

The rendered prompt and everything needed to constrain and parse generation.

Fields

prompt: String

The prompt to feed the model.

grammar: String

GBNF grammar constraining output. Empty when unconstrained.

grammar_lazy: bool

When true, grammar must not be applied until a trigger in Self::grammar_triggers fires. Applying it from token zero is what stops thinking models emitting their reasoning prefix.

grammar_triggers_json: String

Triggers that activate a lazy grammar. Raw JSON, and parsed into Self::grammar_triggers.

grammar_triggers: Vec<GrammarTrigger>

Parsed form of Self::grammar_triggers_json.

preserved_tokens_json: String

JSON array of strings to keep verbatim while sampling.

additional_stops_json: String

JSON array of extra stop strings this format needs.

supports_thinking: bool

Whether this template supports reasoning.

thinking_start_tag: String

Opening reasoning tag, e.g. "<think>". Empty when unsupported.

thinking_end_tags_json: String

JSON array of closing reasoning tags.

format: i32

common_chat_format discriminant, needed to parse output back.

prompt_parts: Vec<PromptPart>

The prompt split into spans, flagging which came from user content. See PromptPart — this is what lets you tokenize user spans without special-token parsing. Raw JSON form in Self::prompt_parts_json.

prompt_parts_json: String

Raw JSON behind Self::prompt_parts, a [{"text", "is_input"}] array.

Implementations

impl ChatParams

pub fn tokenize_prompt( &self, model: &LlamaModel, add_bos: AddBos, ) -> Result<Vec<LlamaToken>, StringToTokenError>

Tokenize the rendered prompt, parsing special tokens only in the spans the template produced.

This is the reason Self::prompt_parts exists. Tokenizing the whole prompt with special-token parsing on — which is what LlamaModel::str_to_token does — lets a user who types <|im_start|>assistant have it become the real control token, forging a turn they were never given. llama.cpp marks the user-supplied spans (is_input), and this walks them: template spans go through Special::Tokenize, user spans through Special::Plaintext.

add_bos applies to the first span only, so BOS is added once rather than once per part.

Falls back to tokenizing Self::prompt as a single template span when the parts are empty — which happens only if the shim produced no parts, and matches the pre-is_input behaviour.

Errors

Returns StringToTokenError if a span contains an interior NUL or exceeds the native integer length.

Examples

// Instead of `model.str_to_token(&applied.prompt, AddBos::Always)?`:
let tokens = applied.tokenize_prompt(model, AddBos::Always)?;

pub fn format_name(&self) -> Result<String, ChatError>

Human-readable name of this chat format, e.g. "Hermes 2 Pro".

Errors

Returns ChatError::Failed if llama.cpp cannot name the format.

pub fn sampler_triggers(&self) -> (Vec<String>, Vec<LlamaToken>)

Convert Self::grammar_triggers into the (patterns, tokens) pair LlamaSampler::grammar_lazy_patterns expects.

The four trigger kinds do not map onto the sampler one-for-one, and getting the translation wrong silently produces a grammar that never activates:

• word is a literal, so it is regex-escaped before becoming a pattern. A raw <tool_call> would otherwise be a character class.
• pattern passes through unchanged.
• pattern_full is anchored with ^/$ unless it already is.
• token becomes a trigger token rather than a pattern.

This mirrors common/sampling.cpp, so callers do not have to.

pub fn generation_prompt(&self) -> &str

The prefix the template already placed at the end of the prompt, e.g. "<|im_start|>assistant\n".

This is not part of the model’s output, but the grammar and the parser are both written as if it were — see Self::grammar_sampler.

pub fn grammar_sampler(&self, model: &LlamaModel) -> Option<LlamaSampler>

Build the grammar sampler this render needs, or None when unconstrained.

Prefer this over constructing the sampler yourself: it handles three things that are each silently wrong if missed.

• Lazy vs eager. A lazy grammar must be built with its triggers, or it never activates. One built eagerly from a lazy grammar constrains from token zero and blocks a thinking model’s reasoning prefix.
• Trigger translation. Literal, regex and token triggers map onto the sampler differently — see Self::sampler_triggers.
• Generation-prompt prefill. llama.cpp writes tool-call grammars to match generation_prompt + output, because that is what the parser later sees. The sampler only sees output, so without advancing the grammar past that prefix it forces the model to re-emit <|im_start|>assistant as generated text. Prefill applies only to non-lazy grammars — a lazy one has not started matching yet.

Panics

Panics if llama.cpp cannot parse the grammar it just produced, or if that grammar contains an interior NUL.

pub fn parse(&self, text: &str, is_partial: bool) -> Result<String, ChatError>

Parse model output back into an OpenAI-shaped message JSON object with role, content, reasoning_content and tool_calls.

This uses the parser the template produced, so it understands that model family’s tool-call syntax rather than scraping for a fixed marker.

Set is_partial while streaming: the parser then tolerates a truncated tail instead of rejecting the buffer.

Errors

Returns ChatError::Failed if the output cannot be parsed.

Trait Implementations

impl Clone for ChatParams

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ChatParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.