Struct ChatParams 

pub struct ChatParams {
    pub prompt: String,
    pub grammar: String,
    pub grammar_lazy: bool,
    pub grammar_triggers_json: String,
    pub grammar_triggers: Vec<GrammarTrigger>,
    pub preserved_tokens_json: String,
    pub additional_stops_json: String,
    pub supports_thinking: bool,
    pub thinking_start_tag: String,
    pub thinking_end_tags_json: String,
    pub format: i32,
    /* private fields */
}

The rendered prompt and everything needed to constrain and parse generation.

Fields

prompt: String

The prompt to feed the model.

grammar: String

GBNF grammar constraining output. Empty when unconstrained.

grammar_lazy: bool

When true, grammar must not be applied until a trigger in Self::grammar_triggers fires. Applying it from token zero is what stops thinking models emitting their reasoning prefix.

grammar_triggers_json: String

Triggers that activate a lazy grammar. Raw JSON, and parsed into Self::grammar_triggers.

grammar_triggers: Vec<GrammarTrigger>

Parsed form of Self::grammar_triggers_json.

preserved_tokens_json: String

JSON array of strings to keep verbatim while sampling.

additional_stops_json: String

JSON array of extra stop strings this format needs.

supports_thinking: bool

Whether this template supports reasoning.

thinking_start_tag: String

Opening reasoning tag, e.g. "<think>". Empty when unsupported.

thinking_end_tags_json: String

JSON array of closing reasoning tags.

format: i32

common_chat_format discriminant, needed to parse output back.

Implementations

impl ChatParams

pub fn format_name(&self) -> Result<String, ChatError>

Human-readable name of this chat format, e.g. "Hermes 2 Pro".

Errors

Returns ChatError::Failed if llama.cpp cannot name the format.

pub fn sampler_triggers(&self) -> (Vec<String>, Vec<LlamaToken>)

Convert Self::grammar_triggers into the (patterns, tokens) pair LlamaSampler::grammar_lazy_patterns expects.

The four trigger kinds do not map onto the sampler one-for-one, and getting the translation wrong silently produces a grammar that never activates:

• word is a literal, so it is regex-escaped before becoming a pattern. A raw <tool_call> would otherwise be a character class.
• pattern passes through unchanged.
• pattern_full is anchored with ^/$ unless it already is.
• token becomes a trigger token rather than a pattern.

This mirrors common/sampling.cpp, so callers do not have to.

pub fn generation_prompt(&self) -> &str

The prefix the template already placed at the end of the prompt, e.g. "<|im_start|>assistant\n".

This is not part of the model’s output, but the grammar and the parser are both written as if it were — see Self::grammar_sampler.

pub fn grammar_sampler(&self, model: &LlamaModel) -> Option<LlamaSampler>

Build the grammar sampler this render needs, or None when unconstrained.

Prefer this over constructing the sampler yourself: it handles three things that are each silently wrong if missed.

• Lazy vs eager. A lazy grammar must be built with its triggers, or it never activates. One built eagerly from a lazy grammar constrains from token zero and blocks a thinking model’s reasoning prefix.
• Trigger translation. Literal, regex and token triggers map onto the sampler differently — see Self::sampler_triggers.
• Generation-prompt prefill. llama.cpp writes tool-call grammars to match generation_prompt + output, because that is what the parser later sees. The sampler only sees output, so without advancing the grammar past that prefix it forces the model to re-emit <|im_start|>assistant as generated text. Prefill applies only to non-lazy grammars — a lazy one has not started matching yet.

Panics

Panics if llama.cpp cannot parse the grammar it just produced, or if that grammar contains an interior NUL.

pub fn parse(&self, text: &str, is_partial: bool) -> Result<String, ChatError>

Parse model output back into an OpenAI-shaped message JSON object with role, content, reasoning_content and tool_calls.

This uses the parser the template produced, so it understands that model family’s tool-call syntax rather than scraping for a fixed marker.

Set is_partial while streaming: the parser then tolerates a truncated tail instead of rejecting the buffer.

Errors

Returns ChatError::Failed if the output cannot be parsed.

pub fn parse_with( &self, text: &str, is_partial: bool, parse_tool_calls: bool, reasoning_in_content: bool, ) -> Result<String, ChatError>

Self::parse with control over tool-call parsing and whether reasoning is left inline in content.

Errors

Returns ChatError::Failed if the output cannot be parsed.

Trait Implementations

impl Clone for ChatParams

fn clone(&self) -> ChatParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ChatParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.