Enum ChatTemplate 

pub enum ChatTemplate {
    UserAssistant,
    ChatML,
    Llama2,
    None,
}

Detected chat template format from the GGUF metadata.

Variants

UserAssistant

<|user|>\n...<|assistant|>\n format

ChatML

ChatML: <|im_start|>user\n...<|im_end|>\n<|im_start|>assistant\n

Llama2

Llama-2: [INST] ... [/INST]

None

No template detected, use raw text.

Implementations

impl ChatTemplate

pub fn from_tokenizer_config(path: &Path) -> Option<Self>

Detect chat template from a HuggingFace tokenizer_config.json file.

Reads the chat_template field (a Jinja2 string) and pattern-matches against known token markers. Returns None on any IO or parse error.

pub fn detect_from_model_type(model_type: Option<&str>) -> Self

Detect chat template from model type string (for ONNX models without GGUF metadata).

pub fn detect(gguf: &GgufFile) -> Self

Detect chat template from GGUF metadata.

pub fn wrap_prompt(&self, prompt: &str) -> String

Wrap a raw prompt in the appropriate chat format.

pub fn format_first_turn( &self, system_prompt: &str, user_message: &str, ) -> String

Format a chat message with system prompt for the first turn.

pub fn format_continuation(&self, user_message: &str) -> String

Format a continuation turn (not the first message).

pub fn stop_patterns(&self) -> &[&str]

Patterns that indicate the model is trying to generate a new user turn (i.e., the response is complete).

Trait Implementations

impl Clone for ChatTemplate

fn clone(&self) -> ChatTemplate

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ChatTemplate

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for ChatTemplate

fn eq(&self, other: &ChatTemplate) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for ChatTemplate