Struct ChatEngine 

pub struct ChatEngine { /* private fields */ }

High-level chat engine that maintains conversation state.

Wraps an Engine with conversation history, context management, and automatic chat template formatting.

Implementations

impl ChatEngine

pub fn new(engine: Engine, system_prompt: Option<String>) -> Self

Create a new chat engine from a loaded Engine.

pub fn engine(&self) -> &Engine

Get a reference to the underlying engine.

pub fn system_prompt(&self) -> &str

Get the current system prompt.

pub fn context_len(&self) -> usize

Get the number of tokens in the current conversation context.

pub fn chat(&mut self, message: &str) -> Result<String, EngineError>

Send a message and get the full response.

pub fn chat_with_prefix( &mut self, message: &str, prefix: &str, ) -> Result<String, EngineError>

Send a message and get the full response, with a prefix injected as the start of the assistant’s reply. The prefix tokens are prefilled alongside the prompt tokens so the model continues from the prefix text. The prefix is prepended to the returned string.

This is useful for forcing the model to start with a particular token sequence (e.g. { for JSON output).

pub fn chat_streaming( &mut self, message: &str, ) -> Result<ChatStream<'_>, EngineError>

Send a message and stream the response token by token.

Returns an iterator of Result<String, EngineError> where each item is a decoded token chunk.

pub fn clear_history(&mut self)

Clear conversation history and reset context.