Struct LLMSession 

pub struct LLMSession { /* private fields */ }

A session that manages communication with an LLM

Implementations

impl LLMSession

pub fn new( config: LLMSessionConfig, from_llm: Sender<FromLLMPayload>, cancel_token: CancellationToken, channel_size: usize, ) -> Result<Self, LlmError>

Creates a new LLM session

Arguments

• config - Session configuration
• from_llm - Sender for outgoing responses
• cancel_token - Token for session cancellation
• channel_size - Buffer size for the session’s input channel

Errors

Returns an error if the LLM client fails to initialize (e.g., TLS setup failure)

pub fn id(&self) -> i64

Returns the session ID

pub fn created_at(&self) -> Instant

Returns when the session was created

pub fn model(&self) -> &str

Returns the model for this session

pub fn set_max_tokens(&self, max_tokens: i64)

Sets the default maximum tokens for the session

pub fn max_tokens(&self) -> i64

Returns the current max tokens setting

pub fn context_limit(&self) -> i32

Returns the context limit for this session’s model

pub async fn set_system_prompt(&self, prompt: String)

Sets the default system prompt for the session

pub async fn clear_system_prompt(&self)

Clears the default system prompt

pub async fn system_prompt(&self) -> Option<String>

Returns the current system prompt

pub async fn set_tools(&self, tools: Vec<LLMTool>)

Sets the tool definitions for this session. Tools will be included in all subsequent LLM API calls.

pub async fn clear_tools(&self)

Clears all tool definitions for this session.

pub async fn tools(&self) -> Vec<LLMTool>

Returns a copy of the current tool definitions.

pub async fn clear_conversation(&self)

Clears the conversation history and compact summaries.

pub async fn force_compact(&self) -> CompactResult

Forces compaction to run immediately, regardless of threshold. Returns a CompactResult with details about what happened.

pub async fn send(&self, msg: ToLLMPayload) -> bool

Sends a message to the LLM session for processing. Returns false if the session is shutdown or the channel is closed.

pub async fn interrupt(&self)

Interrupts the currently executing LLM request. This cancels any in-flight request and removes all messages from the current turn from conversation history. Does not shutdown the session.

pub fn shutdown(&self)

Gracefully shuts down the session. After calling this, the session will not accept new messages.

pub fn is_shutdown(&self) -> bool

Returns true if the session has been shutdown

pub async fn start(&self)

Starts the session’s main processing loop. This method processes requests from the ToLLM channel until shutdown. Should be spawned as a tokio task.