pub struct ChatSession { /* private fields */ }Expand description
A chat-oriented Session with tool registration and turn management.
Dereferences to Session, so all base methods
(process_request,
process_streaming_request, …) are
available directly.
Implementations§
Source§impl ChatSession
impl ChatSession
Sourcepub async fn new(model: &Model) -> Result<ChatSession, FoundryLocalError>
pub async fn new(model: &Model) -> Result<ChatSession, FoundryLocalError>
Open a chat session on a loaded model.
Returns FoundryLocalError::Validation if the model’s task is not
"chat-completion" or "vision-language-chat".
Sourcepub async fn add_tool_definition(
&self,
definition: ToolDefinition,
) -> Result<(), FoundryLocalError>
pub async fn add_tool_definition( &self, definition: ToolDefinition, ) -> Result<(), FoundryLocalError>
Register a ToolDefinition for the lifetime of the session.
Sourcepub async fn remove_tool_definition(
&self,
name: impl Into<String>,
) -> Result<bool, FoundryLocalError>
pub async fn remove_tool_definition( &self, name: impl Into<String>, ) -> Result<bool, FoundryLocalError>
Remove a previously-registered tool by name. Returns whether one was removed.
Sourcepub fn turn_count(&self) -> usize
pub fn turn_count(&self) -> usize
The number of completed conversation turns.
Sourcepub async fn undo_turns(&self, count: usize) -> Result<(), FoundryLocalError>
pub async fn undo_turns(&self, count: usize) -> Result<(), FoundryLocalError>
Rewind the last count turns, dropping their messages and replies.
Sourcepub fn into_session(self) -> Session
pub fn into_session(self) -> Session
Consume this handle, yielding the underlying base Session.
Methods from Deref<Target = Session>§
Sourcepub async fn process_request(
&self,
request: Request,
) -> Result<Response, FoundryLocalError>
pub async fn process_request( &self, request: Request, ) -> Result<Response, FoundryLocalError>
Process a request and return the complete response.
Runs on a blocking worker thread; the returned future resolves when generation finishes.
§Cancellation
The returned future is cancel-on-drop: if it is dropped before it
resolves — for example via tokio::time::timeout, tokio::select!, or
aborting the task — the in-flight native request is cancelled and
generation stops as soon as possible rather than running to completion on
the detached worker. This mirrors the drop-cancels-generation behaviour of
process_streaming_request.
Sourcepub fn process_streaming_request(&self, request: Request) -> ItemStream
pub fn process_streaming_request(&self, request: Request) -> ItemStream
Process a request, streaming each output Item as it is
produced.
The returned ItemStream yields items until generation completes;
dropping it cancels generation.
Sourcepub async fn set_options(
&self,
options: RequestOptions,
) -> Result<(), FoundryLocalError>
pub async fn set_options( &self, options: RequestOptions, ) -> Result<(), FoundryLocalError>
Apply session-scoped options that persist across subsequent requests.
Sourcepub fn create_input_queue(&self) -> Result<ItemQueue, FoundryLocalError>
pub fn create_input_queue(&self) -> Result<ItemQueue, FoundryLocalError>
Create an ItemQueue for streaming incremental input into a request.
Attach the returned queue to a Request via
Request::with_input_queue, then push items as they become available.
Trait Implementations§
Source§impl Clone for ChatSession
impl Clone for ChatSession
Source§fn clone(&self) -> ChatSession
fn clone(&self) -> ChatSession
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read more