Trait StreamingChat 

pub trait StreamingChat<M, R>: WasmCompatSend + WasmCompatSync
where
    M: CompletionModel + 'static,
    <M as CompletionModel>::StreamingResponse: WasmCompatSend,
    R: Clone + Unpin + GetTokenUsage,
{
    type Hook: PromptHook<M>;

    // Required method
    fn stream_chat<I, T>(
        &self,
        prompt: impl Into<Message> + WasmCompatSend,
        chat_history: I,
    ) -> StreamingPromptRequest<M, Self::Hook>
       where I: IntoIterator<Item = T> + WasmCompatSend,
             T: Into<Message>;
}

Trait for high-level streaming chat interface with conversation history.

This trait provides an interface for streaming chat completions with support for maintaining conversation history. Implementations can optionally support prompt hooks for observing and controlling the agent’s execution lifecycle.

Required Associated Types

type Hook: PromptHook<M>

The hook type used by this streaming chat implementation.

If your implementation does not need prompt hooks, use () as the hook type:

impl<M, R> StreamingChat<M, R> for MyType<M>
where
    M: CompletionModel + 'static,
    // ... other bounds ...
{
    type Hook = ();

    fn stream_chat(
        &self,
        prompt: impl Into<Message>,
        chat_history: Vec<Message>,
    ) -> StreamingPromptRequest<M, ()> {
        // ...
    }
}

Required Methods

fn stream_chat<I, T>( &self, prompt: impl Into<Message> + WasmCompatSend, chat_history: I, ) -> StreamingPromptRequest<M, Self::Hook>

where I: IntoIterator<Item = T> + WasmCompatSend, T: Into<Message>,

Stream a chat with history to the model.

The messages returned by the model can be accessed via FinalResponse::history()

You are responsible for managing history, a simple linear solution could look like:

 let mut history = vec![];

 loop {
     let prompt = "Create GPT-67, make no mistakes";
     let mut stream = agent.stream_chat(prompt, &history).await;

     while let Some(msg) = stream.next().await {
        match msg {
             Ok(MultiTurnStreamItem::FinalResponse(fin)) => {
                 history.extend_from_slice(fin.history().unwrap_or_default());
                 break;
            }
            Ok(_other) => { /* Do something with this chunk */ }
            Err(e) => return Err(e.into()),
        }
    }
}

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors

impl<M, P> StreamingChat<M, <M as CompletionModel>::StreamingResponse> for Agent<M, P>

where M: CompletionModel + 'static, M::StreamingResponse: GetTokenUsage, P: PromptHook<M> + 'static,

type Hook = P