Skip to main content

StreamingChat

rig_core::streaming

Trait StreamingChat 

Source 
pub trait StreamingChat<M, R>: WasmCompatSend + WasmCompatSync
where
    M: CompletionModel + 'static,
    <M as CompletionModel>::StreamingResponse: WasmCompatSend,
    R: Clone + Unpin + GetTokenUsage,
{
    // Required method
    fn stream_chat<I, T>(
        &self,
        prompt: impl Into<Message> + WasmCompatSend,
        chat_history: I,
    ) -> StreamingPromptRequest<M>
       where I: IntoIterator<Item = T> + WasmCompatSend,
             T: Into<Message>;
}
Expand description

Trait for high-level streaming chat interface with conversation history.

This trait provides an interface for streaming chat completions with support for maintaining conversation history. Implementations can optionally support prompt hooks for observing and controlling the agent’s execution lifecycle.

Required Methods§

Source

fn stream_chat<I, T>( &self, prompt: impl Into<Message> + WasmCompatSend, chat_history: I, ) -> StreamingPromptRequest<M>
where I: IntoIterator<Item = T> + WasmCompatSend, T: Into<Message>,

Stream a chat with history to the model.

The messages returned by the model can be accessed via PromptResponse::messages()

You are responsible for managing history, a simple linear solution could look like:

ⓘ
 let mut history = vec![];

 loop {
     let prompt = "Create GPT-67, make no mistakes";
     let mut stream = agent.stream_chat(prompt, &history).await;

     while let Some(msg) = stream.next().await {
        match msg {
             Ok(MultiTurnStreamItem::FinalResponse(fin)) => {
                 history.extend_from_slice(fin.messages().unwrap_or_default());
                 break;
            }
            Ok(_other) => { /* Do something with this chunk */ }
            Err(e) => return Err(e.into()),
        }
    }
}

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§