Trait ChatStreamExt 

pub trait ChatStreamExt: Stream<Item = Result<StreamEvent>> + Send {
    // Required method
    fn collect_response(
        self,
    ) -> impl Future<Output = Result<ChatResponse>> + Send;

    // Provided methods
    fn next_event(
        &mut self,
    ) -> impl Future<Output = Option<Result<StreamEvent>>> + Send
       where Self: Unpin { ... }
    fn with_collector(self) -> WithCollector<Self>
       where Self: Sized { ... }
}

Extension methods for chat streams.

Required Methods

fn collect_response(self) -> impl Future<Output = Result<ChatResponse>> + Send

Collect a complete response.

Errors

Fails on the first stream error, on protocol violations, and with a retryable StreamErrorKind::Incomplete error when the stream ends early.

Provided Methods

fn next_event( &mut self, ) -> impl Future<Output = Option<Result<StreamEvent>>> + Send

where Self: Unpin,

The next event, or None at the end of the stream.

The same as StreamExt::next from futures-util, without importing a stream extension trait: a loop over a chat stream needs nothing beyond allama::prelude, and a tokio_stream or futures import of your own never makes the call ambiguous.

use allama::prelude::*;
use allama::mock::MockProvider;

let provider = MockProvider::with_text("Hello");
let mut stream = provider.chat_stream(&ChatRequest::new("m").user("hi")).await?;
while let Some(event) = stream.next_event().await {
    if let StreamEvent::TextDelta { text, .. } = event? {
        print!("{text}");
    }
}

fn with_collector(self) -> WithCollector<Self>

where Self: Sized,

Collect the response while streaming; each event comes with the block it completes.

Use it to show events as they arrive and still get the ChatResponse at the end; use collect_response when only the response matters.

Each item is a CollectedEvent: the event and, for a BlockStop, the block it completed, so a caller can act on a complete tool call or text block while the rest of the response is still streaming. Read the response with WithCollector::finish. WithCollector::next_event reads the next item without a stream extension trait in scope:

use allama::prelude::*;
use allama::mock::MockProvider;

let provider = MockProvider::with_text("Hello");
let mut stream = provider
    .chat_stream(&ChatRequest::new("m").user("hi"))
    .await?
    .with_collector();
while let Some(item) = stream.next_event().await {
    let item = item?;
    if let StreamEvent::TextDelta { text, .. } = &item.event {
        print!("{text}");
    }
    if let Some(ContentBlock::ToolCall(call)) = &item.completed {
        println!("call {}", call.name);
    }
}
let response = stream.finish()?;

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl<S: Stream<Item = Result<StreamEvent>> + Send> ChatStreamExt for S