Trait AsyncEngineStream 

pub trait AsyncEngineStream<T: Data>:
    Stream<Item = T>
    + AsyncEngineContextProvider
    + Send { }

A streaming asynchronous engine operation.

This trait combines Stream semantics with context provider capabilities, representing a continuous async operation that produces multiple messages over time. The canonical boxed form is EngineStream<T> (= crate::pipeline::ManyOut<T>), the stream of response chunks an engine emits; ResponseStream is the canonical concrete implementor.

Trait Implementations

impl<T, U> AsyncEngine<Context<AddressedRequest<T>>, Pin<Box<dyn AsyncEngineStream<U, Item = U>>>, Error> for AddressedPushRouter

where T: Data + Serialize, U: Data + for<'de> Deserialize<'de> + MaybeError,

fn generate<'life0, 'async_trait>( &'life0 self, request: SingleIn<AddressedRequest<T>>, ) -> Pin<Box<dyn Future<Output = Result<ManyOut<U>, Error>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Generate a stream of completion responses.

impl<T, U> AsyncEngine<Context<RequestStream<T>>, Pin<Box<dyn AsyncEngineStream<U, Item = U>>>, Error> for PushRouter<T, U>

where T: Data + Serialize, U: Data + for<'de> Deserialize<'de> + MaybeError,

Bidirectional AsyncEngine impl for streaming-input workloads (e.g. the OpenAI Realtime API). Reserves a sticky worker up front — before any inbound frame is observed — and binds the whole input stream to that worker. KV and Direct modes inherit the same bail! invariants as the unary impl.

Reserve-before-observe rationale. The router-mode strategies (RoundRobin, Random, PowerOfTwoChoices, LeastLoaded, DeviceAwareWeighted) don’t depend on frame contents, so selection runs immediately and connection setup proceeds in parallel with the client producing its first frame. A client that connects but never sends one still releases the slot via the response-stream-drop path; the dispatch-side cancel_both cleanup covers the early-bail case.

fn generate<'life0, 'async_trait>( &'life0 self, input: ManyIn<T>, ) -> Pin<Box<dyn Future<Output = Result<ManyOut<U>, Error>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Generate a stream of completion responses.

impl<T, U> AsyncEngine<Context<T>, Pin<Box<dyn AsyncEngineStream<U, Item = U>>>, Error> for PushRouter<T, U>

where T: Data + Serialize, U: Data + for<'de> Deserialize<'de> + MaybeError,

fn generate<'life0, 'async_trait>( &'life0 self, request: SingleIn<T>, ) -> Pin<Box<dyn Future<Output = Result<ManyOut<U>, Error>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Generate a stream of completion responses.

impl<T, U> AsyncEngine<Context<T>, Pin<Box<dyn AsyncEngineStream<U, Item = U>>>, Error> for Egress<SingleIn<T>, ManyOut<U>>

where T: Data + Serialize, U: for<'de> Deserialize<'de> + Data,

fn generate<'life0, 'async_trait>( &'life0 self, request: SingleIn<T>, ) -> Pin<Box<dyn Future<Output = Result<ManyOut<U>, Error>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Generate a stream of completion responses.

impl<T: Data> AsyncEngineContextProvider for Pin<Box<dyn AsyncEngineStream<T>>>

fn context(&self) -> Arc<dyn AsyncEngineContext>

impl<T: Data> From<Pin<Box<dyn AsyncEngineStream<T, Item = T>>>> for DataStream<T>

fn from(stream: EngineStream<T>) -> Self

Converts to this type from the input type.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl<R: Data> AsyncEngineStream<R> for ResponseStream<R>