Trait AsyncEngine 

pub trait AsyncEngine<Req: Send + 'static, Resp: AsyncEngineContextProvider, E: Data>: Send + Sync {
    // Required method
    fn generate<'life0, 'async_trait>(
        &'life0 self,
        request: Req,
    ) -> Pin<Box<dyn Future<Output = Result<Resp, E>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;
}

Engine is a trait that defines the interface for a streaming engine. The synchronous Engine version is does not need to be awaited.

This is the core trait for all async engine implementations. It provides:

• Generic type parameters for request, response, and error types
• Async generation capabilities with proper error handling
• Thread-safe design with Send + Sync bounds

Type Parameters

• Req: The request type — required to be Send + 'static. The Sync bound was removed from Req for convenience: forcing Sync on Req propagates a + Sync constraint onto every type that flows in (in particular, every input-side trait-object alias), and no existing implementation of AsyncEngine relies on the Sync nature of the request. Revisit if a future implementation genuinely needs shared-reference access to a request value across threads.
• Resp: The response type that implements AsyncEngineContextProvider
• E: The error type that implements Data

Implementation Notes

Implementations should ensure proper error handling and resource management. The generate method should be cancellable via the response’s context provider.

Required Methods

fn generate<'life0, 'async_trait>( &'life0 self, request: Req, ) -> Pin<Box<dyn Future<Output = Result<Resp, E>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Generate a stream of completion responses.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl<In: PipelineIO + Sync, Out: PipelineIO> AsyncEngine<In, Out, Error> for SegmentSource<In, Out>

impl<In: PipelineIO + Sync, Out: PipelineIO> AsyncEngine<In, Out, Error> for ServiceFrontend<In, Out>

impl<T, U> AsyncEngine<Context<AddressedRequest<T>>, Pin<Box<dyn AsyncEngineStream<U, Item = U>>>, Error> for AddressedPushRouter

where T: Data + Serialize, U: Data + for<'de> Deserialize<'de> + MaybeError,

impl<T, U> AsyncEngine<Context<T>, Pin<Box<dyn AsyncEngineStream<U, Item = U>>>, Error> for Egress<SingleIn<T>, ManyOut<U>>

where T: Data + Serialize, U: for<'de> Deserialize<'de> + Data,

impl<T, U> AsyncEngine<Context<T>, Pin<Box<dyn AsyncEngineStream<U, Item = U>>>, Error> for PushRouter<T, U>

where T: Data + Serialize, U: Data + for<'de> Deserialize<'de> + MaybeError,

impl<T, U> AsyncEngine<Pin<Box<dyn AsyncEngineStream<T, Item = T>>>, Pin<Box<dyn AsyncEngineStream<U, Item = U>>>, Error> for PushRouter<T, U>

where T: Data + Serialize, U: Data + for<'de> Deserialize<'de> + MaybeError,

Bidirectional AsyncEngine impl for streaming-input workloads (e.g. the OpenAI Realtime API). Selects a sticky instance on the first inbound frame and binds the whole input stream to that worker. Required so engines of shape BidirectionalStreamingEngine<T, U> can be stored as a PushRouter in WorkerSet.

Remote per-frame dispatch over AddressedPushRouter / PushWorkHandler is not yet implemented; this impl currently bails after selecting the worker. KV and Direct modes inherit the same bail! invariants as the unary impl.

impl<UpIn, UpOut, DownIn, DownOut> AsyncEngine<UpIn, UpOut, Error> for PipelineOperator<UpIn, UpOut, DownIn, DownOut>

where UpIn: PipelineIO + Sync, DownIn: PipelineIO + Sync, DownOut: PipelineIO, UpOut: PipelineIO,

A PipelineOperator is an AsyncEngine for the upstream AsyncEngine<UpIn, UpOut, Error>.