Trait RuntimeModel 

pub trait RuntimeModel: Send + Sync {
    // Required method
    fn predict<'life0, 'async_trait>(
        &'life0 self,
        request: PredictRequest,
    ) -> Pin<Box<dyn Future<Output = Result<RuntimeModelPrediction, RuntimeError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;

    // Provided methods
    fn predict_group<'life0, 'async_trait>(
        &'life0 self,
        requests: Vec<PredictRequest>,
    ) -> Pin<Box<dyn Future<Output = Vec<Result<RuntimeModelPrediction, RuntimeError>>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait { ... }
    fn fuses_predicts(&self) -> bool { ... }
    fn wants_history(&self) -> bool { ... }
    fn reset_adapter<'life0, 'async_trait>(
        &'life0 self,
        _request: ResetAdapterRequest,
    ) -> Pin<Box<dyn Future<Output = Result<(), RuntimeError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait { ... }
    fn release_adapter<'life0, 'async_trait>(
        &'life0 self,
        _request: ReleaseAdapterRequest,
        _timeout: Duration,
    ) -> Pin<Box<dyn Future<Output = Result<(), String>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait { ... }
}

The model side of a route: predict, plus the per-episode adapter-state lifecycle (evict on episode end, release at session end).

Required Methods

fn predict<'life0, 'async_trait>( &'life0 self, request: PredictRequest, ) -> Pin<Box<dyn Future<Output = Result<RuntimeModelPrediction, RuntimeError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Predict the ordered action frames for the batched observation (frame 0 is this step; any further frames replay open-loop before the next call).

Provided Methods

fn predict_group<'life0, 'async_trait>( &'life0 self, requests: Vec<PredictRequest>, ) -> Pin<Box<dyn Future<Output = Vec<Result<RuntimeModelPrediction, RuntimeError>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Predict for several groups at once, one result per request in order. The default runs the predicts concurrently, which is enough for a transport that coalesces concurrent predicts itself; an implementation that can fuse them into one forward pass overrides this and says so with fuses_predicts.

fn fuses_predicts(&self) -> bool

Whether predict_group fuses its requests into one forward pass. The driver then keeps one grouped predict in flight and batches the groups that become ready behind it into the next. Default false: the driver puts one predict per group in flight, together, and applies each as it lands, so a lane never waits behind the slowest predict of a batch.

fn wants_history(&self) -> bool

Whether the model’s route negotiated observation history at resolve (RouteNeeds::history). The driver then carries every env step it executed from a replayed chunk, without predicting, as a PredictRequest.history row on the next predict, and stamps requests and rows with the group’s step counter. An async prefetch lead keeps working: the prefetched request carries the rows buffered so far, with the observation it predicts from as its own rather than a row. Default false: no rows are buffered.

fn reset_adapter<'life0, 'async_trait>( &'life0 self, _request: ResetAdapterRequest, ) -> Pin<Box<dyn Future<Output = Result<(), RuntimeError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Evict the model’s per-episode adapter state (frame-stack buffers) for the ended episodes. Best-effort GC, not a correctness gate: because episode ids never repeat (UUIDv7), a dropped ResetAdapter only leaks memory and can never alias a new episode. Default no-op for impls that hold no per-episode state.

fn release_adapter<'life0, 'async_trait>( &'life0 self, _request: ReleaseAdapterRequest, _timeout: Duration, ) -> Pin<Box<dyn Future<Output = Result<(), String>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Release the model endpoint within timeout. Default no-op.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors