Trait ModelRunner 

pub trait ModelRunner: Send + Sync {
    // Required methods
    fn card(&self) -> ModelCard;
    fn infer(&self, cx: &mut Cx, request: ModelRequest) -> Result<ModelResponse>;

    // Provided methods
    fn infer_stream(
        &self,
        cx: &mut Cx,
        request: ModelRequest,
        sink: &mut dyn ModelEventSink,
    ) -> Result<ModelResponse> { ... }
    fn bid(&self, _request: &ModelRequest) -> Result<ModelBid> { ... }
}

Executable contract for a model backend or model-like runtime surface.

Required Methods

fn card(&self) -> ModelCard

Returns metadata describing the runner’s advertised model surface.

fn infer(&self, cx: &mut Cx, request: ModelRequest) -> Result<ModelResponse>

Executes one non-streaming inference request.

Provided Methods

fn infer_stream( &self, cx: &mut Cx, request: ModelRequest, sink: &mut dyn ModelEventSink, ) -> Result<ModelResponse>

Executes one streaming inference request.

The default implementation falls back to Self::infer and emits a synthesized final event.

fn bid(&self, _request: &ModelRequest) -> Result<ModelBid>

Returns an availability and score hint for market-style routing.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors