Trait Inference 

pub trait Inference:
    RequestResponse<PredictRequest, PredictResponse>
    + Send
    + Sync {
    // Required methods
    fn predict<'life0, 'async_trait>(
        &'life0 self,
        request: PredictRequest,
    ) -> Pin<Box<dyn Future<Output = Result<PredictResponse, InferenceError>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait;
    fn descriptor(&self) -> InferenceDescriptor;
}

Inference is the model-serving runtime trait. Adapters cover text generation (vLLM, TGI), classification/regression (Triton KServe v2), embeddings (BentoML, custom), image/audio inference, and arbitrary tensor protocols. NOT chat completion — that lives in rskit-llm.

This trait extends rskit_provider::RequestResponse<PredictRequest, PredictResponse> so inference adapters natively expose the canonical request/response shape.

Required Methods

fn predict<'life0, 'async_trait>( &'life0 self, request: PredictRequest, ) -> Pin<Box<dyn Future<Output = Result<PredictResponse, InferenceError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Execute one prediction request against the serving runtime.

fn descriptor(&self) -> InferenceDescriptor

Return the adapter descriptor and executable permission envelope.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl Inference for Echo