Trait ComputeModel 

pub trait ComputeModel<InP: Payload, OutP: Payload> {
    // Required methods
    fn init(&mut self) -> Result<(), InferenceError>;
    fn infer_one(
        &mut self,
        inp: &InP,
        out: &mut OutP,
    ) -> Result<(), InferenceError>;
    fn drain(&mut self) -> Result<(), InferenceError>;
    fn reset(&mut self) -> Result<(), InferenceError>;
    fn metadata(&self) -> ModelMetadata;

    // Provided method
    fn infer_batch(
        &mut self,
        inps: Batch<'_, InP>,
        outs: &mut [OutP],
    ) -> Result<(), InferenceError> { ... }
}

A loaded model that can perform inference.

Required Methods

fn init(&mut self) -> Result<(), InferenceError>

Prepare internal state (allocate work buffers, compile kernels, etc.).

fn infer_one(&mut self, inp: &InP, out: &mut OutP) -> Result<(), InferenceError>

Single-item inference (1×1).

fn drain(&mut self) -> Result<(), InferenceError>

Ensure outstanding device work is complete (if any).

fn reset(&mut self) -> Result<(), InferenceError>

Reset internal state to a known baseline (drop caches, etc.).

fn metadata(&self) -> ModelMetadata

Return model metadata (I/O placement preferences, limits).

Provided Methods

fn infer_batch( &mut self, inps: Batch<'_, InP>, outs: &mut [OutP], ) -> Result<(), InferenceError>

Optional: batched inference. Default loops infer_one.

Implementors

impl ComputeModel<Tensor<u32, TEST_TENSOR_ELEMENT_COUNT, 2>, Tensor<u32, TEST_TENSOR_ELEMENT_COUNT, 2>> for TestTensorModel