Struct BatchProcessor 

pub struct BatchProcessor<M: Model> { /* private fields */ }

Batch processor for efficient model serving

Implementations

impl<M: Model> BatchProcessor<M>

pub fn new(model: M, config: BatchConfig) -> Self

Creates a new batch processor

pub fn infer(&self, input: RasterBuffer) -> Result<RasterBuffer>

Submits a request for batch inference

When dynamic batching is enabled, requests are collected and processed together to improve throughput. For single requests, use DynamicBatchProcessor for more advanced batching with priority queuing and adaptive sizing.

Errors

Returns an error if inference fails

pub fn infer_batch( &self, inputs: Vec<RasterBuffer>, ) -> Result<Vec<RasterBuffer>>

Processes a batch of inputs

Errors

Returns an error if inference fails

pub fn infer_batch_with_progress( &self, inputs: Vec<RasterBuffer>, show_progress: bool, ) -> Result<Vec<RasterBuffer>>

Processes a batch of inputs with optional progress tracking

Arguments

• inputs - Input raster buffers to process
• show_progress - Whether to display a progress bar

Errors

Returns an error if inference fails

pub fn stats(&self) -> BatchStats

Returns the batch statistics

pub fn reset_stats(&self)

Resets the statistics