Struct BatchConfig 

pub struct BatchConfig {
    pub max_batch: usize,
    pub latency_deadline_us: u64,
}

Batch inference configuration.

Fields

max_batch: usize

Maximum batch size for pipelined inference. Must be ≤ model’s max dynamic batch axis.

latency_deadline_us: u64

Collect at most this many frames before dispatching a batch, even if max_batch is not reached (latency bound).

Trait Implementations

impl Clone for BatchConfig

fn clone(&self) -> BatchConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for BatchConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for BatchConfig

fn default() -> Self

Returns the “default value” for a type.