Struct BenchLimits 

pub struct BenchLimits {
    pub warmup_loops: usize,
    pub warmup_time: Duration,
    pub max_loops: usize,
    pub max_time: Duration,
}

Fields

warmup_loops: usize

warmup_time: Duration

max_loops: usize

max_time: Duration

Implementations

impl BenchLimits

pub fn warmup( &self, runnable: &Arc<dyn Runnable>, inputs: &RunTensors, ) -> TractResult<()>

pub fn bench( &self, runnable: &Arc<dyn Runnable>, inputs: &RunTensors, ) -> TractResult<(usize, Duration)>

impl BenchLimits

pub fn bench_streams( &self, runnable: &Arc<dyn Runnable>, inputs: &RunTensors, streams: usize, ) -> TractResult<StreamsBench>

One saturating loop per stream: streams threads, a state each, every one of them feeding turns as fast as it can until the limits run out. The limits are per stream, the wall clock is shared.

This is what an autobatch runtime is compared against: the same load offered to the plain runtime is streams independent states running side by side.

pub fn bench_streams_paced( &self, runnable: &Arc<dyn Runnable>, inputs: &RunTensors, streams: usize, pacing: &Pacing, ) -> TractResult<PacedBench>

One paced loop per stream: streams threads, a state each, every one of them owing a turn every period of the wall clock, the way a live stream does. Turn n of a stream is due at a fixed offset from its first, so a turn which runs long does not push the ones behind it back – the stream catches up or falls further behind, and added is what says which.

Streams are phased evenly over one period. Each session runs one turn before the clock starts, so the buffers a first turn allocates are not charged to the load.

Under churn a stream is a seat in a steady population rather than one session: it holds a session for a draw around hold, gives it up, and admits another in its place, so the lanes turn over while the load does not.

Trait Implementations

impl Default for BenchLimits

fn default() -> Self

Returns the “default value” for a type.