Trait Benchmark

Source

pub trait Benchmark {
    type Input: Clone;
    type Output;

    // Required methods
    fn prepare(&self) -> Self::Input;
    fn execute(&self, input: Self::Input) -> Result<Self::Output, String>;
    fn name(&self) -> String;
    fn sync(&self);

    // Provided methods
    fn num_samples(&self) -> usize { ... }
    fn options(&self) -> Option<String> { ... }
    fn shapes(&self) -> Vec<Vec<usize>> { ... }
    fn work(&self) -> Option<Work> { ... }
    fn profile(&self, args: Self::Input) -> Result<ProfileDuration, String> { ... }
    fn profile_full(&self, args: Self::Input) -> Result<ProfileDuration, String> { ... }
    fn run(
        &self,
        timing_method: TimingMethod,
    ) -> Result<BenchmarkDurations, String> { ... }
}

Expand description

Benchmark trait.

Required Associated Types§

Source

Required Methods§

Source

fn prepare(&self) -> Self::Input

Prepare the benchmark, run anything that is essential for the benchmark, but shouldn’t count as included in the duration.

§Notes

This should not include warmup, the benchmark will be run at least one time without measuring the execution time.

Source

fn execute(&self, input: Self::Input) -> Result<Self::Output, String>

Execute the benchmark and returns the logical output of the task executed.

It is important to return the output since otherwise deadcode optimization might optimize away code that should be benchmarked.

Source

fn name(&self) -> String

Name of the benchmark, should be short and it should match the name defined in the crate Cargo.toml

Source

fn sync(&self)

Wait for computation to complete.

Provided Methods§

Source

fn num_samples(&self) -> usize

Number of samples per run required to have a statistical significance.

Source

fn options(&self) -> Option<String>

The options passed to the benchmark.

Source

fn shapes(&self) -> Vec<Vec<usize>>

Shapes dimensions

Source

fn work(&self) -> Option<Work>

The work one execution performs, for scoring the run against measured peak throughput. None when the benchmark has no such figure to report. Coarse by necessity: this crate cannot name a throughput key, so calculate_bounds (in cubecl-runtime) is what turns this single figure into per-resource bounds, and a caller wanting the achieved rate against each of those scores them at the client layer, which does have keys.

Source