Struct Criterion

Source

pub struct Criterion<M: Measurement = WallTime> { /* private fields */ }

Expand description

The benchmark manager

Criterion lets you configure and execute benchmarks

Each benchmark consists of four phases:

Warm-up: The routine is repeatedly executed, to let the CPU/OS/JIT/interpreter adapt to the new load
Measurement: The routine is repeatedly executed, and timing information is collected into a sample
Analysis: The sample is analyzed and distilled into meaningful statistics that get reported to stdout, stored in files, and plotted
Comparison: The current sample is compared with the sample obtained in the previous benchmark.

Implementations§

Source §

impl<M: Measurement> Criterion<M>

Source

pub fn with_measurement<M2: Measurement>(self, m: M2) -> Criterion<M2>

Changes the measurement for the benchmarks run with this runner. See the Measurement trait for more details

Source

pub fn with_location(self, file: &str, module_path: &str) -> Criterion<M>

Configure file and module paths for use with codspeed.

Source

pub fn sample_size(self, n: usize) -> Criterion<M>

Changes the default size of the sample for benchmarks run with this runner.

A bigger sample should yield more accurate results if paired with a sufficiently large measurement time.

Sample size must be at least 10.

§Panics

Panics if n < 10

Source

pub fn warm_up_time(self, dur: Duration) -> Criterion<M>

Changes the default warm up time for benchmarks run with this runner.

§Panics

Panics if the input duration is zero

Source

pub fn measurement_time(self, dur: Duration) -> Criterion<M>

With a longer time, the measurement will become more resilient to transitory peak loads caused by external programs

Note: If the measurement time is too “low”, Criterion will automatically increase it

§Panics

Panics if the input duration in zero Changes the default measurement time for benchmarks run with this runner.

Source

pub fn nresamples(self, n: usize) -> Criterion<M>

Changes the default number of resamples for benchmarks run with this runner.

Number of resamples to use for the bootstrap

A larger number of resamples reduces the random sampling errors, which are inherent to the bootstrap method, but also increases the analysis time

§Panics

Panics if the number of resamples is set to zero

Source

pub fn noise_threshold(self, threshold: f64) -> Criterion<M>

Changes the default noise threshold for benchmarks run with this runner. The noise threshold is used to filter out small changes in performance, even if they are statistically significant. Sometimes benchmarking the same code twice will result in small but statistically significant differences solely because of noise. This provides a way to filter out some of these false positives at the cost of making it harder to detect small changes to the true performance of the benchmark.

The default is 0.01, meaning that changes smaller than 1% will be ignored.

§Panics

Panics if the threshold is set to a negative value

Source

pub fn confidence_level(self, cl: f64) -> Criterion<M>

Changes the default confidence level for benchmarks run with this runner. The confidence level is the desired probability that the true runtime lies within the estimated confidence interval. The default is 0.95, meaning that the confidence interval should capture the true value 95% of the time.

§Panics

Panics if the confidence level is set to a value outside the (0, 1) range

Source

pub fn significance_level(self, sl: f64) -> Criterion<M>

Changes the default significance level for benchmarks run with this runner. This is used to perform a hypothesis test to see if the measurements from this run are different from the measured performance of the last run. The significance level is the desired probability that two measurements of identical code will be considered ‘different’ due to noise in the measurements. The default value is 0.05, meaning that approximately 5% of identical benchmarks will register as different due to noise.

This presents a trade-off. By setting the significance level closer to 0.0, you can increase the statistical robustness against noise, but it also weakens Criterion.rs’ ability to detect small but real changes in the performance. By setting the significance level closer to 1.0, Criterion.rs will be more able to detect small true changes, but will also report more spurious differences.

§Panics

Panics if the significance level is set to a value outside the (0, 1) range

Source §

impl<M> Criterion<M>
where M: Measurement + 'static,

Source

pub fn bench_function<F>(&mut self, desc: &str, f: F) -> &mut Criterion<M>
where F: FnMut(&mut Bencher<'_, M>),

Benchmarks a function.

§Example

use wasm_bindgen_test::{Criterion, wasm_bindgen_bench};

#[wasm_bindgen_bench]
fn bench(c: &mut Criterion) {
    // Setup (construct data, allocate memory, etc)
    c.bench_function(
        "bench desc",
        |b| b.iter(|| {
            // Code to benchmark goes here
        }),
    );
}

Source

pub async fn bench_async_function<F>( &mut self, desc: &str, f: F, ) -> &mut Criterion<M>
where for<'b> F: FnMut(&'b mut Bencher<'_, M>) -> Pin<Box<dyn Future<Output = ()> + 'b>>,

Benchmarks a future.

§Example

use wasm_bindgen_test::{Criterion, wasm_bindgen_bench};

#[wasm_bindgen_bench]
async fn bench(c: &mut Criterion) {
    // Setup (construct data, allocate memory, etc)
    c.bench_async_function(
        "bench desc",
        |b| {
            Box::pin(
                b.iter_future(|| async {
                    // Code to benchmark goes here
                })
            )
        }
    ).await;
}