Struct Benchmarks

pub struct Benchmarks<C, W, E>
where
    C: Clone + Display,
    W: Clone + Display,
    Error: From<E>,
{ /* private fields */ }

Run and analyze a benchmarks suite

• C - configuration
• W - workload unit
• E - error type

Implementations

impl<C, W, E> Benchmarks<C, W, E>

where C: Clone + Display, W: Clone + Display, Error: From<E>,

pub fn new(name: &str) -> Benchmarks<C, W, E>

Create a new Benchmarks

• name - the name of the benchmark session

pub fn run(&mut self) -> Result<(), Error>

Run all benchmarks

pub fn add( &mut self, name: &str, f: fn(stop_watch: &mut StopWatch, config: C, workload_point: W) -> Result<(), E>, config: C, work: Vec<W>, repeat: usize, ramp_up: usize, ) -> Result<(), Error>

Create and add a Benchmark

• name - the name of the benchmark series. The result will be accessible by the name as a key from the summary.
• f - the function that runs the benchmark.
• config - the configuration value for this benchmark series
• work - workload points vector for this benchmark series. Elements of this vector are passed to f in each iteration
• repeat - number of times the benchmark will be repeated
• ramp_up - number of times the benchmark will be performed before the measurement is taken

pub fn summary(&self) -> Summary

Produce Summary for all series

pub fn summary_as_json(&self) -> String

Produce Summary for all series as JSON string.

pub fn summary_as_csv( &self, with_headers: bool, with_config: bool, ) -> HashMap<String, Vec<String>>

Produce summary for each series as vector of CSV lines. The key for series data is the series name used in Self::add method, values are placed as the headers returned by [Self::csv_headers]

pub fn save_to_csv( &self, dir: PathBuf, with_headers: bool, with_config: bool, ) -> Result<(), Error>

Save each benchmark summary to its own CSV file

The name of each CSV file is the name of the benchmark.

• dir - directory to store results. If doesn’t exist - create it.
• with_headers - add column headers on the first line
• with_config - add the configuration string in the headers row

use std::path::PathBuf;
use benchmark_rs::benchmarks::Benchmarks;
let benchmarks = Benchmarks::<usize, usize, anyhow::Error>::new("example");
benchmarks.save_to_csv(PathBuf::from("./target/benchmarks"), true, true).expect("failed to save to csv");

pub fn save_to_json(&self, dir: PathBuf) -> Result<(), Error>

Save the summary to a json file.

The name of the JSON file is the name of the suite of benchmarks. If dir doesn’t exist - create it.

use std::path::PathBuf;
use anyhow::anyhow;
use benchmark_rs::benchmarks::Benchmarks;
let benchmarks = Benchmarks::<usize, usize, anyhow::Error>::new("example");
benchmarks.save_to_json(PathBuf::from("./target/benchmarks")).expect("failed to save to json");

pub fn configs(&self) -> HashMap<String, String>

Produce description of configurations for each series. The key for series config is the series name used in Self::add method.

pub fn name(&self) -> &String

The benchmarks suite name

pub fn analyze( &self, prev_result_string_opt: Option<String>, threshold: f64, ) -> Result<AnalysisResult, Error>

Compare the current result against a previous result.

• prev_result_string_opt - a JSON string of the Summary of previous run
• threshold - threshold used to determine equality.