cargo-criterion is a Plugin for Cargo which handles much of the heavy lifting for analyzing and reporting on Criterion-rs benchmarks.
Table of Contents
Features
- Charts: Uses gnuplot or plotters to generate detailed graphs of benchmark results
- Reports: In addition to the reports generated by Criterion.rs, cargo-criterion generates a historical report showing the performance of a function over time.
- Configurable: cargo-criterion's plot generation can be configured using a Criterion.toml file.
Quickstart
This assumes that you already have benchmarks which use Criterion-rs. If not, see the Criterion-rs Quickstart Guide. Note that Criterion.rs version 0.3.3 or later is recommended. Benchmarks which do not use Criterion.rs, or which use an earlier version, will run correctly but will not benefit from some of cargo-criterion's features.
First install cargo-criterion:
cargo install cargo-criterion --version 1.0.0
Then you can use it to run your Criterion-rs benchmarks:
cargo criterion
Goals
- cargo-criterion seeks to improve iteration time for Criterion-rs benchmarks. By moving functionality into a separate executable which can be installed once and reused, Criterion-rs can shrink - meaning less code to compile and link into the benchmarks themselves.
- Because cargo-criterion can oversee the whole benchmark process from beginning to end, it's better placed to deliver features that would be difficult to implement in Criterion-rs. These include:
- Machine-readable output
- Tracking benchmarked performance over time
Contributing
First, thank you for contributing.
One great way to contribute to cargo-criterion is to use it for your own benchmarking needs and report your experiences, file and comment on issues, etc.
Code or documentation improvements in the form of pull requests are also welcome. If you're not sure what to work on, try checking the Beginner label.
If your issues or pull requests have no response after a few days, feel free to ping me (@bheisler).
For more details, see the CONTRIBUTING.md file.
Compatibility Policy
cargo-criterion supports the last three stable minor releases of Rust. At time of writing, this means Rust 1.41 or later. Older versions may work, but are not tested or guaranteed.
Currently, the oldest version of Rust believed to work is 1.40. Future versions of cargo-criterion may break support for such old versions, and this will not be considered a breaking change. If you require cargo-criterion to work on old versions of Rust, you will need to stick to a specific patch version of cargo-criterion.
Maintenance
cargo-criterion was originally developed and is currently maintained by Brook Heisler (@bheisler).
License
cargo-criterion is dual licensed under the Apache 2.0 license and the MIT license.