# criterion-markdown
`criterion-markdown` reads Criterion benchmark JSON output from `target/criterion` and renders a markdown summary table.
## What It Does
- Walks a Criterion result directory and discovers benchmark runs.
- Reads `benchmark.json` and `estimates.json` from the latest run and selected baseline.
- Produces grouped markdown tables with human-readable timings and change indicators.
- Computes improvements and regressions at render time, so any saved Criterion baseline can be selected.
## Usage
Use `Renderer` to select datasets, filter benchmarks, and configure the output:
```rust
use criterion_markdown::{ChangeThresholds, Renderer};
fn main() -> anyhow::Result<()> {
let thresholds = ChangeThresholds::default()
.improvement_ratio(1.1)
.strong_improvement_ratio(1.5)
.regression_ratio(0.95);
let markdown = Renderer::new("target/criterion")
.candidate("new")
.baseline_root("artifacts/criterion")
.baseline("main")
.benchmark("group/benchmark/1")
.benchmarks(["group/benchmark/2", "group/benchmark/3"])
.change_thresholds(thresholds)
.summary_limit(5)
.title("Benchmark Results")
.collapsible(true)
.render()?;
println!("{markdown}");
Ok(())
}
```
The builder defaults to the `new` candidate, all benchmark entries, the title `Benchmarks`, and non-collapsible output. When no baseline is specified, it uses Criterion's default `base` dataset if present and otherwise omits comparison information. An explicitly selected baseline that cannot be found returns an error. Calls to `benchmark` and `benchmarks` are additive. Use `baseline` to select a different dataset, `title` to configure the top-level Markdown heading or `<summary>` label, and `collapsible(true)` to wrap the report in a `<details>` element.
The summary appears before the detailed benchmark tables. It lists the top three improvements and top three regressions by default, using the same configured thresholds as the table indicators and omitting either category when it has no entries. If no compared benchmark crosses either threshold, the summary says so explicitly. Use `summary_limit` to set the maximum number shown in each category; a limit of `0` omits the summary.
Change thresholds use the ratio `baseline time / candidate time`. The defaults classify changes as follows:
- `ratio <= 0.9`: ❌ regression
- `0.9 < ratio < 1.1`: ➖ neutral
- `1.1 <= ratio < 1.8`: ↗️ improvement
- `ratio >= 1.8`: 🚀 strong improvement
Invalid or incorrectly ordered threshold configurations return an error from `render`.
`baseline_root` can point to a separate Criterion output tree, such as a downloaded CI artifact. It is also where the renderer looks for the default `base` dataset when no baseline is specified. For each candidate benchmark, the renderer reads the baseline from the same relative benchmark path beneath that root.
The existing free functions remain available as convenience entrypoints:
- `criterion_markdown::render(criterion_dir, allowlist)` renders the `new` candidate and compares it with `base` when available.
- `criterion_markdown::render_with_options(criterion_dir, allowlist, &options)` additionally configures an optional baseline, title, and collapsible output through `RenderOptions`.
The change point estimate is computed with the same formula Criterion uses:
`candidate_mean / baseline_mean - 1`. Selecting the same candidate and baseline Criterion used for a run therefore produces the same point estimate as its `change/estimates.json` output. If the selected baseline is absent for every benchmark, the report omits comparison information. If the baseline exists for only some benchmarks, entries without it render `---` for their change.
For comparisons rendered after the benchmark run, prefer a stable named baseline created with Criterion's `--save-baseline <name>` option. Criterion's default save mode can replace `base` with the new measurements after computing its change, so that directory may no longer contain the historical data used by the precomputed comparison.
## How This Differs From criterion-table
- This crate reads benchmark data directly from the JSON files generated by Criterion in `target/criterion`.
- It does not depend on `cargo-criterion`.
- This crate is a library you can embed and call from your own project code; `criterion-table` is primarily used as a standalone binary tool.
- You can point it at an existing Criterion output directory and render markdown without changing how benchmarks are run.
## Development
Run checks locally:
```bash
cargo +nightly fmt --check
cargo test
cargo clippy --all-targets -- -D warnings
```