Crate protest_criterion 

Property-Based Benchmarking with Criterion

This crate provides integration between Protest property-based testing and Criterion benchmarking.

Overview

Property-based benchmarking allows you to:

• Benchmark functions with diverse, generated inputs
• Measure performance across the full input space
• Detect performance regressions with statistical analysis
• Understand worst-case and average-case performance

Quick Start

use criterion::{criterion_group, criterion_main, Criterion};
use protest_criterion::PropertyBencher;
use protest::primitives::IntGenerator;

fn bench_abs(c: &mut Criterion) {
    c.bench_function_over_inputs(
        "i32::abs",
        |b, &input: &i32| b.iter(|| input.abs()),
        IntGenerator::new(-1000, 1000),
        100, // sample count
    );
}

criterion_group!(benches, bench_abs);
criterion_main!(benches);

Features

• Generator Integration: Use any Protest generator for benchmark inputs
• Statistical Analysis: Leverage Criterion’s statistical analysis
• Regression Detection: Automatic detection of performance changes
• Input Distribution: Benchmark across diverse input distributions
• Reproducible: Seed-based generation for consistent benchmarks

Examples

Benchmarking Sorting Algorithms

use criterion::{criterion_group, criterion_main, Criterion, BenchmarkId};
use protest_criterion::PropertyBencher;
use protest::primitives::VecGenerator;
use protest::{Generator, config::GeneratorConfig};

fn bench_sort_algorithms(c: &mut Criterion) {
    let mut group = c.benchmark_group("sorting");

    for size in [10, 100, 1000].iter() {
        let generator = VecGenerator::new(
            protest::primitives::IntGenerator::new(0, 1000),
            *size,
            *size
        );

        group.bench_with_input(
            BenchmarkId::new("sort", size),
            &generator,
            |b, generator| {
                let config = GeneratorConfig::default();
                b.iter_batched(
                    || generator.generate(&mut rand::thread_rng(), &config),
                    |mut v| v.sort(),
                    criterion::BatchSize::SmallInput
                )
            }
        );
    }

    group.finish();
}

criterion_group!(benches, bench_sort_algorithms);
criterion_main!(benches);

Benchmarking with Properties

use criterion::{criterion_group, criterion_main, Criterion};
use protest_criterion::PropertyBencher;
use protest::primitives::VecGenerator;
use protest::Generator;

fn bench_property(c: &mut Criterion) {
    c.bench_property(
        "vec reversal is involutive",
        VecGenerator::new(
            protest::primitives::IntGenerator::new(0, 100),
            0,
            1000
        ),
        |v: &Vec<i32>| {
            let mut reversed = v.clone();
            reversed.reverse();
            reversed.reverse();
            assert_eq!(v, &reversed);
        },
        100 // sample count
    );
}

criterion_group!(benches, bench_property);
criterion_main!(benches);

Traits

PropertyBencher

Extension trait for Criterion to enable property-based benchmarking

PropertyBenchmarkGroup

Helper for creating benchmark groups with property-based inputs