armdb 0.14.2

//! T4 gate: the ceiling of a decoded value cache, measured without building one.
//!
//! Fix round 1: the original version of this benchmark measured a `pread64`
//! per read in the `var_tree`/`var_typed_tree` cells at 256 B and 4 KiB. The
//! block cache only caches a block read from a **rotated-out (immutable)**
//! file (`is_full_block = active_read.is_none() && ...`,
//! `armdb/src/shard.rs`), and the default `max_file_size` is hundreds of MiB —
//! far bigger than this benchmark's dataset — so no rotation ever happened and
//! every `block_cache`-tier read (values <= 8 KiB) fell through to a real
//! `pread`, unnoticed because the crate never surfaces "this read touched
//! disk". This version:
//!
//! 1. uses a small `max_file_size` (`hot_cache_config`) and writes padding
//!    rows, under keys never read during measurement, until every measured
//!    key's block has rotated into an immutable file — only then does the
//!    block cache actually accept it. Scoped to the `block_cache` tier
//!    (<= 8 KiB) — see "fix round 2" below for why 32 KiB stays unpadded even
//!    though an equalized index size was tried;
//! 2. proves residency instead of assuming it: `check_residency` reads
//!    `/proc/self/io`'s `syscr:` line (cumulative read syscalls for the
//!    process) before and after a representative read pass per cell and
//!    prints the delta — nonzero means that cell is not a clean cache-hit
//!    measurement;
//! 3. changes the ceiling's floor from `TypedTree::get` (which skips
//!    mechanics a decoded-cache hit would still pay: `shard_for`, `ValueKey`
//!    construction, the cache lookup, verify-on-hit, the handle clone) to the
//!    **value-cache-hit `VarTree::get` read at 32 KiB** — operation-for-
//!    operation what a decoded-cache hit does, and size-independent. See
//!    `armdb/docs/decoded-cache-ceiling.md` for the corrected verdict; the
//!    `typed_tree` column/variant stays in this file for reference only.
//!
//! Fix round 1 (a first pass at step 1 above) padded only the sizes it
//! believed needed it (<= 8 KiB), reasoning that `value_cache` (32 KiB)
//! accepts unconditionally and gains nothing from padding. That padding
//! itself, though, built a **fresh shaped value per padding row** via
//! `$make(...)`, which measurably fragmented the allocator ahead of
//! `var_typed_tree`'s per-iteration decode allocation — a self-inflicted ~4x
//! slowdown at `string/32KiB`, caught by noticing the number moved and
//! re-running with padding removed there. The fix at the time was "stop
//! padding 32 KiB". Fix round 2 found the same defect still live at
//! `string/4KiB` (a shaped padding value is a ~4 KiB `String` there too) —
//! confirmed against an independently measured clean-decode figure, see the
//! results doc's decode cross-check — and diagnosed the first-order cause
//! correctly: **a shaped per-row padding value is the bug, at any size, in
//! either arm** — not "padding a tier that doesn't need it." This version
//! builds **one** padding buffer of raw, hoisted bytes per size
//! (`PADDING_ROW_BYTES`, sized to stay in the `block_cache` tier regardless
//! of the size under test) and reuses it verbatim for every padding row, in
//! **both** arms — the `VarTypedTree` arm writes it via `as_inner().put()`,
//! bypassing the codec so no shaped value or encode buffer is ever
//! constructed for a padding row.
//!
//! Round 2 also tried applying that byte-only padding to **all three** sizes,
//! including 32 KiB, to equalize index size between the padded cells and the
//! unpadded floor (an acknowledged bias — see the results doc's fix-history
//! section for the disclosure). That attempt was reverted: even with zero
//! per-row allocation in the padding writes themselves, padding 32 KiB still
//! regressed
//! `var_typed_tree/32KiB` by 30-320% (verified per the coordinator's
//! explicit stability check), while `var_tree/32KiB` — same padding, no
//! decode — moved only 3-11%. Padding stays scoped to <= 8 KiB; the
//! resulting index-size mismatch at 32 KiB is disclosed, not silently fixed
//! at the cost of reintroducing a multi-microsecond artifact.
//!
//! **The same effect was still present at 4 KiB itself**, initially missed
//! because the decode cross-check (below) wasn't written until after that
//! first pass landed: `PADDING_ROWS` at a 4x safety factor (4096 rows) gave
//! `string/4KiB` an implied decode of ~910-955 ns against a clean baseline of
//! ~135-139 ns — off by 6.7-6.9x, reproducible across repeated runs, and
//! confirmed by a standalone isolation harness (fresh process, a single
//! `var`/`var_typed` collection pair, no criterion, no other cells) that
//! swept the padding row count: `decode from plain Vec<u8> copy` — a decode
//! call with no armdb involvement in the timed loop at all, on bytes already
//! extracted — jumped from a clean ~140-160 ns to a corrupted ~910-955 ns
//! specifically between 3500 and 4096 rows, and stayed corrupted at every
//! value tried up to 4096, while every value from 0 to 3500 (including 0,
//! i.e. no rotation and no residency either) stayed clean. This is the same
//! allocator-sensitivity mechanism as the 32 KiB case — the code path that
//! allocates a fresh multi-KB buffer per call is what's exposed, regardless
//! of size — just triggered by **padding volume alone**, not by anything
//! about per-row allocation (which round 2's byte-only fix had already
//! eliminated). The fix: `PADDING_ROWS`'s safety factor is 2x, not 4x —
//! independently confirmed clean and still resident across repeated runs in
//! the same isolation harness, with a comfortable margin below the observed
//! 3500-4096 threshold. See that constant's doc comment for how to re-verify
//! this if the padding recipe ever changes.
//!
//! `armdb/docs/decoded-cache-ceiling.md` records a **decode cross-check**:
//! each cell's implied decode (`var_typed_tree − var_tree`) compared against
//! an independently measured clean-`rapira::deserialize` figure with no
//! armdb/storage involved at all. Any cell more than ~2x off is a defect in
//! *this benchmark*, not a finding about armdb — round 2 found exactly one
//! (`string/4KiB`, per above) and traced it, first to the per-row shaped
//! padding value, then — after that fix didn't fully close the gap — to
//! padding volume itself; this check exists so a future change to this file
//! gets caught immediately instead of shipping a wrong ceiling. **Caveat**:
//! this check is blind to inflation shared by both arms of the subtraction —
//! see fix round 3 below, which is exactly what it caught.
//!
//! Fix round 3: re-review found the ceiling's floor itself was contaminated.
//! `numeric/var_tree/32KiB` read 78.824 ns in round 2's published run — a
//! +39% jump from every other measurement of the same code path (52.44 ns,
//! 53.92 ns, 52.96 ns for `string`; 55.54 ns, 56.69 ns for `numeric`, across
//! rounds 1-2) on a path this round never touched (32 KiB has been unpadded
//! since round 1). `VarTree` is untyped — the value-cache hit path is
//! `shard_for` + `ValueKey` + a hash `get` + two integer compares + a
//! `ByteView` clone (`armdb/src/var_tree.rs`), nothing reads the value's
//! bytes, and the `string`/`numeric` 32 KiB payloads differ by only 4 bytes
//! (32768 vs 32764) — so the two shapes' `var_tree`/32KiB numbers measure
//! *the same operation* and have no legitimate reason to diverge, let alone
//! by 49%. Re-measured `var_tree/32KiB` for both shapes 3 times each via
//! `cargo bench ... -- "var_tree/32KiB"`: `numeric` returned 50.966 ns,
//! 54.628 ns, 60.247 ns — back in `string`'s own range (53.088 ns, 53.701 ns,
//! 57.749 ns) — confirming 78.824 ns was a one-off outlier, not a
//! reproducible shape effect. Fix: **the per-shape floor is retired**. Since
//! the code path is shape-independent by construction (not just by
//! observation), a single **pooled floor** — the mean of these six
//! re-measured samples, 55.06 ns, range 50.97-60.25 ns — replaces it for
//! both shapes' `ceiling %`. See `armdb/docs/decoded-cache-ceiling.md` for
//! the full re-measurement table, the corrected `ceiling %` for every cell,
//! and why the old "32 KiB row's `ceiling` == `decode-only share`" framing
//! was an algebraic identity (floor = that row's own `var_tree`, true even
//! for an anomalous sample), not the consistency check it was described as.
//! Also added: a committed, reproducible decode-cross-check tripwire (the
//! earlier baseline was hardcoded from an uncommitted ad hoc probe) — as a
//! `clean_decode` benchmark group *inside this file*, which fix round 4 then
//! had to move out; see below.
//!
//! Fix round 4: that tripwire was **desensitised by construction** and must
//! not live in this file. It ran in this process, after `bench_shape!`'s setup
//! for all six cells had already executed — criterion's `-- <filter>` argument
//! gates only *sampling* (`filter_matches` → `if do_run { analysis::common(..) }`,
//! `criterion-0.8.2/src/benchmark_group.rs:303`), never the plain Rust setup
//! here, which is unconditional. The failure mode the tripwire exists to catch
//! is process-wide: round 2 measured a plain-`Vec<u8>` decode with no armdb in
//! the timed loop inflating from ~140-160 ns to ~910-955 ns under the
//! corrupting condition. A baseline sharing that process inflates in lockstep
//! with the implied decode it checks, the ratio stays near 1.0x, and the
//! tripwire reports "passes" on the exact failure round 2 caught at 6.9x. That
//! is the whole argument and it is a priori — it needs no claim that any past
//! run *was* contaminated. (Round 4's first draft also cited the gap between
//! round 3's in-process baseline and an external bare-process probe as
//! corroboration; the new target's own bare-process run refuted that reading —
//! see `armdb/docs/decoded-cache-ceiling.md`, "Decode cross-check". History,
//! not evidence.) **The tripwire baseline must come
//! from a process that has not run `bench_ceiling`'s setup**, so it now lives
//! in its own bench target, `armdb/benches/clean_decode.rs` — run it with
//! `cargo bench -p armdb --bench clean_decode --features rapira-codec`. Do not
//! re-add a clean-decode group to this file: being a separate binary *is* the
//! mechanism.
//!
//! Three variants read the same data with every byte cache hot (now actually
//! verified, not assumed):
//!
//! | variant | cost |
//! |---|---|
//! | `VarTree::get`      | index + byte-cache hit + copy — `var_tree`/32 KiB feeds the pooled T4 floor |
//! | `VarTypedTree::get` | the same **plus** a codec decode — today's cost |
//! | `TypedTree::get`    | index lookup + hand back a pointer — reference only |
//!
//! Access pattern is strictly sequential (`i = (i + 1) % ROWS`) — the
//! friendliest case for the CPU prefetcher; treat these as a floor on
//! real-world random-access cost, not a ceiling.
//!
//! Run: `cargo bench -p armdb --bench decoded_ceiling \
//!       --features "typed-tree rapira-codec var-collections"`
//!
//! Adding `-- "var_tree/32KiB"` re-samples only the floor cells — but note
//! that a criterion filter skips **sampling only**: the setup above for all
//! six `(shape, size)` cells still runs, in this same process, whatever the
//! filter says. No filter of this benchmark can produce a bare process; that
//! is why the decode-cross-check baseline is a separate target
//! (`armdb/benches/clean_decode.rs`).

use std::hint::black_box;

use armdb::{CacheConfig, Config, RapiraCodec, TypedTree, VarTree, VarTypedTree};
use criterion::{Criterion, criterion_group, criterion_main};
use rapira::Rapira;
use tempfile::TempDir;

/// Worst case for decoding: UTF-8 validation plus several small allocations.
#[derive(Clone, Debug, PartialEq, Rapira)]
struct StringShape {
    id: u64,
    body: String,
    tags: Vec<String>,
}

/// Best case: one bulk numeric allocation, no UTF-8 validation.
#[derive(Clone, Debug, PartialEq, Rapira)]
struct NumShape {
    id: u64,
    values: Vec<u64>,
}

/// Number of rows per dataset. 256 x 32 KiB = 8 MiB, comfortably inside the
/// 64 MiB byte-cache budgets below, so every measured read is a cache hit
/// (verified per cell by `check_residency`, not merely assumed).
const ROWS: u64 = 256;

const SIZES: [(&str, usize); 3] = [("256B", 256), ("4KiB", 4096), ("32KiB", 32768)];

/// Small enough that a modest burst of padding writes reliably rotates every
/// shard's active file at least once. The block cache only caches blocks read
/// from a rotated-out (immutable) file (`armdb/src/shard.rs:
/// is_full_block`), so this — not the byte-cache size — is what makes
/// `block_cache`-tier hits (<= 8 KiB) actually cache-resident.
const MAX_FILE_SIZE: u64 = 1 << 20; // 1 MiB
/// Must be <= `MAX_FILE_SIZE` and >= 4096 (`Config::validate`).
const WRITE_BUFFER_SIZE: usize = 256 << 10; // 256 KiB
const SHARD_COUNT: u64 = 4;
const PADDING_ROW_BYTES: usize = 4096;
/// 2x the expected-per-shard minimum needed to force one rotation, so hash
/// imbalance across `SHARD_COUNT` shards still rotates every shard at least
/// once. Padding keys (`ROWS..ROWS + PADDING_ROWS`) are never read.
///
/// This safety factor was 4x through fix round 2's first pass, which turned
/// out to be **inside a heap-fragmentation threshold**: even fully byte-only,
/// hoisted, symmetric padding (see "fix round 2" above) measurably corrupts
/// `var_typed_tree`'s decode cost once padding volume crosses some point
/// between 3500 and 4096 rows here — confirmed by a standalone isolation
/// harness (fresh process, single collection pair, `string/4KiB` only) that
/// swept the row count and found `decode from plain Vec<u8> copy` (no armdb
/// call in the timed loop at all) jump from ~140-160 ns to ~910-955 ns at
/// 4096 rows, reproducibly across repeated runs, while every value from 0 to
/// 3500 stayed clean. 2x (2048 rows here) sits with margin below that
/// threshold and was independently confirmed stable across repeated runs in
/// the same harness. If this constant, `MAX_FILE_SIZE`, or `SHARD_COUNT`
/// change, re-run that sweep before trusting the decode cross-check again —
/// this is a real allocator effect, not a tunable-away one.
///
/// **This margin is not a hard guarantee.** In the full 18-cell run (all six
/// `(shape, size)` cells sharing one process — far more cumulative
/// allocation volume than the single-cell isolation harness above), one run
/// out of five during this investigation still showed the corrupted pattern
/// at 2x (`string/4KiB` implied decode ~955 ns) while the other four were
/// clean (~170-190 ns, matching the independent baseline within ~30%) — most
/// likely address-space-layout-driven nondeterminism in glibc's arena
/// placement, not a deterministic function of this constant alone. The decode
/// cross-check in `armdb/docs/decoded-cache-ceiling.md` exists precisely to
/// catch this: **re-check it after any run before trusting the numbers**, and
/// treat a single clean run as good evidence, not proof.
const PADDING_ROWS: u64 = (2 * SHARD_COUNT * MAX_FILE_SIZE) / PADDING_ROW_BYTES as u64;

/// Passes over the full `ROWS` key range in the residency check — enough to
/// catch a residual cache miss without adding meaningful wall-clock time.
const RESIDENCY_PASSES: u64 = 4;

fn key(i: u64) -> [u8; 8] {
    i.to_be_bytes()
}

fn string_shape(id: u64, target_bytes: usize) -> StringShape {
    // 4 tags of ~8 bytes each plus the u64 and the length prefixes; the rest
    // goes into `body` so the encoded size lands on `target_bytes`.
    let tags: Vec<String> = (0..4).map(|t| format!("tag{t:04}")).collect();
    let overhead = rapira::serialize(&StringShape {
        id,
        body: String::new(),
        tags: tags.clone(),
    })
    .len();
    let body_len = target_bytes.saturating_sub(overhead);
    StringShape {
        id,
        body: "x".repeat(body_len),
        tags,
    }
}

fn num_shape(id: u64, target_bytes: usize) -> NumShape {
    let overhead = rapira::serialize(&NumShape {
        id,
        values: Vec::new(),
    })
    .len();
    let n = target_bytes.saturating_sub(overhead) / 8;
    NumShape {
        id,
        values: (0..n as u64).collect(),
    }
}

/// Byte caches large enough that no measured read touches the disk, and a
/// `max_file_size` small enough that the padding-write step (see
/// `bench_shape!`) can force rotation without writing an unreasonable amount
/// of data. `block_cache` serves values up to two blocks, `value_cache` the
/// rest.
fn hot_cache_config() -> Config {
    Config::balanced()
        .shard_count(SHARD_COUNT as usize)
        .max_file_size(MAX_FILE_SIZE)
        .write_buffer_size(WRITE_BUFFER_SIZE)
        .block_cache(CacheConfig {
            max_size: 64 << 20,
            estimated_items: 32_768,
        })
        .value_cache(CacheConfig {
            max_size: 64 << 20,
            estimated_items: 4_096,
        })
        .build()
}

/// Cumulative read syscalls for this process. `/proc/self/syscr` is not a
/// real file on Linux — the counter lives in the `syscr:` line of
/// `/proc/self/io` (task I/O accounting, on by default; confirmed present in
/// the environment this benchmark was run in).
fn syscr() -> u64 {
    let io = std::fs::read_to_string("/proc/self/io").expect("read /proc/self/io");
    io.lines()
        .find_map(|line| line.strip_prefix("syscr:"))
        .expect("/proc/self/io has no syscr: line")
        .trim()
        .parse()
        .expect("parse syscr count")
}

/// Fixed self-cost of one `syscr()` call: `/proc/self/io` is a virtual file
/// with no advertised length, so `read_to_string` issues several `read()`
/// calls to grow its buffer to fit (measured at a constant 4 on this system,
/// via `strace -c` on a bracketing pair with nothing in between). A
/// `before`/`after` bracket therefore over-counts by exactly one call's worth
/// of this self-cost — calibrated once and subtracted below, so a genuinely
/// cache-resident cell reports a delta of 0 instead of a constant false
/// positive.
fn syscr_self_overhead() -> u64 {
    let a = syscr();
    let b = syscr();
    b - a
}

/// Run `read_one` over every key in `0..ROWS`, `passes` times, and report the
/// read-syscall delta straddling that pass, net of `syscr_self_overhead`. A
/// nonzero delta means at least one read in this cell fell through to a real
/// `pread` — the criterion timing recorded right after this call (same
/// reads, same order) is then not a clean cache-hit measurement. Printed
/// unconditionally so both outcomes are visible in the bench log, not just
/// failures.
fn check_residency(label: &str, passes: u64, mut read_one: impl FnMut(u64)) -> u64 {
    let overhead = syscr_self_overhead();
    let before = syscr();
    for _ in 0..passes {
        for i in 0..ROWS {
            read_one(i);
        }
    }
    let after = syscr();
    let raw = after - before;
    let delta = raw.saturating_sub(overhead);
    let reads = passes * ROWS;
    if delta == 0 {
        eprintln!(
            "RESIDENCY[{label}] ok: 0 read syscalls over {reads} reads \
             (raw={raw}, self_overhead={overhead})"
        );
    } else {
        eprintln!(
            "RESIDENCY[{label}] FAILED: {delta} read syscalls over {reads} reads \
             (raw={raw}, self_overhead={overhead}) — NOT cache-resident, timing below is INVALID"
        );
    }
    delta
}

macro_rules! bench_shape {
    ($c:expr, $shape_name:literal, $make:ident, $ty:ty) => {{
        let mut group = $c.benchmark_group(concat!("decoded_ceiling/", $shape_name));
        for (label, size) in SIZES {
            // One padding buffer of raw bytes, built once and reused verbatim
            // for every padding row below, in *both* arms (see the module doc
            // comment's "fix round 2" note). It never needs to be a decoded
            // shape — padding exists solely to rotate measured keys into an
            // immutable file — so building it once here means the padding
            // loops do zero per-row allocation, and both arms pay the exact
            // same setup cost.
            let padding: Vec<u8> = vec![0u8; PADDING_ROW_BYTES];

            // --- VarTree: bytes only, no decode -----------------------------
            let dir = TempDir::new().unwrap();
            let var: VarTree<[u8; 8]> = VarTree::open(dir.path(), hot_cache_config()).unwrap();
            for i in 0..ROWS {
                let encoded = rapira::serialize(&$make(i, size));
                var.put(&key(i), &encoded).unwrap();
            }
            // Padding under keys outside the measured range, until every
            // measured key's block has rotated into an immutable file (see
            // module doc comment). Never read during measurement. Scoped to
            // the block_cache tier (<= 8 KiB) only — see the module doc
            // comment's "fix round 2, attempt 2" note: applying this same
            // byte-only padding to 32 KiB was tried and reverted, because it
            // still measurably regressed `var_typed_tree`'s decode allocation
            // even with zero per-row allocation in the padding writes
            // themselves.
            if size <= 8192 {
                for j in 0..PADDING_ROWS {
                    var.put(&key(ROWS + j), &padding).unwrap();
                }
            }
            var.flush_buffers().unwrap();
            var.warmup().unwrap();
            for i in 0..ROWS {
                black_box(var.get(&key(i))); // existing read pass — warm every byte cache tier
            }
            let resident = check_residency(
                &format!(concat!($shape_name, "/var_tree/{}"), label),
                RESIDENCY_PASSES,
                |i| {
                    black_box(var.get(&key(i)));
                },
            );
            group.bench_function(format!("var_tree/{label}"), |b| {
                let mut i = 0u64;
                b.iter(|| {
                    i = (i + 1) % ROWS;
                    black_box(var.get(&key(i)).map(|v| v.len()))
                })
            });
            if resident != 0 {
                eprintln!(
                    "WARNING: decoded_ceiling/{}/var_tree/{} is NOT cache-resident \
                     ({resident} read syscalls) — exclude from the verdict.",
                    $shape_name, label
                );
            }

            // --- VarTypedTree: bytes + decode (today's cost) ----------------
            let dir_vt = TempDir::new().unwrap();
            let var_typed: VarTypedTree<[u8; 8], $ty, RapiraCodec> =
                VarTypedTree::open(dir_vt.path(), hot_cache_config(), RapiraCodec).unwrap();
            for i in 0..ROWS {
                var_typed.put(&key(i), &$make(i, size)).unwrap();
            }
            // Same prebuilt `padding` bytes as the `VarTree` arm (same size
            // gate too — see that arm's comment), written through
            // `as_inner()` to bypass the codec entirely: a padding row is
            // never decoded, so it must never pay an encode either. This is
            // what makes `decode-only share` (var_typed_tree − var_tree,
            // computed from the doc's table) a fair same-size subtraction at
            // the sizes that do pad: both arms run an identical padding
            // setup, not one that allocates a fresh shaped value per row and
            // one that doesn't.
            if size <= 8192 {
                for j in 0..PADDING_ROWS {
                    var_typed.as_inner().put(&key(ROWS + j), &padding).unwrap();
                }
            }
            var_typed.flush_buffers().unwrap();
            var_typed.warmup().unwrap();
            for i in 0..ROWS {
                black_box(var_typed.get(&key(i)));
            }
            let resident = check_residency(
                &format!(concat!($shape_name, "/var_typed_tree/{}"), label),
                RESIDENCY_PASSES,
                |i| {
                    black_box(var_typed.get(&key(i)));
                },
            );
            group.bench_function(format!("var_typed_tree/{label}"), |b| {
                let mut i = 0u64;
                b.iter(|| {
                    i = (i + 1) % ROWS;
                    black_box(var_typed.get(&key(i)).map(|v| v.id))
                })
            });
            if resident != 0 {
                eprintln!(
                    "WARNING: decoded_ceiling/{}/var_typed_tree/{} is NOT cache-resident \
                     ({resident} read syscalls) — exclude from the verdict.",
                    $shape_name, label
                );
            }

            // --- TypedTree: reference only (never touches storage on read) --
            let dir_t = TempDir::new().unwrap();
            let typed: TypedTree<[u8; 8], $ty, RapiraCodec> =
                TypedTree::open(dir_t.path(), hot_cache_config(), RapiraCodec).unwrap();
            for i in 0..ROWS {
                typed.put(&key(i), $make(i, size)).unwrap();
            }
            let resident = check_residency(
                &format!(concat!($shape_name, "/typed_tree/{}"), label),
                RESIDENCY_PASSES,
                |i| {
                    black_box(typed.get(&key(i)));
                },
            );
            group.bench_function(format!("typed_tree/{label}"), |b| {
                let mut i = 0u64;
                b.iter(|| {
                    i = (i + 1) % ROWS;
                    black_box(typed.get(&key(i)).map(|v| v.id))
                })
            });
            if resident != 0 {
                eprintln!(
                    "WARNING: decoded_ceiling/{}/typed_tree/{} unexpectedly touched storage \
                     ({resident} read syscalls).",
                    $shape_name, label
                );
            }
        }
        group.finish();
    }};
}

fn bench_ceiling(c: &mut Criterion) {
    bench_shape!(c, "string", string_shape, StringShape);
    bench_shape!(c, "numeric", num_shape, NumShape);
}

criterion_group!(benches, bench_ceiling);
criterion_main!(benches);