pub struct RebuildTriggers {
pub tail_hits_per_row: f64,
pub min_tail_hits: u64,
pub tail_bytes: u64,
pub fold_ratio: f64,
}Expand description
When the Arrow/stree projection is rebuilt from the redb tail. Either trigger fires; both are per repository and per generation.
Fields§
§tail_hits_per_row: f64The miss trigger, as a multiple of the rows already in the projection.
See DEFAULT_TAIL_HITS_PER_ROW for why
it is relative and not a constant.
min_tail_hits: u64Floor and off-switch for the miss trigger: the threshold is never below
this, and 0 disables the miss trigger entirely whatever
tail_hits_per_row says.
tail_bytes: u64Rebuild once this many stored object bytes have been appended since the
last rebuild. 0 disables the volume trigger. On a persisting stack this
is the floor of the volume threshold; see fold_ratio.
fold_ratio: f64On a persisting stack, the volume threshold scales with the index:
a fold fires once the object bytes appended since the last fold reach
fold_ratio × the persisted index’s own size (never below tail_bytes).
0.0 leaves the fixed floor alone. See
DEFAULT_FOLD_RATIO for why the value is 1.
Ignored on a stack that does not persist — there a rebuild writes nothing
and the fixed 64 MiB is the right cadence.
Implementations§
Source§impl RebuildTriggers
impl RebuildTriggers
Sourcepub const DEFAULT_TAIL_HITS_PER_ROW: f64 = 1.0
pub const DEFAULT_TAIL_HITS_PER_ROW: f64 = 1.0
1.0 — one tail-served lookup per row in the projection.
The miss trigger is relative because the two costs it balances scale
differently, and a single constant is therefore wrong at one end or the
other. Both costs MEASURED on oden 2026-08-07, 1-min loadavg 2.1–2.2,
examples/read_stack_bench.rs, sha1 oids, batch 1000, 3 runs per cell,
worst run-to-run spread 24.9%:
| 100 000 objects | 1 000 000 objects | |
|---|---|---|
| lookup the projection answers | 80 ns | 221 ns |
| lookup the tail answers | 463 ns | 937 ns |
| the fall-through costs | 383 ns | 716 ns |
| rebuild (ordered tail scan + projection build) | 40.9 ms | 547.9 ms |
| per row in the repository | 409 ns | 548 ns |
| break-even, tail-served lookups per row | 1.07 | 0.77 |
The rebuild has paid for itself once the fall-through has carried roughly one lookup per row, at both sizes and an order of magnitude apart — which is why the ratio is the right shape for this trigger and 1.0 is the right value in it. A fixed 4096 would rebuild a million-object repository (0.55 s) to save 4096 × 716 ns ≈ 2.9 ms — 190× the wrong way.
Re-measure if the tail engine or the projection build changes: this is a ratio of two measured costs and nothing else.
Sourcepub const DEFAULT_MIN_TAIL_HITS: u64 = 4096
pub const DEFAULT_MIN_TAIL_HITS: u64 = 4096
4096 — the floor under the relative threshold.
On an empty or nearly-empty projection the ratio above is ~0 and would rebuild on the first miss, over and over, during exactly the period when objects are still arriving. 4096 misses is ~1.6 ms of fall-through at the measured 100 000-object price — cheap enough to be worth waiting for on any repository, and enough that a burst of small pushes coalesces into one rebuild.
Sourcepub const DEFAULT_TAIL_BYTES: u64
pub const DEFAULT_TAIL_BYTES: u64
64 MiB of appended object bytes.
A git push is a packfile, and the pack is the unit that lands in the tail. 64 MiB is comfortably more than one ordinary push and less than a big one, so a busy repository rebuilds on the order of once per large push rather than once per push — while a repository taking a 2 GiB initial import rebuilds ~32 times over that import instead of once at the end, which is what keeps the fall-through from carrying the whole import. It is the trigger that carries the normal case; the miss trigger is the backstop for a repository that is read hard and written rarely.
Sourcepub const DEFAULT_FOLD_RATIO: f64 = 1.0
pub const DEFAULT_FOLD_RATIO: f64 = 1.0
1.0 — fold when the bytes appended since the last fold equal the index’s own size.
Since 2026-09-04 a rebuild on a persisting stack WRITES the projection
(objects.index), so the cadence above stopped being free. MEASURED on
the kernel mirror with the fixed 64 MiB floor alone: an 8.72 GB import
folded ≈136 times, each rewriting the whole growing file — ≈51 GB of
sequential writes for a 747 MB index (INFERRED from the arithmetic; ≥18
distinct file sizes OBSERVED at 10 s sampling). The ack did not slow on
oden’s NVMe; on the appliance’s disks it would.
A ratio of 1 gives a bound that is easy to say and holds by construction:
the fold’s total writes never exceed the push’s own bytes. Each fold
writes an index roughly the size of the bytes that triggered it, so the
sum of index writes over an import is ≈ the bytes imported — ~8.7 GB for
the kernel instead of ~51, and the number of folds falls from ~136 to
O(log n). The price is the redb tail’s high-water mark: with the index at
53.5 B/object and pack bytes at ~620 B/object, “appended bytes = index
bytes” is ≈ 8.6 % of the folded rows in redb before a fold, i.e. ~1.2 M
rows ≈ 250 MB of redb rows on the kernel mirror (redb allocates in
doublings, so a ~512 MiB file) against 270 MB with the floor alone.
Re-measure on the rig after changing either number; both are in
.nornir/storage-levers-2026-09-03.md.
Sourcepub fn miss_threshold(&self, projection_rows: u64) -> Option<u64>
pub fn miss_threshold(&self, projection_rows: u64) -> Option<u64>
The miss threshold for a projection of projection_rows rows, or None
when the miss trigger is off.