Struct ObjectReadStack 

pub struct ObjectReadStack<S: ObjectIndex = OneTableFourColumns> { /* private fields */ }

The read stack: Ragnar stree → Arrow columns → redb, in one object.

Parameterised by the Arrow layout so it drops into examples/index_layout_bench.rs against FourTables and OneTableFourColumns unchanged — it implements the same ObjectIndex trait, and ObjectIndex::build produces a stack whose projection already covers every entry, so on that benchmark it is those arms plus one redb round trip per genuine miss.

Implementations

impl<S: ObjectIndex> ObjectReadStack<S>

pub fn open( tail_path: &Path, triggers: RebuildTriggers, cache_bytes: usize, ) -> Result<Self>

Open (or create) one repository’s stack at tail_path, warm-starting the projection from everything already in the tail. cache_bytes is redb’s page-cache ceiling for this database. A parameter and not a getenv down here, so the whole store costs one environment read at construction and none below it; see crate::arms::redb_cache_bytes, and note that the stock Database::create would take redb’s 1 GiB default instead.

pub fn projection_name(&self) -> &'static str

The name the built projection reports about itself — the applied output that says which crate::arms::IndexArm actually got built.

Deliberately not ObjectIndex::name on the stack, which answers "ObjectReadStack" for every arm because that is what the stack is. S is the layout, and only S can say which one it is. It is the value crate::arms::IndexArm::projection_name is written to be compared against, so a caller that selected an arm can prove the selection took rather than echo the selector back to itself.

pub fn in_memory(triggers: RebuildTriggers) -> Result<Self>

A stack whose tail has no file behind it. Used by ObjectIndex::build and by benchmarks; the redb code path is identical, only the backend differs, so a test of this stack is a test of the durable one.

pub fn projection_is_complete(&self) -> bool

The projection covers every row in the repository, so a miss in the stree is an absence and the tail need not be asked.

Sound by construction, not by hope: the projection is built from a scan of the tail, so its rows are always a subset of the tail’s; the archive is append-only, so no row ever leaves; therefore equal counts mean equal sets. A concurrent append between this check and the lookup only means the lookup answers as of an instant before that append, which is what a lookup issued a microsecond earlier would have done anyway.

This is worth the paragraph because of what it buys. MEASURED on oden 2026-08-07 (read_stack_bench, 100 000 objects, batch 1000): a have negotiation — 10% hit, so 90% of the oids are in no repository at all — cost 544 ns per oid when every one of those absences took a redb round trip, against 77 ns for the bare Arrow arm. With this fast path the same workload costs 68 ns, i.e. the Arrow arm’s own cost inside the noise band. A have negotiation is mostly misses, so without this the stack would be the dominant cost of the most common operation a git server performs.

pub fn append(&self, entries: &[IndexEntry]) -> Result<Option<RebuildReason>>

Append objects to the tail.

Append-only is enforced here, not assumed. An oid already present with identical facts is a no-op (a re-pushed pack repeats objects, and that is normal); an oid already present with different facts is an error and nothing in the batch is written. Without that refusal a stree hit could return a superseded row and the projection would be not merely incomplete but wrong, which is the one thing this design must not allow.

Runs a rebuild inline if the batch trips a threshold; the reason is returned so the caller can see that it happened.

pub fn rebuild_due(&self) -> Option<RebuildReason>

Which threshold, if any, is currently tripped. Pure read; a caller on a read-only repository can poll this from its own maintenance tick.

pub fn maybe_rebuild(&self) -> Result<Option<RebuildReason>>

Rebuild if rebuild_due says so.

pub fn rebuild(&self) -> Result<()>

Rebuild the Arrow/stree projection from a full ordered scan of the tail and swap it in, resetting the counters and bumping the generation.

The old projection stays live for every reader until the swap; the write lock is held only for the pointer store, not for the build.

pub fn stats(&self) -> StackStats

pub fn projection_len(&self) -> usize

Rows in the projection right now. stats().total_rows - this is the un-absorbed tail.

pub fn oids_in_order(&self) -> Result<Vec<Vec<u8>>>

Every oid in the repository, in oid order — which is ordinal order, since redb keys the tail by the raw oid.

A full scan, and it is here for the one caller that genuinely needs the whole set at once: a GC, which has to name what is not live.

pub fn extents_with_rows(&self, extents: &[(u64, u64)]) -> Result<Vec<bool>>

Which of these archive extents already have rows here — the crash-recovery diff, and the reason §13.12’s indexed bit is derived rather than stored.

A pack is unabsorbed iff its extent is in the journal and its rows are not in the index. Both of those are already durable — the journal is fsynced on the ack path, the tail is a redb commit — so the bit is a diff of two durable facts and there is nothing for it to drift from. A stored bit would be a third fact that can disagree with the two it describes, which is precisely how a pack ends up marked absorbed with no rows behind it: fast and wrong, in the one direction (absent) that loses a client’s objects during negotiation.

out[i] answers extents[i]. The tail and not the projection, because the projection is a snapshot of a prefix of the tail and the question is about what survived the crash.

Cost

One ordered scan of the tail that stops the moment every extent has been hit. Rows arrive in oid order, which is uncorrelated with the pack an object came from, so a store whose packs are all absorbed answers after a few rows per pack rather than after a full scan — the coupon-collector case, and the one every clean reopen takes. The scan only runs to the end when some pack genuinely has no rows, which is exactly the case where the caller is about to re-resolve that whole pack anyway.

MEASURED on oden 2026-08-08, release, file-backed redb tail, 1-minute loadavg 7.1–7.5 (another tenant’s work — read these as ratios):

packs	rows	every pack absorbed	one pack un-absorbed
1 000	1 000 000	0.77 ms	120.6 ms
10 000	1 000 000	11.7 ms	141.8 ms
1 000	3 686	0.40 ms	0.39 ms

The first column is what a clean reopen pays and it is bounded by the pack count, not the object count — 1000 packs over a million objects costs the same order as 1000 packs over four thousand. The second column is the crash path, where the scan runs to the end: 120 ms over a million rows, against the ~160 ms a single 2687-object pack takes to re-absorb (a_push_ends_in_object_rows_and_no_read_asked_for_them). The diff is therefore never the expensive half of a recovery. The third row is the shape [crate::git_ops::tests::the_derive_on_open_diff_costs_milliseconds_at_a_realistic_pack_count] asserts on every run, over real pushed packs rather than synthetic rows.

A zero-length extent has no rows by construction and is answered false without looking.

pub fn retain(&self, live: &dyn Fn(&[u8]) -> bool) -> Result<u64>

Drop every row live rejects. Returns how many rows went.

This is the one operation that is not append-only, and it exists for exactly one caller: GitOps::gc, which computes reachability and then has to remove what is unreachable before base znippy compacts the payload — so that base’s notion of “live” already means what git means (§13.20).

The projection is rebuilt inside the same call. It has to be: it is a snapshot of a tail that no longer says the same thing, and a stree that still answers for a deleted oid would be wrong rather than merely incomplete, which is the one failure this design does not tolerate.

Trait Implementations

impl<S: ObjectIndex> ObjectIndex for ObjectReadStack<S>

fn build(entries: &[IndexEntry]) -> Result<Self>

Build a stack over entries with an in-memory tail. Every entry lands in redb and the projection is built from it, so the stack starts fully absorbed — which is what makes it directly comparable with the two Arrow arms on the layout benchmark.

fn lookup(&self, oid: &[u8]) -> Option<IndexRow>

Projection first; a miss falls through to redb, which always knows. None here means the tail said no, never that the stree said no.

fn lookup_batch(&self, oids: &[&[u8]]) -> Vec<Option<IndexRow>>

Two phases, and the second is not optional: one pipelined stree pass, then one redb transaction for every slot the projection left empty.

fn ordinals_batch(&self, oids: &[&[u8]]) -> Vec<Option<u32>>

The floor path — oid → ordinal, no payload column touched. The tail is still consulted for the misses, because a miss is still not an answer; a tail-served ordinal carries TAIL_ORDINAL_BIT.

fn extents_batch(&self, oids: &[&[u8]]) -> Vec<Option<(u64, u64)>>

The partial-row path — byte extent only.

fn sum_uncompressed(&self) -> u64

The quota gate, over the whole repository and without a scan of the tail: the projection’s column scan plus the rows appended after it. Exact because the archive is append-only — a row the projection already holds can never gain or lose bytes, so the two terms cannot overlap.

fn count_type(&self, t: ObjType) -> usize

sum_uncompressed’s argument, per type.

fn len(&self) -> usize

Objects in the repository, not in the projection — the stack answers for the whole repository and this has to agree with what lookup will resolve. projection_len is the other number.

fn resident_bytes(&self) -> usize

The projection’s resident bytes. The tail is on disk (or in redb’s own page cache) and is not counted here — counting it would make this incomparable with the two Arrow arms, which is the number’s only use.

fn name(&self) -> &'static str

fn ipc_bytes(&self) -> usize

Total bytes of Arrow IPC this index holds resident. Four sections or one, this counts the same payload, so it is comparable across arms.

fn is_empty(&self) -> bool