Skip to main content

GitStore

Struct GitStore 

Source
pub struct GitStore<S: ObjectIndex = OneTableFourColumns> { /* private fields */ }
Expand description

One repository’s git store.

on diskwhat it iswho owns the format
<root>/objects.packthe verbatim pack bytes, appended[SafeWriter]
<root>/objects.pack.journalthe durable extent rows[SafeWriter]
<root>/objects.tailthe un-sealed index tailObjectReadStack (redb)
<root>/objects.exploded§14’s resolved content, droppableExplodedArchive (one Arrow IPC table)
<root>/refs.logthe ref transaction logRefLog (Arrow IPC frames)
<root>/<name>.znippythe sealed archive GC compactsbase znippy

Nothing in that table is new. The store is the composition, not a fifth format.

§Which implementation of each trait, and how a caller picks

Three of the four boxes in the diagram at the top of this file have more than one implementation. StoreConfig is the choice, crate::arms documents what each arm promises, and the dispatch is split deliberately:

traitselected bydispatchcost
ObjectIndexthe type parameter Sstaticnone — the hot path is monomorphised
ArchiveWriteStoreConfig::writerdyn, inside PushPath as it already wasone virtual call per pack append, not per object
GcStoreConfig::gcdynone virtual call per gc() run

S defaults to OneTableFourColumns, so GitStore still names exactly what it named before and every existing caller compiles unchanged.

Implementations§

Source§

impl<S: ObjectIndex + 'static> GitStore<S>

Source

pub fn seal(&self) -> Result<Vec<ReservedSection>>

CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW

Fold the live logs into the reserved Arrow sections an archive carries: __gunnar_refs__, __gunnar_graph__, __gunnar_reach__.

The twelfth method, and the one that is NOT on GitOps: it returns znippy_common::ReservedSection (Arrow RecordBatch payloads), which a gix backend has no analog for — so it is inherent on the concrete store, which is how gunnar already calls it (never through the trait).

It absorbs first, because a section built while a pack is still un-indexed would be silently incomplete, and a silently incomplete index is worse than none (the same argument [crate::lib] makes about not wiring GitIndexBuilder into the CLI). Every generation number in the graph section is recomputed by that fold.

…and then it writes the archive. Until 2026-08-08 it only returned the sections and nothing on any path created archive_path at all — so gc()’s last step died on stat repository.znippy: No such file or directory, having already done its first four. Both Gc implementations compact an archive that exists; this is what makes one exist. seal_generation_zero has the layout argument.

The sections are still returned, and they are the same values that were sealed rather than a second derivation of them — ReservedSection is Clone for exactly that reason.

Source§

impl GitStore<OneTableFourColumns>

Source

pub fn open(root: &Path, account: &str) -> Result<Self>

Open (creating) the store for one repository under root.

The default arms, and it reads no environment to get them. An operator who exported ZNIPPY_GIT_WRITER=fast does not change what this builds — a caller that never asked for a selector must not have its durability contract moved out from under it. Use open_from_env when the selection is wanted.

⚠ One variable is read here, and it is not an arm: ZNIPPY_GIT_REDB_CACHE_BYTES (see crate::arms::redb_cache_bytes). It selects no implementation, changes no durability contract and leaves the bytes on disk identical — it caps redb’s page cache, whose own default is 1 GiB per database, of which this store opens two. The sentence above is about arms, and a memory ceiling is not one.

Source

pub fn open_with(root: &Path, account: &str, hash: GitHashKind) -> Result<Self>

Same, with an explicit oid width. sha256 repositories exist and the store does not get to assume otherwise; every oid it stores is GitHashKind::oid_len bytes wide and the tail refuses a mixed batch.

§What a reopen recovers, and why nothing had to be stored for it

A store opened over an objects.pack some earlier process was still indexing has to end up in the same state that process would have reached. §13.12’s indexed bit is what says so, and it is derived here rather than persisted: a pack is unabsorbed iff its extent is in the journal and its rows are not in the index, and both of those are already durable (see [Absorber::adopt_journal]). The packs that come back clear are put straight onto the account’s channel, so the drain finishes the interrupted work with no read having to ask — and until it does, every read falls back exactly as it would have before the crash. Slower, never wrong, across a restart as well as within one.

The bits are set before the channel is fed, which is the same ordering the ack path uses for the same reason: a read that arrives in between finds the bit clear and waits, never an index that does not mention the pack.

§What a reopen recovers of the DERIVED tables, and how that was fixed

[Derived::graph] and [Derived::trees] come back in full, for every pack, whether or not anything is re-queued. They are folded from §14’s exploded table ([Absorber::refold]), which is on disk, so a clean shutdown carries them across.

This was a live correctness hole until 2026-08-08 and it is recorded rather than quietly repaired. The graph was accumulated in RAM as packs were absorbed and stored nowhere; a store killed mid-absorb got it back because the interrupted pack was re-queued and re-resolved, but a store that shut down cleanly re-queued nothing and came back with an empty graph. MEASURED on oden 2026-08-08: 2687 rows and 0 of 551 commits, reachable on a commit returning that commit alone instead of its closure — a quiet wrong answer — and gc seeing an empty live set. The fix is the table, because the payloads the graph is folded from had no home; [tests::a_clean_shutdown_and_reopen_keeps_the_graph_and_reachable] is what proves it, over a real process that exits normally.

The exploded table is still droppable: if its row count is short of the objects table’s, [Absorber::adopt_journal] declares no pack absorbed and every one of them is re-queued and re-exploded. Absent means rebuild, never wrong (§13.12, applied unchanged to a derived table).

Also the default arms, and also no environment read — see open.

Source§

impl<S: ObjectIndex + 'static> GitStore<S>

Source

pub fn open_with_arms( root: &Path, account: &str, hash: GitHashKind, arms: StoreConfig, ) -> Result<Self>

The one constructor, with every arm named.

S picks the ObjectIndex layout at compile time; arms picks the ArchiveWrite and Gc implementations at run time. open and open_with are this function with StoreConfig::DEFAULT, which is why nothing about them moved.

arms.index is not consulted here — S is the index arm, and a value that disagreed with the type would be a second source of truth. open_from_env is the one place the two are tied together, and it does it by choosing S from the value.

use znippy_plugin_git::arms::{GcArm, StoreConfig, WriterArm};
use znippy_plugin_git::index_layout::PackedPayload;
use znippy_plugin_git::{GitHashKind, GitStore};

// A rebuildable import mirror: the ceiling writer, the packed payload
// layout, and an in-place compaction.
let arms = StoreConfig::DEFAULT
    .with_writer(WriterArm::Fast)
    .with_gc(GcArm::CompactInPlace);
let store = GitStore::<PackedPayload>::open_with_arms(
    std::path::Path::new("/srv/repo"),
    "rickard",
    GitHashKind::Sha1,
    arms,
)?;
Source

pub fn arms(&self) -> StoreConfig

The arms this store was built with. A description of what was constructed, not a switch — nothing on a serving path reads it.

Source

pub fn writer_name(&self) -> &'static str

The selected writer’s name, as it goes on a bench row.

Source

pub fn writer_durability(&self) -> &'static str

What the selected writer’s append actually promises. Printed next to a throughput, because a throughput without it is not a comparison.

Source

pub fn blobs_path(&self) -> &Path

Where the verbatim packs are.

Source

pub fn archive_path(&self) -> &Path

The sealed archive GC operates on.

Source

pub fn root(&self) -> &Path

Source

pub fn hash_kind(&self) -> GitHashKind

Source

pub fn index(&self) -> &ObjectReadStack<S>

The objects table, for a caller that wants the index directly (the bench does).

Source

pub fn object_count(&self) -> usize

Objects in the objects table.

Source

pub fn commit_count(&self) -> usize

Commits in the graph.

Source

pub fn content(&self, oid: Oid<'_>) -> Result<Option<(GitObjectKind, Vec<u8>)>>

One object’s resolved content — §14’s exploded table, in one lookup.

Not one of the twelve and deliberately not on GitOps: the twelve are the wire vocabulary and they hand back stored bytes (GitOps::get), which for a delta entry is a delta. This is the other question — what does this object contain — and it is the one §14’s table exists to make a point lookup instead of a chain walk.

It absorbs any pending pack first, for the same reason GitOps::has does: a store with durable bytes it has not indexed cannot answer “absent” without lying.

Source

pub fn exploded_stats(&self) -> ExplodedStats

§14’s table, counted: rows, rows written, reads it served, reads that fell through to re-deriving from the verbatim truth.

Applied output. Both content paths return identical bytes, so served against rederived is the only thing that can say which one ran.

Source

pub fn exploded_path(&self) -> &Path

Where §14’s table lives. Deleting this file while no store holds it is the sanctioned way to drop it; the next open re-explodes every pack.

Source

pub fn exploded_of_kind( &self, kind: GitObjectKind, ) -> Result<Vec<(Vec<u8>, Vec<u8>)>>

Every live row of one kind, with its payload — what the graph fold reads, exposed so a measurement can ask the table how many bytes it holds rather than dividing a file size by what the pack claimed.

Source

pub fn graph_snapshot(&self) -> Vec<CommitNode>

The commit graph as it stands, generations included. A snapshot: the caller holds no lock and the next fold replaces it wholesale.

Source

pub fn unindexed_packs(&self) -> usize

Packs whose objects are not in the index yet — §13.12’s bit, counted.

Source

pub fn indexer(&self) -> Arc<AccountIndexer>

This account’s indexer: the thread that drains the channel a push queues on. is_indexed(pack_id) on it is §13.12’s bit at the pack level, and it is set only once that pack’s objects are in the index.

Source

pub fn wait_indexed(&self)

Block until the background drain has absorbed everything this store has pushed. What a maintenance tick, a seal or a test uses instead of racing it.

Source

pub fn absorb_pending(&self) -> Result<usize>

The indexer’s half of the split, and not one of the twelve.

Runs on the account indexer’s worker (§13.9), and on a read that arrives before that worker got there — the same function either way, which is why there is no such thing as a background index that behaves differently from the fallback one.

Source

pub fn oids_in_extent(&self, extent: Extent) -> Result<Vec<Vec<u8>>>

The object ids a stored pack introduced, and not one of the twelve.

GitOps::put_pack cannot answer this and never will: the walk it does reads pack entries, and a pack entry does not carry its oid — that is why crate::pack_walk::Closure can only name the REF_DELTA bases and says so. The oids exist once the indexer has absorbed the pack, so this absorbs first rather than racing the drain, and is therefore the one caller that deliberately pays §13.9’s deferral back.

Only call it when something needs the set. A push whose policy has no gate over objects must not pay this, which is the whole reason it is a separate call and not a field on TxId.

The extent is the identity: §14 stores a pack verbatim as one contiguous append, so “introduced by this pack” and “indexed at an offset inside this pack’s extent” are the same set, with no pack column to keep in step. Bases the push deltas against live in other extents and are excluded by construction — which is what a thin push needs, and what reading a written .idx back could never do.

Cost is one pass over the index, so it is O(objects in the repository), not O(objects pushed). Recorded rather than hidden: a per-pack ordinal range would make it O(pushed), and needs the indexer to keep one.

Source

pub fn emit_set( &self, oids: &[Oid<'_>], ofs_delta_ok: bool, thin_haves: Option<&[Oid<'_>]>, ) -> Result<Vec<EmitEntry>>

Everything needed to emit a pack for EXACTLY oids — nothing added, nothing dropped.

Not one of the twelve. It sits beside absorb_pending for the same reason those do: it is about how storage is shaped, not about what a caller stores or reads.

§🔴 It used to add the delta bases, and that shipped broken clones

Until 2026-08-11 this closed the request over its delta bases: an OFS_DELTA names its base by position, so a base outside the set cannot be encoded, and adding the base looked like the answer that is always safe. It is not, and the failure is silent on the server:

git clone --bare --single-branch --branch base <url>
    fatal: did not receive expected object 8601ec33920b7d701c9887fa04916501136d6e90
    fatal: fetch-pack: invalid index-pack output
event="git.upload_pack.served" objects=33773      ← the server said SUCCESS

Measured on the h2h-linear-sha1-2048c-1024f-16k fixture: base reaches 31 805 objects, the delta-base closure added 1 968 more — 33 773, exactly the count the server logged — and 507 of those additions were trees. A tree that is not reachable from the wants drags its own children in as a requirement, because git index-pack --check-self-contained-and-connected (which is what a clone runs, and which sets strict) walks every received object’s links and then demands each one exist. Those 507 trees named 204 objects the pack did not contain. 8601ec33… is one of them.

So a delta base pulled into a narrowed request is not free: it satisfies the pack format and violates connectivity. Closing over the added bases’ children in turn does terminate — the same fixture reaches a fixed point at 34 534 objects — but it is an over-send by construction, and on a --filter=blob:none fetch it adds back exactly the blobs the filter excluded, which is the defect emit_pack_emits_exactly_the_set_it_is_given_and_never_its_closure exists to forbid.

§What it does instead: the base decides how the entry is encoded

The set is never widened. For each requested object:

  • base inside the request → the stored entry is copied byte for byte, which is the whole point of this engine and is what a full clone does for every single object;
  • base outside the request → the entry is rebuilt from §14’s resolved content and marked EmitEntry::recompressed — as a computed delta against a base the request does carry where one can be found (see Self::in_request_chain_ancestor and crate::delta), and whole where it cannot.

That is what stock pack-objects does — it reuses a stored delta only when the base is also being packed — and it keeps the receipt a fact: copied and recompressed add up to the objects written, and recompressed is 0 for a whole-repository clone because such a selection contains every base. The cost lands on the boundary a narrowed request cuts through and nowhere else: 1 968 of 33 773 entries, 5.9 %, on the fixture above.

ofs_delta_ok is the client’s ofs-delta capability. An old client that cannot parse one is not refused any more: the entry is re-headed as a REF_DELTA naming its base by oid, carrying the same compressed delta payload, so that arm stays a copy too. Only a base outside the request costs a rebuild.

§thin_haves — the boundary entry that does NOT have to be rebuilt

Some(tips) says the client consented to a thin pack and the negotiation found common ground; [crate::serve::GitStore::emit_oids] is where the two halves are required together, because either alone is a failed fetch rather than a smaller one. Given it, a base outside the request is checked against the client's own objects, and one the client already holds is named by oid in a REF_DELTA that carries the stored delta stream unchanged. That entry counts as copied, because it is: nothing was inflated and nothing was deflated.

None — a clone, or a client that did not ask for a thin pack — behaves exactly as before, and a full clone is byte-identical either way, because a whole-repository request has no base outside itself for any of this to apply to.

It does nothing for the clone half of the boundary cost, and cannot: a clone’s receiver holds nothing, so there is no external base to name. That half is the arm below it — crate::delta computes a delta against something the pack does carry, which is the only move left when there is no receiver to lean on. The two do not overlap: thin_haves reaches a boundary entry first and costs nothing when it fires, so a fetch stays a pure copy and only a clone pays for a computed delta.

§An oid this repository does not hold is REFUSED

It used to be skipped, which is the same silent-under-send shape as the closure defect above wearing a different hat: N asked, N-1 emitted, PackStats reporting success, and the client the first to know. gunnar’s in-memory arm has always refused it, so the skip also meant the two engines behind one contract disagreed about what a missing object means. A partial-clone filter does not depend on the skip and cannot: a filter removes oids from the selection, it never adds one the store lacks.

§Cost

O(objects requested). The previous shape read oids_in_order() and lookup_batch over the whole repository on every call, to build an offset→oid map it needed only for the closure; with no closure the selection’s own offsets answer “is this base in the request”, so that scan is gone and no reverse index has to exist.

Source§

impl<S: ObjectIndex + 'static> GitStore<S>

Source

pub fn emit_oids( &self, oids: &[Oid<'_>], have: &[Oid<'_>], caps: &Caps, out: &mut dyn Write, ) -> Result<PackStats>

Emit a packfile for an explicit object set, closed over its delta bases — and over nothing else.

This is the whole of GitServe::emit_pack as of 2026-08-10: that method is this call and a discarded have. It was written as the inherent escape route for a trait method that closed over its input, so that a caller told “walk it yourself” by GitServe::select’s Ok(None) had somewhere to put the set it had walked. The trait method stopped closing over its input, so the escape route and the front door are now the same door. It stays inherent because the gix arm needs no equivalent and the contract does not grow a method for it.

§🔴 No closure at all any more, and that is the 2026-08-11 fix

GitStore::emit_set used to add a delta’s base when the base fell outside the request, on the reasoning that an OFS_DELTA names its base by position and so cannot be encoded without it. That reasoning is correct about the pack format and wrong about the clone: a base pulled in is an object the client did not ask for, and if it is a tree it arrives owing children the pack does not contain. git index-pack --check-self-contained-and-connected — what a clone runs — then dies with did not receive expected object, while this server logs git.upload_pack.served. See emit_set for the measured numbers.

So the set is now exactly the caller’s, and the base decides how each entry is encoded rather than what the pack contains: base inside the request, copy the stored bytes; base outside it, rebuild the object whole. That is stock pack-objects’ rule, and recompressed counts the second case honestly instead of being a literal zero.

§Three steps, all of them znippy’s own
  1. GitStore::emit_set reads each requested entry and decides copy or rebuild, never adding an object;
  2. crate::pack_walk::topological_order puts every base before the delta naming it, which backwards distances require;
  3. crate::pack_walk::emit_pack re-encodes each entry header — the one thing that must change, because a distance is relative to a position in the input pack — and streams every payload byte for byte.

copied + recompressed == objects always, and for a whole-repository clone recompressed is 0 because such a request contains every base. Both are counted off what happened rather than asserted: a byte count and a wall clock cannot tell a pack-copy from a re-deflate, and both pass index-pack --strict.

Trait Implementations§

Source§

impl<S: ObjectIndex + 'static> GitOps for GitStore<S>

Source§

fn put(&self, pack: &[u8], refs: &[RefUpdate]) -> Result<TxId>

One push, and the order inside it is the contract.

  1. the pack’s bytes, durable — put_pack
  2. then the refs that point into them — put_refs

Not interchangeable, and it is the same argument SafeWriter makes one level down about the blob and its journal row: a crash between the two leaves objects nobody points at, which a GC reclaims. The reverse order leaves a ref pointing at objects that are not there, which is a corrupt repository that no later pass can repair.

Source§

fn put_pack(&self, bytes: &[u8]) -> Result<TxId>

CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW

The ack path. Walk, check, store verbatim, ack, then queue the index.

  1. Walk every entry (crate::pack_walk). Not optional and not an extra pass: a pack entry has no length field, so finding the entry boundaries is how the pack gets split at all.
  2. The closure check, free from that walk (§13.7). Every OFS_DELTA base must land on an entry boundary in this pack — answered from the boundary set the walk just produced, consulting no index. Only a REF_DELTA base that is outside the pack is looked up, and that one reads objects.oid and nothing else, exactly as §13’s table says receive-pack does. A pack that fails is refused before a byte is stored.
  3. The bytes, verbatim. SafeWriter::append writes the caller’s buffer at its own address — no re-compression, no re-encoding, no re-framing, not one copy in userspace. The client’s own deflate and every delta chain survive, which is what makes a later clone a byte-range copy instead of a re-pack (§14).
  4. Durable before returning: the blob is fsynced, then the journal row that references it, then that is fsynced. znippy’s hot.rs ordering, so a crash between the two leaves orphan bytes nobody points at rather than a dangling reference.
  5. Then the index job goes on the channel and this returns. Nothing that could be done later is done here: no oid is computed, no table is built, no chain is resolved.
Source§

fn put_refs(&self, updates: &[RefUpdate]) -> Result<TxId>

CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW

One ref transaction: one Arrow IPC frame, fsynced.

The frame boundary is the transaction (see crate::pushlog) — three branches in one push are three rows in one batch, and they either all land or none do. There is no lock file and no second journal.

Every target is checked against the index first. A ref that points at an object the repository does not have is a corrupt repository, and it is refused here rather than written and discovered later. A deletion has no target and is not checked.

Source§

fn get(&self, oid: Oid<'_>) -> Result<Option<Stored>>

CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW

The object’s stored bytes: one index lookup and one pread of the extent, with no decode of any kind in between.

What comes back is the pack entry exactly as the client sent it, which for a delta is a delta. Stored::obj_type says which, so the bytes cannot be mistaken for the object’s content — §14 makes the verbatim bytes the truth and the resolved object a derived cache, and that cache does not exist yet, so handing back delta bytes labelled “the object” would be the one thing this crate refuses everywhere: a wrong answer where an honest one was available.

Source§

fn has(&self, oid: Oid<'_>) -> Result<bool>

CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW

The negotiation call: have sends up to a thousand of these and most of them miss. It reads objects.oid and no other column.

It takes the serial path, not lookup_batch with one element: that is 1.7× slower (818 ns against 482), and lookup_path is where that decision is written down.

§Why this returns Result<bool> and not bool

The signature changed deliberately. A store with a pack whose bytes are durable but whose objects are not indexed yet cannot answer “no” — the object may be in that pack. So it absorbs the pack first and, if that fails, says so. A bool could only have lied, and a wrong “absent” during negotiation makes a client send nothing and lose data.

Source§

fn size(&self, oid: Oid<'_>) -> Result<Option<u64>>

CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW

The object’s post-resolution size — what it inflates to once its delta chain is applied. That is the fact git’s own .idx and .rev together cannot answer, and it is why the quota gate is index-only here.

Source§

fn extents(&self, oids: &[Oid<'_>]) -> Result<Vec<Option<Extent>>>

CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW

The wire path, and batch by construction: for each oid the two facts a clone needs — offset and len — and no others, so a byte-range copy out of the verbatim pack can start immediately.

out[i] answers oids[i]. One oid takes the serial path (lookup_path); the batch path saturates at BATCH_SATURATES_AT oids, so a larger batch is passed through whole rather than split — splitting would cost a pass and buy nothing.

Source§

fn refs(&self) -> Result<Vec<RefRow>>

CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW

The whole ref namespace, tags peeled — the ref advertisement, and ls-refs after a prefix filter. Name-sorted, because the log folds into a BTreeMap and ls-refs wants a prefix scan.

The cost is a scan of a structure sized by pushes, not by repository size.

§HEAD is not in here, and that is the contract

Changed 2026-08-10, and it is a fix rather than a preference. The other backend’s iter() “walks refs/ (loose and packed) and deliberately excludes the pseudo-refs such as HEAD, which is exactly the contract every other backend honours”. This one did not, so the two arms disagreed about whether HEAD is a row — and the conformance suite could not see it, because it never exercised HEAD at all.

The reason HEAD cannot be a row is a type constraint, not taste: a name type that admits HEAD also admits MERGE_HEAD and FETCH_HEAD, so a row stream carrying pseudo-refs means either widening the name type or filtering at every consumer. It gets an accessor pair instead — GitServe::head and set_head — and this is the one filter, in the one place.

Nothing else changes: the ref log still stores HEAD (a push writes it like any other row), live_set still reaches it because it reads ref_state directly, and the filter is one !=.

Source§

fn update_ref( &self, name: &str, old: Option<Oid<'_>>, new: Option<Oid<'_>>, ) -> Result<TxId>

CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW

Compare-and-swap, in the same log a push writes its refs to — one mechanism, not a second one for single updates.

old is what the caller believes the ref is: None means it must not exist (a create), Some(oid) means it must be exactly this. new of None deletes. A mismatch names both values and writes nothing.

The read-compare-append is serialised on the store’s ref gate. Without it two CAS calls could both read the old value and both append, and the second would silently overwrite an update it had compared against successfully — the classic lost update, and the only thing a CAS is for.

Source§

fn put_refs_cas(&self, edits: &[RefCas<'_>]) -> Result<TxId>

CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW

git push --atomic: every edit or none, in one Arrow IPC frame.

The two halves come from different places and neither is reimplemented:

  • the batch atomicity is the frame. crate::pushlog makes the frame boundary the transaction — three branches in one push are three rows in one batch and they either all land or none do. There is no lock file and no second journal, so there is no partial-apply state to recover from;
  • the compare half is checked here, against one snapshot, before a single RefUpdate is built.

It goes out through put_refs rather than straight to the log, so the dangling-target refusal and the frame format are the same code a push uses (LAW 5): an atomic batch cannot create a ref pointing at an object the repository does not have, which a second writer here would eventually have allowed.

§S-023, and why the check is a whole pass of its own

Every expectation is evaluated before any of them is applied, and the loop deliberately does not fuse with the one that builds the updates. The defect it exists for is gix’s: a backend that short-circuits an edge whose new value already equals the current one never evaluates the expectation the caller wrote, so an old: Nonemust not exist — becomes a silent success and “exactly one creator wins” stops being true. This log has no such short-circuit, but the ordering is what makes that irrelevant rather than lucky, and a future optimisation that adds one cannot break it from here.

§What this arm cannot raise, stated rather than hidden

RefRejection::Locked never comes out of this backend. The read-compare-append is serialised on the store’s ref gate, which a second writer blocks on rather than failing against — so contention here is a wait, never a rejection. The variant is in the contract because the gix arm, which takes real per-ref lock files, raises it. A poisoned gate stays an ordinary error: it is a fault, not the transient thing a caller retries.

Source§

fn reachable(&self, want: &[Oid<'_>], have: &[Oid<'_>]) -> Result<Vec<Vec<u8>>>

CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW

Selection: want minus have, as an andnot of roaring bitmaps.

Every returned oid is an object — commit, tree and blob — because the bitmaps are built by crate::reach::build_reach, which walks the trees. A want that names a commit contributes that commit’s whole closure; a want that names a tag or a blob contributes itself.

A have this repository does not know contributes nothing: the safe direction is to send more, never less.

The bitmaps are over the store’s own ordinal space, rebuilt by the same fold that rebuilds them, never over the Arrow projection’s ordinals — an crate::index_layout::IndexRow::ordinal is a row address within one projection generation, so a bitmap over those would address different objects after any rebuild, silently.

§The per-oid Vec is paid HERE and nowhere else on the serving path

The answer is computed flat (see GitStore::reachable_raw); this splits it back out because the eleven say Vec<Vec<u8>> and this method’s callers are maintenance ones — a GC live set, a conformance harness — not the serving path. crate::serve’s select and emit_pack take the flat form directly.

Source§

fn gc(&self) -> Result<GcReport>

Garbage collection, in the order §13.20 fixes:

  1. compute reachability — every object reachable from every ref, over the same bitmaps reachable uses;
  2. drop the dead rows from the index — the one operation that is not append-only, and the projection is rebuilt inside it;
  3. then base znippy’s compaction, through crate::gc::NewGeneration (link, compact, verify, rename, unlink last).

That order is why base znippy needs no new method: by the time compact_archive runs, “live” already means what git means.

A repository with no ref pointing at anything is refused rather than emptied.

Appended, not reworded: step 1b, the journal. Between the live set and the drop there is now one more durable act. §13.12’s indexed bit is derived on open as extent in the journal, rows not in the index, and dropping every row of a pack produces exactly that state — so before this existed the next open re-queued the pack and every object this GC had just decided was dead came back. GitStore::retire_dead_packs appends a tombstone naming each all-dead pack, and it runs before drop_dead_rows on purpose: killed before it, the rows are still there and the GC simply did not happen; killed after it, the pack can never be re-queued whether the rows went or not. A partly dead pack keeps rows and is never tombstoned, so nothing about it changes.

Step 2 also refolds what the two tables feed — the commit graph, the tree payloads, the ordinal space, the bitmaps — because a derivation that still names a dropped oid is wrong rather than stale. That is inside drop_dead_rows, so any caller of it gets it.

Source§

impl<S: ObjectIndex + 'static> GitServe for GitStore<S>

Source§

fn read(&self, oid: Oid<'_>) -> Result<Option<(ObjType, Vec<u8>)>>

The object, inflated and delta-resolved — §14’s exploded table, one point lookup.

This is the question GitOps::get deliberately does not answer, and GitStore::content has answered it since §14 landed; the trait method is that call plus the kind narrowing. Two paths under it, returning identical bytes and distinguishable only by crate::exploded::ExplodedStats: the table, or a re-derivation from the verbatim truth when the table has been dropped or has not caught up.

The returned ObjType can never be a delta — see [resolved_type].

Source§

fn header(&self, oid: Oid<'_>) -> Result<Option<(ObjType, u64)>>

Kind and post-resolution size, without the payload.

The size is a column, and the right column: uncompressed_size is written after delta application, so it is the fact a .idx and a .rev together cannot answer, and it is read rather than measured.

The kind is not a column — obj_type is the entry’s type in the pack — so for the common case (a non-delta entry) this touches nothing else, and for a delta it walks the chain’s headers through GitStore::resolved_type_at. Still no inflate and still no payload.

Source§

fn sizes(&self, oids: &[Oid<'_>]) -> Result<Vec<Option<u64>>>

Post-resolution sizes in bulk: one index pass, no chain touched.

The half of header that is already data. On the h2h fixture one clone paid 34 124 per-object header walks, every one of which this collapses into a single lookup_batch, to establish that a repository whose largest blob is 16 KiB holds nothing over the 1 GiB default ceiling.

§The preamble is not optional

This reaches GitStore::index directly, so it absorbs pending index work itself. Without it a pack that landed since the last drain reads as absent, and an absent size that a caller treated as a pass would silently skip the ceiling for exactly the objects a push had just introduced. None at a position is “unknown, go ask header and never “fine”.

Source§

fn head(&self) -> Result<Option<RefRow>>

HEAD, which is not a row in GitOps::refs.

The gix arm’s iter() “walks refs/ (loose and packed) and deliberately excludes the pseudo-refs such as HEAD, which is exactly the contract every other backend honours” — and as of this change so does this arm (see GitOps::refs). The reason is a type constraint rather than taste: a name type that admits HEAD also admits MERGE_HEAD and FETCH_HEAD, so putting pseudo-refs in the row stream means either widening the name type or filtering at every consumer.

So it is read here, off the same fold GitOps::refs reads, rather than from a second place that could disagree with it. None for a repository that has never pointed one — an empty repository has no HEAD.

Source§

fn set_head(&self, target: &str) -> Result<TxId>

Point HEAD, through the same log a push writes to.

§The two shapes of a target, and how they are told apart

HEAD is symbolic in every repository anyone serves — ref: refs/heads/main — and detached in the one case git also supports. The contract’s parameter is one &str for both, so:

  • a target starting with refs/ is a symbolic ref, written as one;
  • anything else must be a hex oid of exactly this store’s width, written as a direct target.

The two cannot be confused: a ref name and a 40- or 64-character hex string are disjoint. Anything that is neither is refused rather than guessed at — a HEAD pointing at a name nothing resolves is an unclonable repository, and it is cheaper to say so here.

A symbolic target is deliberately not checked for existence. git init points HEAD at refs/heads/main before that branch exists, and refusing it would make an empty repository unrepresentable. A direct target is checked, by GitOps::put_refs, because a detached HEAD naming an absent object is dangling in exactly the sense that refuses.

Source§

fn emit_pack( &self, objects: &[Oid<'_>], have: &[Oid<'_>], caps: &Caps, out: &mut dyn Write, ) -> Result<PackStats>

Emit a packfile containing exactly objects.

One line of delegation to GitStore::emit_oids, and the absence of a second line is the contract.

§🔴 This must NOT compute a closure. Fixed 2026-08-10.

Until that date this method opened with self.select(objects, have)? and emitted that — it closed over its own input. The caller’s set came back larger than it went in, and the caller is upload-pack holding selection.objects: already post-filter, post-shallow, post-include-tag and deliberately not closed. So a --filter=blob:none, --filter=tree:0 or --depth=N fetch was served exactly the objects it had asked to be left out.

The reason it needed a rename and a guard rather than a shrug is that nothing we own could see it: the over-sent pack passes git index-pack --strict and git fsck, the clone succeeds, the exit code is zero, and the client just silently receives more than it asked for. P-027’s shape — a change no test can see. Pinned now by emit_pack_emits_exactly_the_set_it_is_given_and_never_its_closure, which asserts a count (seen RED at 4 against an expected 1).

It had a second, louder failure mode too, and it is the one that proves the two questions were never the same question: select’s tip check declines a want that is not a commit in the graph, so a selection containing a tree — which every real one does — could not be emitted at all.

§Where the closure went

Nowhere. select still owns the close-over-tips half and still answers Ok(None) when its projection cannot cover a request; that hatch is untouched and still load-bearing. The two halves are now cleanly separated: what to send is asked of select, send exactly this is asked of here. A caller that select told to walk the graph itself hands the walked set straight back to this method.

There is no closure left on this path at all: since 2026-08-11 the base decides how an entry is encoded, never what the pack contains.

§have is READ, and only for thin-pack base selection

The contract defines have as the negotiated common tips, “used for thin-pack base selection, never to derive membership”, and that is exactly and only what this arm does with it. Paired with caps.thin, it lets a boundary entry go out as a REF_DELTA naming a base the receiver already holds — carrying the stored delta stream unchanged — instead of being rebuilt whole. See GitStore::client_bases for the closure and for why the caller has to be able to stand behind the voucher.

It is deliberately not repurposed into an exclusion set. Deriving membership from have is precisely the closure this method stopped doing, in the opposite direction: it would remove objects the caller’s finished selection had already decided to send, which is an under-send, which is the failure that exits zero. The narrowing below changes the encoding of an entry and never whether it is emitted.

Source§

fn select(&self, want: &[Oid<'_>], have: &[Oid<'_>]) -> Result<Option<ReachSet>>

want minus have, out of §13’s bitmaps, with no object opened — plus the two facts the caller cannot recompute cheaply.

§The refusal is the part that matters

GitOps::reachable treats a want it has no bitmap for as contributing itself and nothing else. That is the safe direction for its own caller — live_set over-keeps, and over-keeping is harmless — and a silent under-send for this one. A clone served that way exits zero with one object per branch.

So every tip is checked against the store’s commit graph first, and a tip that is not in it declines the whole request.

§🔴 “In the graph” and “has a bitmap” are NO LONGER the same fact

They were, and this paragraph used to say so, because the live table was built with no commit cap precisely so that they would be. That cost 530 218 allocations a fetch (see crate::git_ops’s LIVE_REACH_COMMITS), and since 2026-08-14 the table is sampled at 512 like the sealed archive’s.

The check below is unchanged and is still exact, but it is now exact for a different reason. It is “in the graph” that matters, not “has a bitmap”: [crate::reach::accumulate] walks a bitmapless commit down to the first bitmapped one behind it, so every graph commit is answerable, with or without a bitmap of its own. A tip that is not a graph row is still refused — there is nothing to walk — and that is the case this check exists for.

Ok(None) is therefore reachable in three ways, and all three are the same statement — the projection does not cover this request: an empty graph (which a clean restart produces), a tip that is not a commit in it, and a graph row whose oid does not parse.

§The selection is NOT re-ordered, and that is measured

reachable answers in ordinal order, which is oid-lexicographic and therefore unrelated to where the bytes are. Sorting by archive offset so a delta’s base precedes it was written and could not be made to fail: measured 2026-08-08 on a 154-object fixture — with the sort, without it, and with the whole selection deliberately reversed, all three give copied=154 recompressed=0 and the identical object graph, because crate::pack_walk::topological_order orders the emission itself. The sort was a batch index lookup per request buying a property the emitter already owns (LAW 5), and its absence is something no guard can see (LAW 2). It is gone.

Auto Trait Implementations§

§

impl<S = OneTableFourColumns> !Freeze for GitStore<S>

§

impl<S = OneTableFourColumns> !RefUnwindSafe for GitStore<S>

§

impl<S = OneTableFourColumns> !UnwindSafe for GitStore<S>

§

impl<S> Send for GitStore<S>
where Arc<Absorber<S>>: Send,

§

impl<S> Sync for GitStore<S>
where Arc<Absorber<S>>: Sync,

§

impl<S> Unpin for GitStore<S>
where Arc<Absorber<S>>: Unpin,

§

impl<S> UnsafeUnpin for GitStore<S>
where Arc<Absorber<S>>: UnsafeUnpin,

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.