pub struct GitStore<S: ObjectIndex = OneTableFourColumns> { /* private fields */ }Expand description
One repository’s git store.
| on disk | what it is | who owns the format |
|---|---|---|
<root>/objects.pack | the verbatim pack bytes, appended | [SafeWriter] |
<root>/objects.pack.journal | the durable extent rows | [SafeWriter] |
<root>/objects.tail | the un-sealed index tail | ObjectReadStack (redb) |
<root>/objects.exploded | §14’s resolved content, droppable | ExplodedArchive (one Arrow IPC table) |
<root>/refs.log | the ref transaction log | RefLog (Arrow IPC frames) |
<root>/<name>.znippy | the sealed archive GC compacts | base znippy |
Nothing in that table is new. The store is the composition, not a fifth format.
§Which implementation of each trait, and how a caller picks
Three of the four boxes in the diagram at the top of this file have more than
one implementation. StoreConfig is the choice, crate::arms documents
what each arm promises, and the dispatch is split deliberately:
| trait | selected by | dispatch | cost |
|---|---|---|---|
ObjectIndex | the type parameter S | static | none — the hot path is monomorphised |
ArchiveWrite | StoreConfig::writer | dyn, inside PushPath as it already was | one virtual call per pack append, not per object |
Gc | StoreConfig::gc | dyn | one virtual call per gc() run |
S defaults to OneTableFourColumns, so GitStore still names exactly
what it named before and every existing caller compiles unchanged.
Implementations§
Source§impl<S: ObjectIndex + 'static> GitStore<S>
impl<S: ObjectIndex + 'static> GitStore<S>
Sourcepub fn seal(&self) -> Result<Vec<ReservedSection>>
pub fn seal(&self) -> Result<Vec<ReservedSection>>
CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW
Fold the live logs into the reserved Arrow sections an archive carries:
__gunnar_refs__, __gunnar_graph__, __gunnar_reach__.
The twelfth method, and the one that is NOT on GitOps: it returns
znippy_common::ReservedSection (Arrow RecordBatch payloads), which a
gix backend has no analog for — so it is inherent on the concrete store,
which is how gunnar already calls it (never through the trait).
It absorbs first, because a section built while a pack is still
un-indexed would be silently incomplete, and a silently incomplete
index is worse than none (the same argument [crate::lib] makes about not
wiring GitIndexBuilder into the CLI). Every generation number in the
graph section is recomputed by that fold.
…and then it writes the archive. Until 2026-08-08 it only returned
the sections and nothing on any path created
archive_path at all — so gc()’s last step
died on stat repository.znippy: No such file or directory, having
already done its first four. Both Gc implementations
compact an archive that exists; this is what makes one exist.
seal_generation_zero has
the layout argument.
The sections are still returned, and they are the same values that were
sealed rather than a second derivation of them — ReservedSection is
Clone for exactly that reason.
Source§impl GitStore<OneTableFourColumns>
impl GitStore<OneTableFourColumns>
Sourcepub fn open(root: &Path, account: &str) -> Result<Self>
pub fn open(root: &Path, account: &str) -> Result<Self>
Open (creating) the store for one repository under root.
The default arms, and it reads no environment to get them. An
operator who exported ZNIPPY_GIT_WRITER=fast does not change what this
builds — a caller that never asked for a selector must not have its
durability contract moved out from under it. Use
open_from_env when the selection is wanted.
⚠ One variable is read here, and it is not an arm:
ZNIPPY_GIT_REDB_CACHE_BYTES (see
crate::arms::redb_cache_bytes). It selects no implementation,
changes no durability contract and leaves the bytes on disk identical —
it caps redb’s page cache, whose own default is 1 GiB per database, of
which this store opens two. The sentence above is about arms, and a
memory ceiling is not one.
Sourcepub fn open_with(root: &Path, account: &str, hash: GitHashKind) -> Result<Self>
pub fn open_with(root: &Path, account: &str, hash: GitHashKind) -> Result<Self>
Same, with an explicit oid width. sha256 repositories exist and the store
does not get to assume otherwise; every oid it stores is
GitHashKind::oid_len bytes wide and the tail refuses a mixed batch.
§What a reopen recovers, and why nothing had to be stored for it
A store opened over an objects.pack some earlier process was still
indexing has to end up in the same state that process would have reached.
§13.12’s indexed bit is what says so, and it is derived here rather
than persisted: a pack is unabsorbed iff its extent is in the journal
and its rows are not in the index, and both of those are already durable
(see [Absorber::adopt_journal]). The packs that come back clear are put
straight onto the account’s channel, so the drain finishes the interrupted
work with no read having to ask — and until it does, every read falls back
exactly as it would have before the crash. Slower, never wrong, across
a restart as well as within one.
The bits are set before the channel is fed, which is the same ordering the ack path uses for the same reason: a read that arrives in between finds the bit clear and waits, never an index that does not mention the pack.
§What a reopen recovers of the DERIVED tables, and how that was fixed
[Derived::graph] and [Derived::trees] come back in full, for every
pack, whether or not anything is re-queued. They are folded from §14’s
exploded table ([Absorber::refold]), which is on disk, so a clean
shutdown carries them across.
This was a live correctness hole until 2026-08-08 and it is recorded
rather than quietly repaired. The graph was accumulated in RAM as packs
were absorbed and stored nowhere; a store killed mid-absorb got it back
because the interrupted pack was re-queued and re-resolved, but a store
that shut down cleanly re-queued nothing and came back with an empty
graph. MEASURED on oden 2026-08-08: 2687 rows and 0 of 551 commits,
reachable on a commit returning that commit alone instead of its
closure — a quiet wrong answer — and gc seeing an empty live set. The
fix is the table, because the payloads the graph is folded from had no
home; [tests::a_clean_shutdown_and_reopen_keeps_the_graph_and_reachable]
is what proves it, over a real process that exits normally.
The exploded table is still droppable: if its row count is short of
the objects table’s, [Absorber::adopt_journal] declares no pack
absorbed and every one of them is re-queued and re-exploded. Absent means
rebuild, never wrong (§13.12, applied unchanged to a derived table).
Also the default arms, and also no environment read — see
open.
Source§impl<S: ObjectIndex + 'static> GitStore<S>
impl<S: ObjectIndex + 'static> GitStore<S>
Sourcepub fn open_with_arms(
root: &Path,
account: &str,
hash: GitHashKind,
arms: StoreConfig,
) -> Result<Self>
pub fn open_with_arms( root: &Path, account: &str, hash: GitHashKind, arms: StoreConfig, ) -> Result<Self>
The one constructor, with every arm named.
S picks the ObjectIndex layout at compile time; arms picks the
ArchiveWrite and Gc
implementations at run time. open and
open_with are this function with
StoreConfig::DEFAULT, which is why nothing about them moved.
arms.index is not consulted here — S is the index arm, and a
value that disagreed with the type would be a second source of truth.
open_from_env is the one place the two are tied together, and it does
it by choosing S from the value.
use znippy_plugin_git::arms::{GcArm, StoreConfig, WriterArm};
use znippy_plugin_git::index_layout::PackedPayload;
use znippy_plugin_git::{GitHashKind, GitStore};
// A rebuildable import mirror: the ceiling writer, the packed payload
// layout, and an in-place compaction.
let arms = StoreConfig::DEFAULT
.with_writer(WriterArm::Fast)
.with_gc(GcArm::CompactInPlace);
let store = GitStore::<PackedPayload>::open_with_arms(
std::path::Path::new("/srv/repo"),
"rickard",
GitHashKind::Sha1,
arms,
)?;Sourcepub fn arms(&self) -> StoreConfig
pub fn arms(&self) -> StoreConfig
The arms this store was built with. A description of what was constructed, not a switch — nothing on a serving path reads it.
Sourcepub fn writer_name(&self) -> &'static str
pub fn writer_name(&self) -> &'static str
The selected writer’s name, as it goes on a bench row.
Sourcepub fn writer_durability(&self) -> &'static str
pub fn writer_durability(&self) -> &'static str
What the selected writer’s append actually promises. Printed next to a
throughput, because a throughput without it is not a comparison.
Sourcepub fn blobs_path(&self) -> &Path
pub fn blobs_path(&self) -> &Path
Where the verbatim packs are.
Sourcepub fn archive_path(&self) -> &Path
pub fn archive_path(&self) -> &Path
The sealed archive GC operates on.
pub fn root(&self) -> &Path
pub fn hash_kind(&self) -> GitHashKind
Sourcepub fn index(&self) -> &ObjectReadStack<S>
pub fn index(&self) -> &ObjectReadStack<S>
The objects table, for a caller that wants the index directly (the
bench does).
Sourcepub fn object_count(&self) -> usize
pub fn object_count(&self) -> usize
Objects in the objects table.
Sourcepub fn commit_count(&self) -> usize
pub fn commit_count(&self) -> usize
Commits in the graph.
Sourcepub fn content(&self, oid: Oid<'_>) -> Result<Option<(GitObjectKind, Vec<u8>)>>
pub fn content(&self, oid: Oid<'_>) -> Result<Option<(GitObjectKind, Vec<u8>)>>
One object’s resolved content — §14’s exploded table, in one lookup.
Not one of the twelve and deliberately not on GitOps: the twelve are
the wire vocabulary and they hand back stored bytes (GitOps::get),
which for a delta entry is a delta. This is the other question — what
does this object contain — and it is the one §14’s table exists to make
a point lookup instead of a chain walk.
It absorbs any pending pack first, for the same reason
GitOps::has does: a store with durable bytes it has not indexed cannot
answer “absent” without lying.
Sourcepub fn exploded_stats(&self) -> ExplodedStats
pub fn exploded_stats(&self) -> ExplodedStats
§14’s table, counted: rows, rows written, reads it served, reads that fell through to re-deriving from the verbatim truth.
Applied output. Both content paths return identical bytes, so served
against rederived is the only thing that can say which one ran.
Sourcepub fn exploded_path(&self) -> &Path
pub fn exploded_path(&self) -> &Path
Where §14’s table lives. Deleting this file while no store holds it is the sanctioned way to drop it; the next open re-explodes every pack.
Sourcepub fn exploded_of_kind(
&self,
kind: GitObjectKind,
) -> Result<Vec<(Vec<u8>, Vec<u8>)>>
pub fn exploded_of_kind( &self, kind: GitObjectKind, ) -> Result<Vec<(Vec<u8>, Vec<u8>)>>
Every live row of one kind, with its payload — what the graph fold reads, exposed so a measurement can ask the table how many bytes it holds rather than dividing a file size by what the pack claimed.
Sourcepub fn graph_snapshot(&self) -> Vec<CommitNode>
pub fn graph_snapshot(&self) -> Vec<CommitNode>
The commit graph as it stands, generations included. A snapshot: the caller holds no lock and the next fold replaces it wholesale.
Sourcepub fn unindexed_packs(&self) -> usize
pub fn unindexed_packs(&self) -> usize
Packs whose objects are not in the index yet — §13.12’s bit, counted.
Sourcepub fn indexer(&self) -> Arc<AccountIndexer> ⓘ
pub fn indexer(&self) -> Arc<AccountIndexer> ⓘ
This account’s indexer: the thread that drains the channel a push queues
on. is_indexed(pack_id) on it is §13.12’s bit at the pack level, and it
is set only once that pack’s objects are in the index.
Sourcepub fn wait_indexed(&self)
pub fn wait_indexed(&self)
Block until the background drain has absorbed everything this store has pushed. What a maintenance tick, a seal or a test uses instead of racing it.
Sourcepub fn absorb_pending(&self) -> Result<usize>
pub fn absorb_pending(&self) -> Result<usize>
The indexer’s half of the split, and not one of the twelve.
Runs on the account indexer’s worker (§13.9), and on a read that arrives before that worker got there — the same function either way, which is why there is no such thing as a background index that behaves differently from the fallback one.
Sourcepub fn oids_in_extent(&self, extent: Extent) -> Result<Vec<Vec<u8>>>
pub fn oids_in_extent(&self, extent: Extent) -> Result<Vec<Vec<u8>>>
The object ids a stored pack introduced, and not one of the twelve.
GitOps::put_pack cannot answer this and never will: the walk it does
reads pack entries, and a pack entry does not carry its oid — that is
why crate::pack_walk::Closure can only name the REF_DELTA bases and
says so. The oids exist once the indexer has absorbed the pack, so this
absorbs first rather than racing the drain, and is therefore the one
caller that deliberately pays §13.9’s deferral back.
Only call it when something needs the set. A push whose policy has no
gate over objects must not pay this, which is the whole reason it is a
separate call and not a field on TxId.
The extent is the identity: §14 stores a pack verbatim as one
contiguous append, so “introduced by this pack” and “indexed at an offset
inside this pack’s extent” are the same set, with no pack column to keep
in step. Bases the push deltas against live in other extents and are
excluded by construction — which is what a thin push needs, and what
reading a written .idx back could never do.
Cost is one pass over the index, so it is O(objects in the repository), not O(objects pushed). Recorded rather than hidden: a per-pack ordinal range would make it O(pushed), and needs the indexer to keep one.
Sourcepub fn emit_set(
&self,
oids: &[Oid<'_>],
ofs_delta_ok: bool,
thin_haves: Option<&[Oid<'_>]>,
) -> Result<Vec<EmitEntry>>
pub fn emit_set( &self, oids: &[Oid<'_>], ofs_delta_ok: bool, thin_haves: Option<&[Oid<'_>]>, ) -> Result<Vec<EmitEntry>>
Everything needed to emit a pack for EXACTLY oids — nothing added,
nothing dropped.
Not one of the twelve. It sits beside absorb_pending
for the same reason those do: it is about how storage is shaped, not
about what a caller stores or reads.
§🔴 It used to add the delta bases, and that shipped broken clones
Until 2026-08-11 this closed the request over its delta bases: an
OFS_DELTA names its base by position, so a base outside the set cannot
be encoded, and adding the base looked like the answer that is always
safe. It is not, and the failure is silent on the server:
git clone --bare --single-branch --branch base <url>
fatal: did not receive expected object 8601ec33920b7d701c9887fa04916501136d6e90
fatal: fetch-pack: invalid index-pack output
event="git.upload_pack.served" objects=33773 ← the server said SUCCESSMeasured on the h2h-linear-sha1-2048c-1024f-16k fixture: base reaches
31 805 objects, the delta-base closure added 1 968 more — 33 773,
exactly the count the server logged — and 507 of those additions were
trees. A tree that is not reachable from the wants drags its own
children in as a requirement, because git index-pack --check-self-contained-and-connected (which is what a clone runs, and
which sets strict) walks every received object’s links and then demands
each one exist. Those 507 trees named 204 objects the pack did not
contain. 8601ec33… is one of them.
So a delta base pulled into a narrowed request is not free: it satisfies
the pack format and violates connectivity. Closing over the added
bases’ children in turn does terminate — the same fixture reaches a fixed
point at 34 534 objects — but it is an over-send by construction, and on
a --filter=blob:none fetch it adds back exactly the blobs the filter
excluded, which is the defect
emit_pack_emits_exactly_the_set_it_is_given_and_never_its_closure
exists to forbid.
§What it does instead: the base decides how the entry is encoded
The set is never widened. For each requested object:
- base inside the request → the stored entry is copied byte for byte, which is the whole point of this engine and is what a full clone does for every single object;
- base outside the request → the entry is rebuilt from §14’s
resolved content and marked
EmitEntry::recompressed— as a computed delta against a base the request does carry where one can be found (seeSelf::in_request_chain_ancestorandcrate::delta), and whole where it cannot.
That is what stock pack-objects does — it reuses a stored delta only
when the base is also being packed — and it keeps the receipt a fact:
copied and recompressed add up to the objects written, and
recompressed is 0 for a whole-repository clone because such a
selection contains every base. The cost lands on the boundary a narrowed
request cuts through and nowhere else: 1 968 of 33 773 entries, 5.9 %, on
the fixture above.
ofs_delta_ok is the client’s ofs-delta capability. An old client that
cannot parse one is not refused any more: the entry is re-headed as a
REF_DELTA naming its base by oid, carrying the same compressed delta
payload, so that arm stays a copy too. Only a base outside the request
costs a rebuild.
§thin_haves — the boundary entry that does NOT have to be rebuilt
Some(tips) says the client consented to a thin pack and the
negotiation found common ground; [crate::serve::GitStore::emit_oids] is
where the two halves are required together, because either alone is a
failed fetch rather than a smaller one. Given it, a base outside the
request is checked against the client's own objects,
and one the client already holds is named by oid in a REF_DELTA that
carries the stored delta stream unchanged. That entry counts as
copied, because it is: nothing was inflated and nothing was deflated.
None — a clone, or a client that did not ask for a thin pack — behaves
exactly as before, and a full clone is byte-identical either way,
because a whole-repository request has no base outside itself for any of
this to apply to.
It does nothing for the clone half of the boundary cost, and cannot: a
clone’s receiver holds nothing, so there is no external base to name.
That half is the arm below it — crate::delta computes a delta against
something the pack does carry, which is the only move left when there
is no receiver to lean on. The two do not overlap: thin_haves reaches a
boundary entry first and costs nothing when it fires, so a fetch stays a
pure copy and only a clone pays for a computed delta.
§An oid this repository does not hold is REFUSED
It used to be skipped, which is the same silent-under-send shape as the
closure defect above wearing a different hat: N asked, N-1 emitted,
PackStats reporting success, and the client the first to know. gunnar’s
in-memory arm has always refused it, so the skip also meant the two
engines behind one contract disagreed about what a missing object means.
A partial-clone filter does not depend on the skip and cannot: a filter
removes oids from the selection, it never adds one the store lacks.
§Cost
O(objects requested). The previous shape read oids_in_order() and
lookup_batch over the whole repository on every call, to build an
offset→oid map it needed only for the closure; with no closure the
selection’s own offsets answer “is this base in the request”, so that
scan is gone and no reverse index has to exist.
Source§impl<S: ObjectIndex + 'static> GitStore<S>
impl<S: ObjectIndex + 'static> GitStore<S>
Sourcepub fn emit_oids(
&self,
oids: &[Oid<'_>],
have: &[Oid<'_>],
caps: &Caps,
out: &mut dyn Write,
) -> Result<PackStats>
pub fn emit_oids( &self, oids: &[Oid<'_>], have: &[Oid<'_>], caps: &Caps, out: &mut dyn Write, ) -> Result<PackStats>
Emit a packfile for an explicit object set, closed over its delta bases — and over nothing else.
This is the whole of GitServe::emit_pack as of 2026-08-10: that
method is this call and a discarded have. It was written as the
inherent escape route for a trait method that closed over its input,
so that a caller told “walk it yourself” by GitServe::select’s
Ok(None) had somewhere to put the set it had walked. The trait method
stopped closing over its input, so the escape route and the front door
are now the same door. It stays inherent because the gix arm needs no
equivalent and the contract does not grow a method for it.
§🔴 No closure at all any more, and that is the 2026-08-11 fix
GitStore::emit_set used to add a delta’s base when the base fell
outside the request, on the reasoning that an OFS_DELTA names its base
by position and so cannot be encoded without it. That reasoning is
correct about the pack format and wrong about the clone: a base
pulled in is an object the client did not ask for, and if it is a tree it
arrives owing children the pack does not contain. git index-pack --check-self-contained-and-connected — what a clone runs — then dies
with did not receive expected object, while this server logs
git.upload_pack.served. See emit_set for the measured numbers.
So the set is now exactly the caller’s, and the base decides how each
entry is encoded rather than what the pack contains: base inside the
request, copy the stored bytes; base outside it, rebuild the object
whole. That is stock pack-objects’ rule, and recompressed counts the
second case honestly instead of being a literal zero.
§Three steps, all of them znippy’s own
GitStore::emit_setreads each requested entry and decides copy or rebuild, never adding an object;crate::pack_walk::topological_orderputs every base before the delta naming it, which backwards distances require;crate::pack_walk::emit_packre-encodes each entry header — the one thing that must change, because a distance is relative to a position in the input pack — and streams every payload byte for byte.
copied + recompressed == objects always, and for a whole-repository
clone recompressed is 0 because such a request contains every base.
Both are counted off what happened rather than asserted: a byte count and
a wall clock cannot tell a pack-copy from a re-deflate, and both pass
index-pack --strict.
Trait Implementations§
Source§impl<S: ObjectIndex + 'static> GitOps for GitStore<S>
impl<S: ObjectIndex + 'static> GitOps for GitStore<S>
Source§fn put(&self, pack: &[u8], refs: &[RefUpdate]) -> Result<TxId>
fn put(&self, pack: &[u8], refs: &[RefUpdate]) -> Result<TxId>
One push, and the order inside it is the contract.
Not interchangeable, and it is the same argument SafeWriter makes one
level down about the blob and its journal row: a crash between the two
leaves objects nobody points at, which a GC reclaims. The reverse order
leaves a ref pointing at objects that are not there, which is a
corrupt repository that no later pass can repair.
Source§fn put_pack(&self, bytes: &[u8]) -> Result<TxId>
fn put_pack(&self, bytes: &[u8]) -> Result<TxId>
CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW
The ack path. Walk, check, store verbatim, ack, then queue the index.
- Walk every entry (
crate::pack_walk). Not optional and not an extra pass: a pack entry has no length field, so finding the entry boundaries is how the pack gets split at all. - The closure check, free from that walk (§13.7). Every
OFS_DELTAbase must land on an entry boundary in this pack — answered from the boundary set the walk just produced, consulting no index. Only aREF_DELTAbase that is outside the pack is looked up, and that one readsobjects.oidand nothing else, exactly as §13’s table says receive-pack does. A pack that fails is refused before a byte is stored. - The bytes, verbatim.
SafeWriter::appendwrites the caller’s buffer at its own address — no re-compression, no re-encoding, no re-framing, not one copy in userspace. The client’s own deflate and every delta chain survive, which is what makes a later clone a byte-range copy instead of a re-pack (§14). - Durable before returning: the blob is fsynced, then the journal
row that references it, then that is fsynced. znippy’s
hot.rsordering, so a crash between the two leaves orphan bytes nobody points at rather than a dangling reference. - Then the index job goes on the channel and this returns. Nothing that could be done later is done here: no oid is computed, no table is built, no chain is resolved.
Source§fn put_refs(&self, updates: &[RefUpdate]) -> Result<TxId>
fn put_refs(&self, updates: &[RefUpdate]) -> Result<TxId>
CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW
One ref transaction: one Arrow IPC frame, fsynced.
The frame boundary is the transaction (see crate::pushlog) — three
branches in one push are three rows in one batch, and they either all land
or none do. There is no lock file and no second journal.
Every target is checked against the index first. A ref that points at an object the repository does not have is a corrupt repository, and it is refused here rather than written and discovered later. A deletion has no target and is not checked.
Source§fn get(&self, oid: Oid<'_>) -> Result<Option<Stored>>
fn get(&self, oid: Oid<'_>) -> Result<Option<Stored>>
CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW
The object’s stored bytes: one index lookup and one pread of the
extent, with no decode of any kind in between.
What comes back is the pack entry exactly as the client sent it, which for
a delta is a delta. Stored::obj_type says which, so the bytes cannot
be mistaken for the object’s content — §14 makes the verbatim bytes the
truth and the resolved object a derived cache, and that cache does not
exist yet, so handing back delta bytes labelled “the object” would be the
one thing this crate refuses everywhere: a wrong answer where an honest
one was available.
Source§fn has(&self, oid: Oid<'_>) -> Result<bool>
fn has(&self, oid: Oid<'_>) -> Result<bool>
CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW
The negotiation call: have sends up to a thousand of these and most of
them miss. It reads objects.oid and no other column.
It takes the serial path, not lookup_batch with one element: that is
1.7× slower (818 ns against 482), and lookup_path is where that
decision is written down.
§Why this returns Result<bool> and not bool
The signature changed deliberately. A store with a pack whose bytes are
durable but whose objects are not indexed yet cannot answer “no” — the
object may be in that pack. So it absorbs the pack first and, if that
fails, says so. A bool could only have lied, and a wrong “absent”
during negotiation makes a client send nothing and lose data.
Source§fn size(&self, oid: Oid<'_>) -> Result<Option<u64>>
fn size(&self, oid: Oid<'_>) -> Result<Option<u64>>
CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW
The object’s post-resolution size — what it inflates to once its delta
chain is applied. That is the fact git’s own .idx and .rev together
cannot answer, and it is why the quota gate is index-only here.
Source§fn extents(&self, oids: &[Oid<'_>]) -> Result<Vec<Option<Extent>>>
fn extents(&self, oids: &[Oid<'_>]) -> Result<Vec<Option<Extent>>>
CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW
The wire path, and batch by construction: for each oid the two facts a
clone needs — offset and len — and no others, so a byte-range copy out
of the verbatim pack can start immediately.
out[i] answers oids[i]. One oid takes the serial path
(lookup_path); the batch path saturates at
BATCH_SATURATES_AT oids, so a
larger batch is passed through whole rather than split — splitting would
cost a pass and buy nothing.
Source§fn refs(&self) -> Result<Vec<RefRow>>
fn refs(&self) -> Result<Vec<RefRow>>
CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW
The whole ref namespace, tags peeled — the ref advertisement, and
ls-refs after a prefix filter. Name-sorted, because the log folds into a
BTreeMap and ls-refs wants a prefix scan.
The cost is a scan of a structure sized by pushes, not by repository size.
§HEAD is not in here, and that is the contract
Changed 2026-08-10, and it is a fix rather than a preference. The
other backend’s iter() “walks refs/ (loose and packed) and
deliberately excludes the pseudo-refs such as HEAD, which is exactly the
contract every other backend honours”. This one did not, so the two arms
disagreed about whether HEAD is a row — and the conformance suite could
not see it, because it never exercised HEAD at all.
The reason HEAD cannot be a row is a type constraint, not taste: a name
type that admits HEAD also admits MERGE_HEAD and FETCH_HEAD, so a
row stream carrying pseudo-refs means either widening the name type or
filtering at every consumer. It gets an accessor pair instead —
GitServe::head and
set_head — and this is the one
filter, in the one place.
Nothing else changes: the ref log still stores HEAD (a push writes it
like any other row), live_set still reaches it because it reads
ref_state directly, and the filter is one !=.
Source§fn update_ref(
&self,
name: &str,
old: Option<Oid<'_>>,
new: Option<Oid<'_>>,
) -> Result<TxId>
fn update_ref( &self, name: &str, old: Option<Oid<'_>>, new: Option<Oid<'_>>, ) -> Result<TxId>
CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW
Compare-and-swap, in the same log a push writes its refs to — one mechanism, not a second one for single updates.
old is what the caller believes the ref is: None means it must not
exist (a create), Some(oid) means it must be exactly this. new of
None deletes. A mismatch names both values and writes nothing.
The read-compare-append is serialised on the store’s ref gate. Without it two CAS calls could both read the old value and both append, and the second would silently overwrite an update it had compared against successfully — the classic lost update, and the only thing a CAS is for.
Source§fn put_refs_cas(&self, edits: &[RefCas<'_>]) -> Result<TxId>
fn put_refs_cas(&self, edits: &[RefCas<'_>]) -> Result<TxId>
CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW
git push --atomic: every edit or none, in one Arrow IPC frame.
The two halves come from different places and neither is reimplemented:
- the batch atomicity is the frame.
crate::pushlogmakes the frame boundary the transaction — three branches in one push are three rows in one batch and they either all land or none do. There is no lock file and no second journal, so there is no partial-apply state to recover from; - the compare half is checked here, against one snapshot, before a
single
RefUpdateis built.
It goes out through put_refs rather than straight to
the log, so the dangling-target refusal and the frame format are the same
code a push uses (LAW 5): an atomic batch cannot create a ref pointing at
an object the repository does not have, which a second writer here would
eventually have allowed.
§S-023, and why the check is a whole pass of its own
Every expectation is evaluated before any of them is applied, and the
loop deliberately does not fuse with the one that builds the updates. The
defect it exists for is gix’s: a backend that short-circuits an edge whose
new value already equals the current one never evaluates the expectation
the caller wrote, so an old: None — must not exist — becomes a silent
success and “exactly one creator wins” stops being true. This log has no
such short-circuit, but the ordering is what makes that irrelevant rather
than lucky, and a future optimisation that adds one cannot break it from
here.
§What this arm cannot raise, stated rather than hidden
RefRejection::Locked never comes out of this backend. The
read-compare-append is serialised on the store’s ref gate, which a second
writer blocks on rather than failing against — so contention here is a
wait, never a rejection. The variant is in the contract because the gix
arm, which takes real per-ref lock files, raises it. A poisoned gate stays
an ordinary error: it is a fault, not the transient thing a caller retries.
Source§fn reachable(&self, want: &[Oid<'_>], have: &[Oid<'_>]) -> Result<Vec<Vec<u8>>>
fn reachable(&self, want: &[Oid<'_>], have: &[Oid<'_>]) -> Result<Vec<Vec<u8>>>
CLAUDE FUCKED UP AND MUST NOT FUCK UP AGAIN. ZNIPP-GIT APACHE ARROR IPC IS LAW
Selection: want minus have, as an andnot of roaring bitmaps.
Every returned oid is an object — commit, tree and blob — because the
bitmaps are built by crate::reach::build_reach, which walks the trees.
A want that names a commit contributes that commit’s whole closure; a
want that names a tag or a blob contributes itself.
A have this repository does not know contributes nothing: the safe
direction is to send more, never less.
The bitmaps are over the store’s own ordinal space, rebuilt by the same
fold that rebuilds them, never over the Arrow projection’s ordinals — an
crate::index_layout::IndexRow::ordinal is a row address within one
projection generation, so a bitmap over those would address different
objects after any rebuild, silently.
§The per-oid Vec is paid HERE and nowhere else on the serving path
The answer is computed flat (see
GitStore::reachable_raw);
this splits it back out because the eleven say Vec<Vec<u8>> and this
method’s callers are maintenance ones — a GC live set, a conformance
harness — not the serving path. crate::serve’s select and
emit_pack take the flat form directly.
Source§fn gc(&self) -> Result<GcReport>
fn gc(&self) -> Result<GcReport>
Garbage collection, in the order §13.20 fixes:
- compute reachability — every object reachable from every ref, over
the same bitmaps
reachableuses; - drop the dead rows from the index — the one operation that is not append-only, and the projection is rebuilt inside it;
- then base znippy’s compaction, through
crate::gc::NewGeneration(link, compact, verify, rename, unlink last).
That order is why base znippy needs no new method: by the time
compact_archive runs, “live” already means what git means.
A repository with no ref pointing at anything is refused rather than emptied.
Appended, not reworded: step 1b, the journal. Between the live set and
the drop there is now one more durable act. §13.12’s indexed bit is
derived on open as extent in the journal, rows not in the index, and
dropping every row of a pack produces exactly that state — so before
this existed the next open re-queued the pack and every object this GC had
just decided was dead came back.
GitStore::retire_dead_packs
appends a tombstone naming each all-dead pack, and it runs before
drop_dead_rows on purpose: killed before it, the rows are still there
and the GC simply did not happen; killed after it, the pack can never be
re-queued whether the rows went or not. A partly dead pack keeps rows and
is never tombstoned, so nothing about it changes.
Step 2 also refolds what the two tables feed — the commit graph, the tree
payloads, the ordinal space, the bitmaps — because a derivation that still
names a dropped oid is wrong rather than stale. That is inside
drop_dead_rows, so any caller of it gets it.
Source§impl<S: ObjectIndex + 'static> GitServe for GitStore<S>
impl<S: ObjectIndex + 'static> GitServe for GitStore<S>
Source§fn read(&self, oid: Oid<'_>) -> Result<Option<(ObjType, Vec<u8>)>>
fn read(&self, oid: Oid<'_>) -> Result<Option<(ObjType, Vec<u8>)>>
The object, inflated and delta-resolved — §14’s exploded table, one point lookup.
This is the question GitOps::get deliberately does not answer, and
GitStore::content has answered it since §14 landed; the trait method
is that call plus the kind narrowing. Two paths under it, returning
identical bytes and distinguishable only by
crate::exploded::ExplodedStats: the table, or a re-derivation from
the verbatim truth when the table has been dropped or has not caught up.
The returned ObjType can never be a delta — see [resolved_type].
Source§fn header(&self, oid: Oid<'_>) -> Result<Option<(ObjType, u64)>>
fn header(&self, oid: Oid<'_>) -> Result<Option<(ObjType, u64)>>
Kind and post-resolution size, without the payload.
The size is a column, and the right column: uncompressed_size is
written after delta application, so it is the fact a .idx and a .rev
together cannot answer, and it is read rather than measured.
The kind is not a column — obj_type is the entry’s type in the pack —
so for the common case (a non-delta entry) this touches nothing else, and
for a delta it walks the chain’s headers through
GitStore::resolved_type_at. Still no inflate and still no payload.
Source§fn sizes(&self, oids: &[Oid<'_>]) -> Result<Vec<Option<u64>>>
fn sizes(&self, oids: &[Oid<'_>]) -> Result<Vec<Option<u64>>>
Post-resolution sizes in bulk: one index pass, no chain touched.
The half of header that is already data. On the
h2h fixture one clone paid 34 124 per-object header walks, every
one of which this collapses into a single lookup_batch, to establish
that a repository whose largest blob is 16 KiB holds nothing over the
1 GiB default ceiling.
§The preamble is not optional
This reaches GitStore::index directly, so it absorbs pending index
work itself. Without it a pack that landed since the last drain reads as
absent, and an absent size that a caller treated as a pass would
silently skip the ceiling for exactly the objects a push had just
introduced. None at a position is “unknown, go ask header” and
never “fine”.
Source§fn head(&self) -> Result<Option<RefRow>>
fn head(&self) -> Result<Option<RefRow>>
HEAD, which is not a row in GitOps::refs.
The gix arm’s iter() “walks refs/ (loose and packed) and deliberately
excludes the pseudo-refs such as HEAD, which is exactly the contract
every other backend honours” — and as of this change so does this arm
(see GitOps::refs). The reason is a type constraint rather than
taste: a name type that admits HEAD also admits MERGE_HEAD and
FETCH_HEAD, so putting pseudo-refs in the row stream means either
widening the name type or filtering at every consumer.
So it is read here, off the same fold GitOps::refs reads, rather than
from a second place that could disagree with it. None for a repository
that has never pointed one — an empty repository has no HEAD.
Source§fn set_head(&self, target: &str) -> Result<TxId>
fn set_head(&self, target: &str) -> Result<TxId>
Point HEAD, through the same log a push writes to.
§The two shapes of a target, and how they are told apart
HEAD is symbolic in every repository anyone serves — ref: refs/heads/main — and detached in the one case git also supports. The
contract’s parameter is one &str for both, so:
- a target starting with
refs/is a symbolic ref, written as one; - anything else must be a hex oid of exactly this store’s width, written as a direct target.
The two cannot be confused: a ref name and a 40- or 64-character hex
string are disjoint. Anything that is neither is refused rather than
guessed at — a HEAD pointing at a name nothing resolves is an
unclonable repository, and it is cheaper to say so here.
A symbolic target is deliberately not checked for existence.
git init points HEAD at refs/heads/main before that branch exists,
and refusing it would make an empty repository unrepresentable. A direct
target is checked, by GitOps::put_refs, because a detached HEAD
naming an absent object is dangling in exactly the sense that refuses.
Source§fn emit_pack(
&self,
objects: &[Oid<'_>],
have: &[Oid<'_>],
caps: &Caps,
out: &mut dyn Write,
) -> Result<PackStats>
fn emit_pack( &self, objects: &[Oid<'_>], have: &[Oid<'_>], caps: &Caps, out: &mut dyn Write, ) -> Result<PackStats>
Emit a packfile containing exactly objects.
One line of delegation to GitStore::emit_oids, and the absence of a
second line is the contract.
§🔴 This must NOT compute a closure. Fixed 2026-08-10.
Until that date this method opened with self.select(objects, have)? and
emitted that — it closed over its own input. The caller’s set came
back larger than it went in, and the caller is upload-pack holding
selection.objects: already post-filter, post-shallow, post-include-tag
and deliberately not closed. So a --filter=blob:none,
--filter=tree:0 or --depth=N fetch was served exactly the objects it
had asked to be left out.
The reason it needed a rename and a guard rather than a shrug is that
nothing we own could see it: the over-sent pack passes
git index-pack --strict and git fsck, the clone succeeds, the exit
code is zero, and the client just silently receives more than it asked
for. P-027’s shape — a change no test can see. Pinned now by
emit_pack_emits_exactly_the_set_it_is_given_and_never_its_closure,
which asserts a count (seen RED at 4 against an expected 1).
It had a second, louder failure mode too, and it is the one that proves
the two questions were never the same question: select’s tip check
declines a want that is not a commit in the graph, so a selection
containing a tree — which every real one does — could not be emitted at
all.
§Where the closure went
Nowhere. select still owns the close-over-tips half
and still answers Ok(None) when its projection cannot cover a request;
that hatch is untouched and still load-bearing. The two halves are now
cleanly separated: what to send is asked of select, send exactly
this is asked of here. A caller that select told to walk the graph
itself hands the walked set straight back to this method.
There is no closure left on this path at all: since 2026-08-11 the base decides how an entry is encoded, never what the pack contains.
§have is READ, and only for thin-pack base selection
The contract defines have as the negotiated common tips, “used for
thin-pack base selection, never to derive membership”, and that is
exactly and only what this arm does with it. Paired with caps.thin, it
lets a boundary entry go out as a REF_DELTA naming a base the receiver
already holds — carrying the stored delta stream unchanged — instead of
being rebuilt whole. See GitStore::client_bases for the closure and
for why the caller has to be able to stand behind the voucher.
It is deliberately not repurposed into an exclusion set. Deriving
membership from have is precisely the closure this method stopped
doing, in the opposite direction: it would remove objects the caller’s
finished selection had already decided to send, which is an under-send,
which is the failure that exits zero. The narrowing below changes the
encoding of an entry and never whether it is emitted.
Source§fn select(&self, want: &[Oid<'_>], have: &[Oid<'_>]) -> Result<Option<ReachSet>>
fn select(&self, want: &[Oid<'_>], have: &[Oid<'_>]) -> Result<Option<ReachSet>>
want minus have, out of §13’s bitmaps, with no object opened —
plus the two facts the caller cannot recompute cheaply.
§The refusal is the part that matters
GitOps::reachable treats a want it has no bitmap for as contributing
itself and nothing else. That is the safe direction for its own
caller — live_set over-keeps, and over-keeping is harmless — and a
silent under-send for this one. A clone served that way exits zero
with one object per branch.
So every tip is checked against the store’s commit graph first, and a tip that is not in it declines the whole request.
§🔴 “In the graph” and “has a bitmap” are NO LONGER the same fact
They were, and this paragraph used to say so, because the live table was
built with no commit cap precisely so that they would be. That cost 530
218 allocations a fetch (see crate::git_ops’s LIVE_REACH_COMMITS),
and since 2026-08-14 the table is sampled at 512 like the sealed
archive’s.
The check below is unchanged and is still exact, but it is now exact for
a different reason. It is “in the graph” that matters, not “has a
bitmap”: [crate::reach::accumulate] walks a bitmapless commit down to
the first bitmapped one behind it, so every graph commit is answerable,
with or without a bitmap of its own. A tip that is not a graph row is
still refused — there is nothing to walk — and that is the case this
check exists for.
Ok(None) is therefore reachable in three ways, and all three are the
same statement — the projection does not cover this request:
an empty graph (which a clean restart produces), a tip that is not a
commit in it, and a graph row whose oid does not parse.
§The selection is NOT re-ordered, and that is measured
reachable answers in ordinal order, which is oid-lexicographic and
therefore unrelated to where the bytes are. Sorting by archive offset so
a delta’s base precedes it was written and could not be made to fail:
measured 2026-08-08 on a 154-object fixture — with the sort, without
it, and with the whole selection deliberately reversed, all three give
copied=154 recompressed=0 and the identical object graph, because
crate::pack_walk::topological_order orders the emission itself. The
sort was a batch index lookup per request buying a property the emitter
already owns (LAW 5), and its absence is something no guard can see
(LAW 2). It is gone.