Module resolve 

Pack entries to oids: the indexer’s half of the split, off the ack path. Pack entries → oids, which is the one thing crate::pack_walk cannot tell you and the objects table is keyed by.

This is the indexer’s work, not the ack path’s (§13.9: “the index is built after the ack, over a channel”). It inflates every entry, applies every delta chain and hashes the result, which is what git index-pack does and what makes it the expensive half of a push. Nothing here is called before a client is told its push landed.

What it keeps and what it throws away

Peak memory is deliberately not “the whole history inflated”. An object is held only while an unresolved entry still has to read it, and for nothing else:

object	held	until
a delta base	yes	its last dependent has consumed it
everything else — blob, commit, tree, tag	no	it leaves through the PayloadSink and is dropped

The base set and, more to the point, how many entries read each base are known before a byte is inflated: the walk already collected every delta base, so the resolver starts with an exact reference count per base and drops a payload the moment that count reaches zero. The last dependent gets the base moved into it rather than cloned, so a linear delta chain — one base, one dependent, the ordinary shape of real history — copies no payload at all.

What this table said before 2026-08-11, and what it cost

It said commits, trees and tags were kept “because the graph and reach tables are built from them”. They are not, and had not been since §14’s exploded table became the fold’s input — absorb_one reads Resolved::index_entry() and nothing else. Two copies of the object set were being held for a reader that no longer existed:

• content kept every base and every typed object for the whole call. Nothing ever removed a row, so “held only while something still needs them” described an intention, not the code.
• Resolved::payload kept a second copy of the same bytes, cloned in.

MEASURED on oden 2026-08-11, tests/resolve_peak_memory.rs, peak live heap over resolve_walked against the pack’s own inflated payload:

pack	before	after
forge-year-8a-240b-12c, 28 893 objects, 111.0 MB	2.03x	0.03x
stage-red-nobitmap, 12 328 objects, 343.5 MB	1.34x (461.2 MB)	0.02x (5.9 MB)
facett, 7 232 objects, 72.8 MB	1.20x	0.42x
znippy’s own, 5 388 objects, 84.4 MB	0.75x	0.14x

The worst pack in the corpus went 2.03x → 0.42x. It is facett, whose delta chains fan out widely enough that a real base working set is genuinely live at once — which is the bound behaving as intended rather than a residue.

End to end, one gunnar-server, one push --mirror, anonymous RSS held afterwards (not peak — this memory was never given back):

fixture	before	after
36 172 objects, 290 MB inflated, 5.9 MB pack	575.7 MB	185.1 MB
352 396 objects, 2.89 GB inflated, 61.8 MB pack	4480.1 MB	439.9 MB

The second row is the one that mattered: 10x the repository cost 10x the memory before and 2.4x after, so the resolver’s footprint stopped tracking the history. What is left is bounded and named — redb’s page cache (ZNIPPY_GIT_REDB_CACHE_BYTES, 64 MiB by default, measured at 129.3 MB held with an 8 MiB ceiling and 303.9 MB with 256 MiB), the exploded table’s FLUSH_BYTES write buffer, and Derived’s in-memory graph, which is genuinely O(repository).

⚠ The clone that got slower, and why it is not this code

Read this before “fixing” the reference counting. vs_forge_clone on the 352 396-object fixture reported this change as a +78 % CPU regression on every znippy arm, and the natural reading — a base dropped at its “last use” has to be re-resolved when something later wants it again — is wrong. It is not even mechanically possible: an undercount cannot cause a re-resolve here, because there is no re-resolve to fall into. resolve_walked fails the whole pack by name.

What it actually is, MEASURED on oden 2026-08-11, one gunnar-server, seed the fixture then clone it, steady state (clones 2-3), quiet box, only the allocator and this file varying:

resolver	allocator	clone CPU	anon after seed
before	glibc	1.47 s	4481 MB
after	glibc	2.62 s	438 MB
before	mimalloc	1.18 s	1767 MB
after	mimalloc	1.17 s	701 MB

With a competent allocator the two resolvers are identical (1.18 against 1.17). The regression is entirely glibc malloc: the old resolver’s retained 4.5 GB was accidentally acting as a pre-warmed pool, so the clone’s own ~2 GB of allocations came off a free list instead of out of the kernel. Take the leak away and glibc charges for the memory it should have been charging for all along.

Three independent measurements say the same thing, and none of them involve this file being slower:

1. Bisect. A build with only Resolved::payload removed — the reference counting NOT applied, content still never pruned — reproduces the whole regression (2.56–2.62 s). Adding the reference counting on top then costs nothing (2.55–2.65 s) and takes anon-after-seed from 2350 MB to 438 MB. The counting is free; the not leaking is what glibc punishes.
2. Restart. Restart the server after seeding, so the derived tables are folded onto a fresh heap: before 2.20–2.22 s, after 2.20–2.24 s — identical to within 1 %. The old resolver’s 1.47 s is the outlier, not the new one’s 2.62 s.
3. The resolver itself. Over the same 6 real packs both revisions accept, 1.058 s before against 0.975 s after: this code is ~8 % faster.

So there is no cheaper trade to find inside this module, and an LRU over bases would buy nothing. The lever, if the CPU is wanted back, is the allocator — and LD_PRELOAD=libmimalloc.so.3 is enough to get it with no code change at all.

Where the payloads it throws away now go

§14’s exploded table is eager (decided 2026-08-08), so every object this module resolves has to reach [crate::exploded::ExplodedTable] — including the blobs the table above says are dropped. They still are: the payload leaves through a PayloadSink as it is produced, rather than being accumulated into the returned Vec. That is the difference between a peak footprint of “the bases plus the typed objects” and one of “the whole pack inflated at once”, and eager resolution does not get to change it.

A thin pack’s REF_DELTA base is an object the client knows the server already has, and it is usually a blob. Resolving it needs that blob’s content, which is exactly what the exploded table now holds — so the BaseSource can answer from it in one point lookup instead of re-resolving a whole pack. A thin pack whose external base is still not re-derivable is refused by name ([ResolveError::MissingBase]) rather than resolved by guesswork, unchanged.

The bounded base set (2026-08-30)

The table above says a base is held until its last dependent. On torvalds/linux that set is unbounded in principle — 11.2 M deltas over 13.9 M objects, and nothing in the pack format promises a base’s last dependent is near it. On the 16 GB appliance the indexer grew past the box and the kernel panicked (see crate::memguard). So the set now has a byte ceiling (ResolveLimits::base_budget, default 25 % of MemTotal, ZNIPPY_GIT_BASE_BUDGET_BYTES):

event	what happens
a base would push the held bytes over the ceiling	the held base whose next dependent is farthest in the pack is dropped (Belady — the walk already lists every dependent’s position, so “next use” is known, not guessed)
the new base itself is the farthest	it is not held at all
a dependent asks for a dropped base	it is re-inflated from the mmapped pack — its own chain re-applied — and counted in ResolveReport::reinflations
the watchdog says the box is short	every held base is dropped, the sink and the row batch are flushed, and the worker waits

Same output, hard ceiling. The ceiling is max(budget, largest single base): one object bigger than the budget is held while it is needed, because the alternative is re-inflating it once per dependent.

What is preallocated

Two scratch buffers live across the whole pass: the delta instruction stream and the payload of an object nothing deltas against. An object that is a base gets its own Vec because it outlives the iteration; every other object — the overwhelming majority — is inflated into the reused buffer, hashed in place (GitHashKind::oid_of_object, no canonical copy) and handed to the sink from there. Per object the hot path allocates the 20-byte oid of its row and nothing else.

Structs

CycleGuard

A termination proof for a chain walk, paid for only by the chains that need one.

NoBases

A source that has nothing. The right one for a self-contained pack, and it makes “this pack was thin” an error rather than a silent success.

ReadCost

What one read actually cost, measured by the read rather than predicted.

ResolveError

The one failure that is a decision rather than a corrupt pack.

ResolveLimits

The knobs and hooks the indexer hands the resolver. ResolveLimits::none is the old behaviour: unbounded bases, no watchdog, no progress.

ResolveReport

What one resolve did — the audit line’s numbers, all applied output.

Resolved

One resolved object: exactly what the objects table stores, and no payload.

SliceArchive

A whole &[u8] as an archive — what a test and a single-pack caller use.

Constants

DEFAULT_BASE_BUDGET_PCT

Default share of MemTotal the held base set may take.

DEFAULT_LOOKAHEAD_BYTES

Ceiling on how far the workers may run ahead of the ordered stream, in bytes of rows and of the payloads the sink asked to be given. Past it a worker waits for the emitter instead of buffering the pack — see ResolveLimits::lookahead_bytes.

DEFAULT_ROW_BATCH

Default rows per batch handed to the row sink.

MAX_DELTA_LINKS

git’s own pack.depth ceiling doubled. The depth at which a chain walk stops assuming it will terminate and starts proving it — see CycleGuard. It is not a refusal any more, and the 🔴 below says why.

MAX_WORKERS

A ceiling on the worker count no environment variable may cross. A pack is resolved by the box, not by a number somebody typed.

Traits

ArchiveBytes

Bytes of the verbatim archive, addressed absolutely.

BaseSource

Where an entry that deltas against something outside the pack gets its base.

Functions

base_budget_from_env

ZNIPPY_GIT_BASE_BUDGET_BYTES, else 25 % of MemTotal, else 1 GiB when the box cannot be measured. Read once per process.

lookahead_from_env

ZNIPPY_GIT_RESOLVE_LOOKAHEAD_BYTES, else DEFAULT_LOOKAHEAD_BYTES. Read once per process.

materialize_at

One object’s content, with only its own chain inflated, addressed by archive offset.

materialize_entry

One object’s content from a pack and an entry, with only its chain inflated — the lazy read path. offset_of_oid answers a REF_DELTA base that is inside the pack (its entry offset); bases answers one that is not. Nothing is cached: a chain of depth d costs d inflates and d−1 delta applications, and nothing else.

resolve

Resolve every entry of pack to an oid, keeping no payloads.

resolve_streaming

The resolver: bounded bases, watchdog checkpoints, rows in batches.

resolve_walked

Same, for a caller that already walked the pack (put does, for the closure check) and must not walk it twice — and for the one that wants the payloads.

row_batch_from_env

ZNIPPY_GIT_INDEX_BATCH_ROWS, else DEFAULT_ROW_BATCH. Read once per process.

set_resolve_lookahead_default

Set the process-wide default resolver lookahead, in bytes, for every resolve started AFTER this call (an explicit crate::arms::ENV_RESOLVE_LOOKAHEAD_BYTES still wins). 0 restores DEFAULT_LOOKAHEAD_BYTES. The twin of crate::arms::set_graph_cache_default — see set_resolve_workers_default for the measurement the pair is for.

set_resolve_workers_default

Set the process-wide default resolver width for every resolve started AFTER this call (an explicit crate::arms::ENV_RESOLVE_WORKERS still wins). 0 restores one per core. The twin of crate::arms::set_graph_cache_default, for an embedder that learns its role at runtime and cannot set the environment — edition 2024 makes set_var unsafe, and gunnar forbids unsafe. MEASURED (gunnar twin_snapshot_memory.rs, 2026-09-15): one worker per core, each inflating a whole blob, is +133 MiB of RssAnon for a 512 MiB snapshot install on a many-core box; two workers with a 16 MiB lookahead is +36 MiB, under gunnar’s 64 MiB twin ceiling.

workers_from_env

ZNIPPY_GIT_RESOLVE_WORKERS, else one per core. Read once per process.