Expand description
The fetch path’s allocation ceilings.
§Why this module exists
gunnar-pack’s IndexOptions::alloc_limit_bytes guards the receive
path against a hostile pack claiming an enormous object. Nothing guarded the
serve path, and the serve path has the same two sources of
attacker-chosen size:
- the request — a v2 argument section and a v0
want/haveexchange are read until the peer sends a flush-pkt, into aVecthat grows with every line. Nothing said how many lines that could be. A client that opens a connection and sendshave <oid>forever makes the server allocate forever, and it does so before any repository is touched, so no traversal budget is in the way; - an object — the whole-object writer materialises each object’s inflated content before it deflates it, so peak memory is the largest object in the transfer. That size is not chosen by this request, but it was chosen by whoever pushed it.
§The principle, restated because it is the same one
gunnar-pack’s read module puts it as: a guard that is off by default
is not a guard. Both ceilings here default to Some, and both are
expressible as None by a caller that asks — Limits::no_alloc_limit —
which is the shape that made the pack indexer’s guard real rather than
optional.
§What each ceiling actually prevents
Limits::request_bytes is preventive: the bytes are counted as they
are framed, and the read stops at the ceiling. Nothing above it is ever held.
Limits::object_bytes is preventive too, and that is why it is applied
where it is. It is enforced in the object selection walk, from the
object’s header, before any content is read — not in the pack writer, which
only ever sees an object that has already been materialised. A ceiling
checked after gix_object::Find::try_find has filled a buffer is a report,
not a guard: the allocation it exists to stop has already happened by the
time it can fire.
An object over the ceiling is refused, not skipped. Skipping it would
produce a pack whose closure does not hold, which is a corrupt clone rather
than a refused one — the same judgement gunnar-store’s write_stream
makes about an oversized declared size.
§What is still O(repository), said plainly
These ceilings bound the two sizes gunnar does not choose. They do not make a fetch constant-memory, and the remaining terms are all the same shape — one machine word or one object id per object in the transfer — so they are listed with what they cost at ten million sha256 objects rather than left as “some overhead”:
| where | ~bytes/object | at 10M | |
|---|---|---|---|
| the selection | graph::Selection::objects | 32 | 320 MB |
| the client’s closure | graph::select_objects step 5 | ~51 | 510 MB |
| the writer’s de-duplicated copy | gunnar_pack::write::prepare_ids | 32 + ~51 transient | 830 MB |
| gix’s count list | gunnar_pack::copy::write_pack_copy | ~48 | 480 MB |
the ofs-delta offset table | gunnar_pack::copy::write_pack_copy | 16 | 160 MB |
And one term whose denominator is the pack, not the transfer, so it is listed apart rather than added to the column above:
| where | ~bytes/object in the pack | at a 10M-object pack | |
|---|---|---|---|
| The entry-extent table | gunnar_pack::copy::PackExtents | 48 (sha256) / 36 (sha1) | 480 MB |
It buys the removal of one full inflate per object served — MEASURED
2026-08-04 over the 19-repository corpus, 1.97× less server CPU on a clone
at the median and 21.4% less peak RSS on znippy.git — and it is built
once per request and shared across iter_from_counts’s clones rather than
per worker. Two thirds of it (Vec<(Offset, ObjectId)>) is a table
iter_from_counts already builds for itself whenever a thin fetch has to
resolve a delta base, so on that path the new term is the 8-byte offset
column alone. It is also one pack’s table and not the store’s: only the
pack gix-odb searched most recently is consulted, so a repository with many
small packs holds one table per pack it actually touched rather than one per
pack it has. A repository whose pack dwarfs the transfer is where this costs
most and where it saves least, and .rev-file support in gix-pack — which
would let the table be read rather than built — is the answer if that ever
shows up.
Since the thin-pack ruling the client’s-closure row is live for longer
rather than larger. It used to die at the end of select_objects; it is
now carried out as graph::Selection::client_has, because vouching for the
base of a thin delta is what lets a fetch ship the deltas it already had
instead of expanding them. There is no second copy — the same set is moved,
and graph::tests::a_fetch_holds_the_clients_object_closure_once still
holds the peak to one — but it now overlaps the last three rows instead of
preceding them.
What bounds it is Exclusion::Edges, the shipping default: that walk opens
up the frontier rather than the client’s history, so the set carried to the
writer is frontier-sized. The same test asserts it, as live ids, against the
Exclusion::Full figure. Under GUNNAR_EXCLUSION=full the row is the
client’s whole closure and does overlap the write.
crate::DEFAULT_BUDGET bounds all of them at once, but it bounds them in
objects, and ten million objects of that bookkeeping is on the order of a
gigabyte before a single byte of content is read. That is the honest reading
of the budget, and the git package’s reachability bitmaps are what removes the first two
rows rather than caps them.
Structs§
- Limits
- The ceilings one connection serves under.
- Walk
Pool - The ceiling on what the whole server holds at once, in bytes of walk bookkeeping, shared by every in-flight transfer.
Constants§
- DEFAULT_
OBJECT_ LIMIT_ BYTES - Ceiling on a single object gunnar will materialise while serving: 1 GiB.
- DEFAULT_
REQUEST_ LIMIT_ BYTES - Ceiling on the bytes held for one peer-supplied request: 8 MiB.
- DEFAULT_
WALK_ LIMIT_ BYTES - Ceiling on the object bookkeeping one fetch may hold: 8 GiB.
- WALK_
BYTES_ PER_ OBJECT - What one object id held by a traversal costs in resident bytes: 384.