Skip to main content

Module limits

Module limits 

Source
Expand description

The fetch path’s allocation ceilings.

§Why this module exists

gunnar-pack’s IndexOptions::alloc_limit_bytes guards the receive path against a hostile pack claiming an enormous object. Nothing guarded the serve path, and the serve path has the same two sources of attacker-chosen size:

  • the request — a v2 argument section and a v0 want/have exchange are read until the peer sends a flush-pkt, into a Vec that grows with every line. Nothing said how many lines that could be. A client that opens a connection and sends have <oid> forever makes the server allocate forever, and it does so before any repository is touched, so no traversal budget is in the way;
  • an object — the whole-object writer materialises each object’s inflated content before it deflates it, so peak memory is the largest object in the transfer. That size is not chosen by this request, but it was chosen by whoever pushed it.

§The principle, restated because it is the same one

gunnar-pack’s read module puts it as: a guard that is off by default is not a guard. Both ceilings here default to Some, and both are expressible as None by a caller that asks — Limits::no_alloc_limit — which is the shape that made the pack indexer’s guard real rather than optional.

§What each ceiling actually prevents

Limits::request_bytes is preventive: the bytes are counted as they are framed, and the read stops at the ceiling. Nothing above it is ever held.

Limits::object_bytes is preventive too, and that is why it is applied where it is. It is enforced in the object selection walk, from the object’s header, before any content is read — not in the pack writer, which only ever sees an object that has already been materialised. A ceiling checked after gix_object::Find::try_find has filled a buffer is a report, not a guard: the allocation it exists to stop has already happened by the time it can fire.

An object over the ceiling is refused, not skipped. Skipping it would produce a pack whose closure does not hold, which is a corrupt clone rather than a refused one — the same judgement gunnar-store’s write_stream makes about an oversized declared size.

§What is still O(repository), said plainly

These ceilings bound the two sizes gunnar does not choose. They do not make a fetch constant-memory, and the remaining terms are all the same shape — one machine word or one object id per object in the transfer — so they are listed with what they cost at ten million sha256 objects rather than left as “some overhead”:

where~bytes/objectat 10M
the selectiongraph::Selection::objects32320 MB
the client’s closuregraph::select_objects step 5~51510 MB
the writer’s de-duplicated copygunnar_pack::write::prepare_ids32 + ~51 transient830 MB
gix’s count listgunnar_pack::copy::write_pack_copy~48480 MB
the ofs-delta offset tablegunnar_pack::copy::write_pack_copy16160 MB

And one term whose denominator is the pack, not the transfer, so it is listed apart rather than added to the column above:

where~bytes/object in the packat a 10M-object pack
The entry-extent tablegunnar_pack::copy::PackExtents48 (sha256) / 36 (sha1)480 MB

It buys the removal of one full inflate per object served — MEASURED 2026-08-04 over the 19-repository corpus, 1.97× less server CPU on a clone at the median and 21.4% less peak RSS on znippy.git — and it is built once per request and shared across iter_from_counts’s clones rather than per worker. Two thirds of it (Vec<(Offset, ObjectId)>) is a table iter_from_counts already builds for itself whenever a thin fetch has to resolve a delta base, so on that path the new term is the 8-byte offset column alone. It is also one pack’s table and not the store’s: only the pack gix-odb searched most recently is consulted, so a repository with many small packs holds one table per pack it actually touched rather than one per pack it has. A repository whose pack dwarfs the transfer is where this costs most and where it saves least, and .rev-file support in gix-pack — which would let the table be read rather than built — is the answer if that ever shows up.

Since the thin-pack ruling the client’s-closure row is live for longer rather than larger. It used to die at the end of select_objects; it is now carried out as graph::Selection::client_has, because vouching for the base of a thin delta is what lets a fetch ship the deltas it already had instead of expanding them. There is no second copy — the same set is moved, and graph::tests::a_fetch_holds_the_clients_object_closure_once still holds the peak to one — but it now overlaps the last three rows instead of preceding them.

What bounds it is Exclusion::Edges, the shipping default: that walk opens up the frontier rather than the client’s history, so the set carried to the writer is frontier-sized. The same test asserts it, as live ids, against the Exclusion::Full figure. Under GUNNAR_EXCLUSION=full the row is the client’s whole closure and does overlap the write.

crate::DEFAULT_BUDGET bounds all of them at once, but it bounds them in objects, and ten million objects of that bookkeeping is on the order of a gigabyte before a single byte of content is read. That is the honest reading of the budget, and the git package’s reachability bitmaps are what removes the first two rows rather than caps them.

Structs§

Limits
The ceilings one connection serves under.
WalkPool
The ceiling on what the whole server holds at once, in bytes of walk bookkeeping, shared by every in-flight transfer.

Constants§

DEFAULT_OBJECT_LIMIT_BYTES
Ceiling on a single object gunnar will materialise while serving: 1 GiB.
DEFAULT_REQUEST_LIMIT_BYTES
Ceiling on the bytes held for one peer-supplied request: 8 MiB.
DEFAULT_WALK_LIMIT_BYTES
Ceiling on the object bookkeeping one fetch may hold: 8 GiB.
WALK_BYTES_PER_OBJECT
What one object id held by a traversal costs in resident bytes: 384.