Expand description
The substrate’s GitPackIngest implement (PackIngest) — ADR-020
§2’s ingestion composition, for real: the received pack indexes into
a budgeted host working area (thin bases resolving through the
substrate Find), the indexed objects unpack into alkblobs as
loose-form objects under the git-object family (the wave-7 object
front’s family put — the store’s verification-on-put is the oid
re-derivation, corruption cannot enter undetected), the guards ride
the w8-pin-registry’s custody seam, connectivity closes the graph
from the pushed tips, and the working area is discarded. The
reference shape is POC-O’s ingest.rs; the reference semantics are
the wave-3 doubles’ MemPackIngest (tests/common/mod.rs in the
core crate).
§The recorded design decisions (the wave-8 gate’s reading list)
- The decompressed-size cap’s enforcement, layered (the task’s
named design point — how the
Limits::max_object_sizebound holds through gix’s own index/inflate machinery):- the counting reader is the defense-in-depth layer the
wave-3 gate confirmed: the pack stream’s byte total is bounded
by
Limits::max_pack_size; a breach is a pack-level failure mapping into theunpackng leg, neverErr(the wire side’s relay breaches separately fail the session closed); - the indexing leg rides gix’s indexing machinery
(
gix_pack::Bundle::write_to_directory) with itsalloc_limit_bytesknob set to the cap: gix’s decode paths vet every allocation-relevant pack-declared quantity against the limit before reserving, so a hostile declaration beyond the cap errors before any allocation, and the machinery’s exact-decompression discipline (a stream that stops early or overproduces against its declared buffer is corruption) makes a lying-small declaration fail too — the hostile-declared-size class maps to the unpack ng leg over the real machinery; - the unpack leg is this module’s own incremental machinery — the wave-5 doubles’ guard verbatim: each entry’s declared size is vetted against the cap before anything is read, the inflate reads in bounded chunks accumulating only produced bytes (the running total aborts at the cap), and each delta’s materialized result is checked against its declaration and produced incrementally — nothing on the unpack path ever sizes an allocation from the pack’s own declaration.
- the counting reader is the defense-in-depth layer the
wave-3 gate confirmed: the pack stream’s byte total is bounded
by
- The working area’s shape: a fresh per-push directory under the
constructor’s root knob (no env — conventions 4/5); the disk
shadow is the pack the counting reader bounds (pack + index ≈ the
pack budget — no new
Limitsfield, ADR-009’s no-new-kinds rule). The authoritative object home is the store; the area (temporary pack + index) is discarded after unpack — on every path, the explicit return paths’ removal riding a best-effort residue removal that also covers a panic’s unwind (Drop’s bounded directory rm — the one blocking best-effort on the async side). - Unpack order and the store’s role: the indexing machinery
normalizes the pack (thin-base entries inserted physically and
their ref-deltas rewritten, in-pack ref-deltas resolved through
the index), so the unpack iterates the written pack in pack order
over the index’s entries; every materialized object re-derives
(the pack-side oid answer), cross-checks against the index’s own
oid, and goes through the store’s batch form — one
alkblobs::Batchgroup over the working set (the store’s pin-before-publish ordering, one commit discipline). Existing content dedups in the store; re-putting every indexed object (rather than skipping known ones) keeps the sweep race closed: an object present-but-sweep-eligible could otherwise vanish mid-window under a bare presence check. Delta bases read back through the Find front (verification-on-get is the same re-derivation guard), one base body at a time — the unpack stage’s memory is the cap’s working set, never O(pack). - Connectivity: the full-graph walk over the substrate Find
from the pushed tips — commits walk parents and trees, trees walk
entries, tags walk targets, blobs terminate (
gix-fsck’sConnectivity::check_commitdoes not walk parents; this is the owning walk, the same two-stage discipline as generation). A missing closure member fails into theunpackng leg naming the missing object. - Status shapes: pack-level validation failures (parse, index,
thin-base miss, the cap, connectivity) land in
PreparedPush::unpack’s ng leg with a server-chosen reason;updatesreturns the input list verbatim on every path (ADR-018 §5);ErrstaysStorageError— infrastructure only (working-area construction, task joins, backing-store legs). The empty pack (zero objects) is valid and reports as such; the push’s tips must then exist in the store (the same connectivity walk, vacuous over deletes-only updates). The ng legs retain nothing — their guards drop with the prepare call and the just-put objects are the same sweep-eligible recovery the crash case rides; a fully ingested push retains itsalkblobs::Batchinto the composition-internalPendingPinRegistrykeyed byrepo_id, correlated content-wise with the verbatim prepared update list (the custody seam the w8-pin-registry task owns).
§Threading
The whole composition runs on tokio::task::spawn_blocking with
owned handles moved in (backend.md §concurrency model — the permit
stays wire-side, the impl’s threading invisible); every sync→async
hop goes through ObjectFront::block_on from the blocking
contexts — no consumer constructs its own bridge.
Structs§
- Pack
Ingest - The substrate’s
GitPackIngestimplement.