Skip to main content

gunnar_wire/
graph.rs

1//! Walking the object graph: what a pack must contain, and whether a push
2//! closes.
3//!
4//! One walker serves both directions (one implementation). `upload-pack` asks
5//! *which objects are reachable from these wants and not from these haves* and
6//! packs the answer; `receive-pack` asks *is everything reachable from these
7//! new tips actually here* and throws the answer away. The difference is which
8//! set is the frontier and whether a missing object is an error, so they are
9//! the same function with two arguments, not two functions that must be kept
10//! agreeing.
11//!
12//! # Honest limits
13//!
14//! The exclusion set is the **full** object closure of the haves, computed by
15//! walking. Correct, since the client demonstrably has every object reachable
16//! from a commit it has, but it costs `O(client history)` per fetch where git
17//! with reachability bitmaps does `O(1)` lookups. The `git` znippy package's
18//! `__gunnar_reach__` module is where that cost goes away; nothing here claims
19//! it has. Sending *fewer* objects than this walk selects would be a bug;
20//! sending more is only slow.
21//!
22//! The *memory* is the same shape and used to be worse than the time: that
23//! closure is `O(repository)` object ids, and [`select_objects`] used to
24//! materialise it **three times over**, once as the exclusion set, once as the
25//! walk's own `seen` set, once more as a `Vec` of the same ids. Then it cloned
26//! the result for step 6. See [`OidSet`].
27
28//! # What this walker reads through, and why it is not `GitOps`
29//!
30//! [`Graph::load`] is the single funnel under [`Graph::peel`],
31//! [`Graph::commit_links`], [`Graph::commit_facts_into`] and
32//! [`Graph::tree_closure`], and every one of them needs the object
33//! **inflated and delta-resolved**. `GitOps::get` deliberately does not give
34//! one: `Stored::bytes` is documented as *"the pack entry, header and all,
35//! byte for byte as the client sent them"* and for the ordinary push that is a
36//! delta, while `Stored::obj_type` is *"the entry's type in the pack"* and may
37//! be `OfsDelta`/`RefDelta`. Handing a delta to `CommitRefIter` would not
38//! error; it would decode garbage.
39//!
40//! So the walk reads through [`GitServe`] — the **reading** contract
41//! (`locked_on_target.md`'s `GitServe` reading contract) — and `GitServe::read`'s kind is the resolved
42//! one. `GitOps` stays the storage contract, narrow, and no adapter in gunnar
43//! re-derives an object model on top of it.
44
45use std::collections::{BinaryHeap, HashMap, HashSet, VecDeque};
46
47use gix_hash::{Kind, ObjectId};
48use gunnar_store::git_storage_trait::{GitServe, ObjType};
49use gunnar_store::object;
50
51use crate::filter::Filter;
52use crate::limits::Limits;
53use crate::{Error, Result};
54
55/// `GitServe`'s resolved object type, as the decoder names it.
56///
57/// **This mapping exists once, here.** `ObjType` is the contract's vocabulary
58/// (six codes, four resolved and two delta) and [`object::Kind`] is the
59/// decoder's (four). The two delta codes cannot come out of `GitServe::read` or
60/// `GitServe::header`, which are defined to answer the *resolved* type — so a
61/// delta code here is a backend that broke its own contract, and it is reported
62/// as a corrupt object rather than quietly mapped onto something.
63///
64/// The refusal itself lives in [`object::Kind::resolved`], next to the four
65/// names; what stays here is the *sentence* said about it, because only this
66/// crate knows which oid was being asked for.
67fn resolved_kind(id: &ObjectId, kind: ObjType) -> Result<object::Kind> {
68    object::Kind::resolved(kind).ok_or_else(|| Error::CorruptObject {
69        oid: *id,
70        reason: format!(
71            "the store answered a resolved read with {}, which is a delta entry \
72             and not an object type",
73            kind.as_str()
74        ),
75    })
76}
77
78/// **The one conversion from a storage fault into a wire error.**
79///
80/// `GitServe` returns `anyhow::Result` — the contract crate depends on
81/// `anyhow` and nothing else so that neither backend has to resolve the
82/// other's dependencies — and everything this crate can say about such a fault
83/// is [`gunnar_store::Error::Backend`]. One function rather than a `map_err`
84/// at each site (one implementation): the day `RefCas`/`RefLocked` (the typed-CAS question)
85/// becomes a typed rejection there is exactly one place that learns about it.
86/// **Nothing may tell two storage faults apart by matching on this string.**
87///
88/// `{:#}` and not `{}`: `anyhow`'s alternate form prints the whole cause chain.
89pub(crate) fn backend(err: impl std::fmt::Display) -> Error {
90    Error::Store(gunnar_store::Error::backend(format!("{err:#}")))
91}
92
93/// One object id out of the contract's vocabulary.
94///
95/// `git-storage-trait` speaks raw bytes — `Oid<'a> = &'a [u8]`, `Vec<u8>` in a
96/// returned row — because it must name no hash library; this crate speaks
97/// `ObjectId`. A length that is not a hash gunnar knows is a **backend fault,
98/// not an object that happens to be absent**, so it is an error rather than a
99/// skip: a selection or an advertisement silently short by the rows it could
100/// not parse is the under-send that exits zero.
101pub(crate) fn oid(bytes: &[u8]) -> Result<ObjectId> {
102    ObjectId::try_from(bytes).map_err(|err| {
103        backend(format_args!(
104            "the store answered with a {}-byte object id: {err}",
105            bytes.len()
106        ))
107    })
108}
109
110/// [`Graph::contains_have`] without a [`Graph`], for the two negotiation sites
111/// that hold only the store.
112///
113/// One implementation, two entry points, because the *reason* a storage fault
114/// reads as `false` here is subtle and must not be written down twice: a
115/// client's `have` naming an object this server does not hold is the normal,
116/// majority outcome, and every consumer of a negative degrades in exactly one
117/// direction — the `have` drops out of `common`, so the pack excludes less and
118/// is **wider and still valid**; it drops out of `Selection::client_has`, so no
119/// thin delta names it as a base and the object is sent whole; `ready` may be
120/// withheld a round longer. A false *positive* would be fatal — a thin pack
121/// whose bases the client lacks is a failed fetch, not a smaller one — and this
122/// cannot produce one.
123///
124/// So a backend fault is not swallowed into a *wrong* answer, only into a
125/// *pessimistic* one. It is still logged, because "the object store is failing"
126/// is not something a server should only be able to infer from a wide pack.
127pub(crate) fn has_for_negotiation(store: &dyn GitServe, id: &gix_hash::oid) -> bool {
128    match store.has(id.as_bytes()) {
129        Ok(present) => present,
130        Err(err) => {
131            tracing::warn!(
132                target: crate::receive_pack::GIT_TARGET,
133                event = "git.have.store_error",
134                oid = %id,
135                error = %format_args!("{err:#}"),
136                "the object store failed answering a `have`; treating it as absent, which \
137                 widens the pack and cannot corrupt it"
138            );
139            false
140        }
141    }
142}
143
144/// How far a tag chain is followed before it is called a loop.
145const MAX_TAG_DEPTH: usize = 16;
146
147/// The default ceiling on object ids one traversal on the **serve** side may
148/// hold at once.
149///
150/// **This is no longer what bounds gunnar's memory.** [`Limits::walk_bytes`] is
151/// — in bytes, which is the unit a machine is sized in and the only unit that
152/// can be pooled across requests. This number survives as an operator's
153/// explicit cap on the size of a clone, reachable as `--fetch-budget`, and it
154/// is charged through the same [`Graph::charge_walk`] so the two cannot drift.
155///
156/// # Three things the comment justifying this number said, and all three were wrong
157///
158/// It said *"objects visited"*. It was charged against objects **emitted** —
159/// `out.len()` on the [`Emit`] each walker happened to be holding.
160///
161/// It said *"well past any real repository (the linux kernel is around ten
162/// million objects total)"*. The kernel is **13 816 503** objects, so it sat
163/// 28% *below* the repository it was chosen against. MEASURED 2026-08-05,
164/// `bench/examples/walkbudget-probe.rs --scale` against a purpose-built
165/// 14 042 000-object repository with no `.bitmap`: **the clone is refused**,
166/// `WalkBudget`, and the same machine served it in 5 460 816 kB of peak RSS and
167/// 64 seconds once the ceiling let it. That is measured now, not extrapolated.
168///
169/// And it was implied to be *one* ceiling. It was compared at four sites
170/// against four different accumulators, so a single fetch could hold two full
171/// budgets and be refused by neither, while the one repository-sized structure
172/// that outlived them both — step 6's `seen` set — was compared against
173/// nothing. [`Graph::charge_walk`] is the single check that replaced them.
174///
175/// # Where it binds, MEASURED, before and after
176///
177/// `bench/examples/walkbudget-probe.rs`, 2026-08-05, one process per
178/// repository, binary-searching the smallest budget at which a clone still
179/// completes rather than assuming the edge is the object count — the probe that
180/// assumed that failed on its first repository, where a budget of 82 served
181/// bifrost's 83 objects.
182///
183/// | corpus | objects | bound at, per-accumulator | bound at, held ids |
184/// |---|---|---|---|
185/// | `h2h-linear-sha1-32c` | 190 | 0.837 | 1.174 |
186/// | `forge-year-8a-8b-12c` | 1 544 | 0.933 | 1.069 |
187/// | `forge-year-8a-240b-12c` | 28 893 | 0.892 | 1.108 |
188/// | `h2h-linear-sha1-2048c` | 36 172 | 0.943 | 1.057 |
189/// | `h2h-linear-sha256-2048c` | 36 172 | 0.943 | 1.057 |
190/// | `h2h-linear-sha1-20480c` | 352 396 | 0.942 | **1.058** |
191///
192/// The old column is the defect in one number: `budget = N` admitted between
193/// 0.84N and 0.94N objects, and **which** was decided by the repository's shape
194/// rather than by anything an operator could see. The new column is the same
195/// ceiling meaning the same thing everywhere — 1.057, 1.057, 1.058 on the three
196/// largest — and it is above 1.0 because a clone holds its peeled tips and its
197/// tags as well as the pack. The two small rows are further out for the same
198/// reason: a fixed handful of extra ids is a large fraction of 190.
199///
200/// **It used to bite only without a bitmap**, and that was the defect rather
201/// than a property: a reachability index answers the clone without entering
202/// this walk at all, which the same probe established by serving a whole clone
203/// at a budget of ONE. Since the storage contract landed the store that answers that way — the znippy
204/// cold tier — is the one the appliance runs, so an operator's `--fetch-budget`
205/// was accepted and ignored on the shipping path. [`Graph::charge_answer`] is
206/// where the same ceiling is now applied to an answer that arrived whole, and
207/// it is the same comparison ([`Graph::charge_budget`]) so the two cannot
208/// drift.
209///
210/// # Why the unit had to change
211///
212/// The thing a ceiling exists to bound is memory, and objects do not convert to
213/// memory at a fixed rate. MEASURED across the same corpus, `VmHWM` per emitted
214/// object: **1 572.9 B** on `forge-year-8a-240b-12c` (28 893 objects, 848 bytes
215/// of content each) against **393.8 B** on the 14M-object corpus (43 bytes of
216/// content each). Four times, at the same order of object count, decided by
217/// what is *in* the repository. See [`WALK_BYTES_PER_OBJECT`] for the
218/// two-term fit that does predict it and for what an id-counted ceiling can
219/// honestly claim.
220///
221/// [`Limits::walk_bytes`]: crate::Limits::walk_bytes
222/// [`WALK_BYTES_PER_OBJECT`]: crate::WALK_BYTES_PER_OBJECT
223pub const DEFAULT_BUDGET: usize = 10_000_000;
224
225/// The ceiling on objects visited by **`receive-pack`'s connectivity check**.
226///
227/// # Why this is not [`DEFAULT_BUDGET`], which would be one constant
228///
229/// Because they bound two different quantities, and one implementation is
230/// about not writing the same thing twice rather than about giving two things
231/// one name. A fetch's budget stands in front of everything the server is
232/// about to *materialise*: several repository-sized id vectors and then the
233/// pack itself. This walk materialises nothing — since the one-walk fix it holds
234/// exactly one repository-sized [`OidSet`], the `seen` set, because
235/// [`verify_connectivity`] tallies its output instead of collecting it — and
236/// it runs over objects the server has **already received and indexed**, whose
237/// size `gunnar_pack::IndexOptions` bounded on the way in. A push has
238/// therefore already paid its admission by the time this runs; a fetch has
239/// not.
240///
241/// Sixty-four million is 4.6x the kernel's 13 816 503 objects. At sha1 the one
242/// set costs ~25 bytes per id held, so a walk that genuinely reached this
243/// ceiling would hold ~1.6 GB — a bound, and an honest one for a machine that
244/// is asked to accept a kernel-sized mirror push at all.
245pub const CONNECTIVITY_BUDGET: usize = 64_000_000;
246
247/// A set of object ids that a traversal grows, and that is therefore
248/// `O(repository)`.
249///
250/// It exists for two reasons, and both are the one-set rule.
251///
252/// **It does not implement `Clone`, deliberately.** It exists to delete one
253/// line, `let mut seen = excluded_objects.clone();`. One character of syntax
254/// that doubled the peak memory of every in-flight fetch, at a size (ten
255/// million ids, ~450 MB) where doubling is an out-of-memory kill rather than a
256/// slowdown. A guard watching for that line is a guard somebody has to keep
257/// running; a type that cannot be cloned is a compile error the day it comes
258/// back. Fix by construction, one implementation.
259///
260/// **It is metered.** Under `cfg(test)` every id held in one of these is
261/// counted, so a test can assert what the *peak* was. See [`meter`] and
262/// `select_objects_holds_the_exclusion_closure_once`. Sets that are bounded by
263/// what the client sent (a shallow frontier, a `have` list) are deliberately
264/// left as plain [`HashSet`]s: they are not the ones that scale with the
265/// repository, and metering them would blur the number that matters.
266///
267/// The API is the narrow slice the walkers use. Everything read-only comes from
268/// `Deref`, so a `&OidSet` still passes where a `&HashSet<ObjectId>` is wanted.
269#[derive(Debug, Default)]
270pub(crate) struct OidSet {
271    ids: HashSet<ObjectId>,
272}
273
274impl OidSet {
275    pub(crate) fn new() -> Self {
276        OidSet::default()
277    }
278
279    /// Add `id`, reporting whether it was new. Same contract as
280    /// [`HashSet::insert`].
281    pub(crate) fn insert(&mut self, id: ObjectId) -> bool {
282        let fresh = self.ids.insert(id);
283        if fresh {
284            meter::acquire(1);
285        }
286        fresh
287    }
288
289    /// Hand the ids over as a plain [`HashSet`], releasing the meter.
290    ///
291    /// The one legitimate way out of this type. It is **not** a `Clone` in
292    /// disguise: the set is moved, so nothing is duplicated and the peak does
293    /// not move — which is the entire property this type exists to keep.
294    pub(crate) fn into_inner(mut self) -> HashSet<ObjectId> {
295        let ids = std::mem::take(&mut self.ids);
296        // `Drop` releases `self.ids.len()`, which is now zero, so release the
297        // real count here. Getting this wrong would leave the meter permanently
298        // high and quietly break every test that reads a peak.
299        meter::release(ids.len());
300        ids
301    }
302
303    pub(crate) fn extend(&mut self, ids: impl IntoIterator<Item = ObjectId>) {
304        let before = self.ids.len();
305        self.ids.extend(ids);
306        meter::acquire(self.ids.len() - before);
307    }
308
309    /// Take `id` back out, reporting whether it was there.
310    ///
311    /// The one caller is [`select_objects`]'s step 6, which uses the same set
312    /// as its don't-send set and as the client's-closure voucher and has to
313    /// separate the two again at the end. Removing is what keeps that *one*
314    /// set: the alternative was a second, repository-sized copy taken before
315    /// the tree walk, which is exactly the line this type was invented to make
316    /// impossible.
317    pub(crate) fn remove(&mut self, id: &ObjectId) -> bool {
318        let gone = self.ids.remove(id);
319        if gone {
320            meter::release(1);
321        }
322        gone
323    }
324}
325
326impl std::ops::Deref for OidSet {
327    type Target = HashSet<ObjectId>;
328    fn deref(&self) -> &HashSet<ObjectId> {
329        &self.ids
330    }
331}
332
333impl Drop for OidSet {
334    fn drop(&mut self) {
335        meter::release(self.ids.len());
336    }
337}
338
339impl FromIterator<ObjectId> for OidSet {
340    fn from_iter<I: IntoIterator<Item = ObjectId>>(ids: I) -> Self {
341        let mut set = OidSet::new();
342        set.extend(ids);
343        set
344    }
345}
346
347/// How many object ids the walkers are holding, and the most they ever held.
348///
349/// # It used to be `#[cfg(test)]`, and that is why the ceiling was wrong
350///
351/// This was a test instrument: in a shipping build every function was an empty
352/// inline body and [`OidSet`] was a `HashSet` with a newtype around it. So the
353/// one place in gunnar that knew what a traversal was **holding** could not be
354/// consulted by the thing whose job is to bound it, and
355/// [`DEFAULT_BUDGET`] instead charged the nearest accumulator to hand —
356/// a different one at each of its four check sites, none of them the total.
357/// See [`Graph::charge_walk`].
358///
359/// It costs a non-atomic `Cell` read-modify-write per id, on a path that has
360/// just done a hash insert. Thread-local rather than atomic because a
361/// traversal is single-threaded — `gunnar-cli` gives each transfer one blocking
362/// thread for its whole duration — so the thread *is* the request, which also
363/// gives libtest's per-test threads a per-test peak for free.
364pub(crate) mod meter {
365    use std::cell::Cell;
366
367    thread_local! {
368        static LIVE: Cell<usize> = const { Cell::new(0) };
369        static PEAK: Cell<usize> = const { Cell::new(0) };
370    }
371
372    pub(crate) fn acquire(count: usize) {
373        if count == 0 {
374            return;
375        }
376        let live = LIVE.with(|live| {
377            let now = live.get() + count;
378            live.set(now);
379            now
380        });
381        PEAK.with(|peak| {
382            if live > peak.get() {
383                peak.set(live);
384            }
385        });
386    }
387
388    pub(crate) fn release(count: usize) {
389        LIVE.with(|live| live.set(live.get().saturating_sub(count)));
390    }
391
392    /// Forget the peak. Live ids are left alone; a set that outlives the reset
393    /// is still held, and pretending otherwise would let a test start from a
394    /// lie.
395    pub(crate) fn reset_peak() {
396        PEAK.with(|peak| peak.set(LIVE.with(|live| live.get())));
397    }
398
399    /// The most ids held at once since the last [`reset_peak`].
400    ///
401    /// Test-only, and the only part of this module that still is: the peak is
402    /// what a test asserts against, while the shipping ceiling reads
403    /// [`live`] — a request is refused for what it is holding now, not for what
404    /// it held before something was freed.
405    #[cfg(test)]
406    pub(crate) fn peak() -> usize {
407        PEAK.with(|peak| peak.get())
408    }
409
410    /// Ids held right now.
411    pub(crate) fn live() -> usize {
412        LIVE.with(|live| live.get())
413    }
414
415    /// Put `live` back to `to`, which is what a request's scope guard does when
416    /// it ends.
417    ///
418    /// An assignment and not a subtraction, on purpose. A [`super::Selection`]
419    /// outlives the walk that produced it and may be dropped on another thread
420    /// — the pack writer's — and that drop releases against a counter that is
421    /// not this one. Subtracting would leave every subsequent request on this
422    /// thread starting from a number that drifts one way forever; assigning
423    /// makes each request's charge exactly its own.
424    pub(crate) fn restore(to: usize) {
425        LIVE.with(|live| live.set(to));
426    }
427}
428
429/// Where a tree walk puts what it finds.
430///
431/// Two callers want two different things out of the same walk. The fetch path
432/// needs the ids, in order, because it is about to pack them. The exclusion
433/// walk needs only that they were *visited*; their ids are already in its
434/// `seen` set, and building a second repository-sized `Vec` of them is half of
435/// One walker, two sinks (one implementation).
436///
437/// **Neither arm is metered, and that is deliberate rather than an omission.**
438///
439/// `Tally` holds nothing by construction. `Keep` holds a `Vec`, and a `Vec` has
440/// no hook to release on — charging its pushes would acquire against
441/// [`meter`] with nothing on the other side, so a `Graph` that served two
442/// selections would bill the second for the first. That is not hypothetical:
443/// it is what `a_fetch_holds_the_clients_object_closure_once` reported the
444/// first time this was tried, 610 ids still charged after the set that held
445/// them was gone.
446///
447/// It costs nothing to leave out. Every id a `Keep` walk emits was put into
448/// that walk's `seen` set on the way — [`Graph::tree_closure`] pushes only what
449/// `seen.insert` just accepted — and `seen` is an [`OidSet`], which is metered
450/// exactly because it has the `Drop` that keeps acquire and release balanced.
451/// So the emitted vector is a second copy of a subset of a set that is already
452/// charged, and [`WALK_BYTES_PER_OBJECT`](crate::WALK_BYTES_PER_OBJECT) is a
453/// whole-process rate measured with that vector, the writer's copy of it and
454/// gix's own lists all resident.
455pub(crate) enum Emit<'v> {
456    /// Keep them.
457    Keep(&'v mut Vec<ObjectId>),
458    /// Count them and throw them away.
459    Tally(usize),
460}
461
462impl Emit<'_> {
463    fn push(&mut self, id: ObjectId) {
464        match self {
465            Emit::Keep(out) => out.push(id),
466            Emit::Tally(count) => *count += 1,
467        }
468    }
469
470    fn len(&self) -> usize {
471        match self {
472            Emit::Keep(out) => out.len(),
473            Emit::Tally(count) => *count,
474        }
475    }
476}
477
478/// What a tree walk remembers, so that each object is visited exactly once.
479///
480/// # Why this is a trait and not an [`OidSet`]
481///
482/// `Graph::tree_closure` is the **one** tree walk in gunnar (one
483/// implementation), and it now has two callers that record what it found in
484/// different shapes. The walk path records object ids, because that is what it
485/// hands back. The bitmap path records *bits*, because its whole point is that
486/// the client's history never becomes a repository-sized set of ids — and
487/// because a bit already set by some commit's stored bitmap is exactly the
488/// signal to stop descending, which an id set cannot give.
489///
490/// Generic rather than `&mut dyn`: this is called once per object in the
491/// transfer, and a virtual call per object is a cost with nothing behind it.
492pub(crate) trait Seen {
493    /// Record `id`, and say whether the walk should descend into it.
494    ///
495    /// `false` means "already accounted for" — either seen in this walk or
496    /// known to be on the far side of the cut.
497    fn insert(&mut self, id: &ObjectId) -> bool;
498}
499
500impl Seen for OidSet {
501    fn insert(&mut self, id: &ObjectId) -> bool {
502        OidSet::insert(self, *id)
503    }
504}
505
506/// The don't-send set, keeping a note of every id **this transfer** added to
507/// it.
508///
509/// [`select_objects`] uses one set for two questions that are not the same
510/// question: *has the walk already accounted for this id?* and *does the client
511/// hold it?* Step 6 answers the first by inserting everything it visits, which
512/// makes the second one wrong for exactly the ids it inserted — and vouching
513/// for one of those is the thin-pack defect [`Selection::client_has`] documents.
514///
515/// So the ids step 6 adds are listed as they go and taken back out at the end.
516/// The list is bounded by what the transfer *touches*, not by the repository,
517/// which is what keeps the one repository-sized set at one.
518struct Carried<'a> {
519    seen: &'a mut OidSet,
520    carried: &'a mut Vec<ObjectId>,
521}
522
523impl Seen for Carried<'_> {
524    fn insert(&mut self, id: &ObjectId) -> bool {
525        let fresh = self.seen.insert(*id);
526        if fresh {
527            self.carried.push(*id);
528        }
529        fresh
530    }
531}
532
533/// What to do about an object that is referenced but absent.
534#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
535pub(crate) enum Missing {
536    /// Fail. Used for everything the server is about to *send*, and for the
537    /// connectivity check on a push.
538    #[default]
539    Fail,
540    /// Skip it. Used for the client's `have`s: a shallow or damaged client may
541    /// name a commit whose ancestry it does not have, and that is its business,
542    /// not a server error.
543    Skip,
544}
545
546/// Read-side view of the object database, with the parsing gunnar needs.
547///
548/// **One of these is one request's traversal scope.** Constructing it snapshots
549/// the thread's live-id count and resets the peak; dropping it puts the count
550/// back and hands any [`WalkPool`] reservation back. See [`Graph::charge_walk`].
551pub(crate) struct Graph<'a> {
552    store: &'a dyn GitServe,
553    hash_kind: Kind,
554    /// An operator's explicit cap on objects held, when one was set.
555    budget: Option<usize>,
556    /// Bytes of bookkeeping this traversal may hold, from
557    /// [`Limits::walk_bytes`].
558    walk_bytes: Option<usize>,
559    /// The server-wide purse, when the server gave this connection one.
560    pool: Option<std::sync::Arc<crate::WalkPool>>,
561    /// What this traversal currently holds from [`Self::pool`].
562    reserved: std::cell::Cell<usize>,
563    /// Ids the thread was already holding when this scope opened.
564    entered_at: usize,
565}
566
567/// The traversal scope ends here, and both halves of it are released.
568///
569/// The [`WalkPool`](crate::WalkPool) reservation must come back or the pool
570/// closes for good after enough requests, and `meter::live` must go back to
571/// what it was or the *next* request on this thread starts already charged. The
572/// second is the one that is easy to get wrong: a [`Selection`] outlives its
573/// `Graph` and is dropped by the pack writer, possibly on another thread, so
574/// this cannot be a subtraction of what the walk acquired.
575impl Drop for Graph<'_> {
576    fn drop(&mut self) {
577        if let Some(pool) = &self.pool {
578            pool.release(self.reserved.get());
579        }
580        meter::restore(self.entered_at);
581    }
582}
583
584impl<'a> Graph<'a> {
585    /// A traversal scope over `store`, for a repository whose ids are
586    /// `hash_kind`.
587    ///
588    /// # Why the hash kind is a parameter now
589    ///
590    /// It was `store.hash_kind()`. [`GitServe`] has no such method: the frozen
591    /// signature leaves
592    /// *"whether `hash_kind()` belongs on `GitOps`"* open, because it needs a
593    /// `HashKind` type in a crate that depends on `anyhow` and nothing else.
594    /// Threading it in from the caller settles the question for this crate
595    /// without pre-empting it for the contract — `RepoStore::hash_kind()`
596    /// already exists and the server already holds one per repository, so the
597    /// value is not being invented, only carried the last hop. The parsers here
598    /// need it to size a parent oid, and one wrong answer silently decodes a
599    /// sha256 commit as sha1, so it is a required argument rather than a
600    /// defaulted one.
601    pub(crate) fn new(store: &'a dyn GitServe, hash_kind: Kind) -> Self {
602        let entered_at = meter::live();
603        meter::reset_peak();
604        Graph {
605            hash_kind,
606            store,
607            budget: None,
608            walk_bytes: None,
609            pool: None,
610            reserved: std::cell::Cell::new(0),
611            entered_at,
612        }
613    }
614
615    /// An operator's object cap, when `gunnar serve` was given one.
616    ///
617    /// `None` is the shipping default and is **not** an unbounded walk:
618    /// [`Limits::walk_bytes`](crate::Limits::walk_bytes) is what bounds it, in
619    /// the unit a machine is sized in. Leaving this at
620    /// [`DEFAULT_BUDGET`] would have kept the kernel-sized clone refused
621    /// however generous the byte ceiling was, which is the whole thing being
622    /// fixed.
623    pub(crate) fn with_budget(mut self, budget: Option<usize>) -> Self {
624        self.budget = budget;
625        self
626    }
627
628    /// The bytes ceiling and the shared purse this traversal serves under.
629    ///
630    /// Separate from [`Self::with_budget`] because they are different
631    /// quantities: `budget` is an operator's explicit cap on how many objects a
632    /// request may touch, and this is what the machine can hold. The push
633    /// side's connectivity walk sets only the first (see
634    /// [`CONNECTIVITY_BUDGET`]), because a push has already paid its admission
635    /// through `gunnar_pack::IndexOptions` by the time it runs.
636    pub(crate) fn with_walk_limit(
637        mut self,
638        walk_bytes: Option<usize>,
639        pool: Option<std::sync::Arc<crate::WalkPool>>,
640    ) -> Self {
641        self.walk_bytes = walk_bytes;
642        self.pool = pool;
643        self
644    }
645
646    /// **The one place a traversal's size is judged.**
647    ///
648    /// # The four checks this replaces, and why four was three too many
649    ///
650    /// [`DEFAULT_BUDGET`] was tested at four sites and each one compared a
651    /// *different* accumulator against the same number:
652    /// [`Graph::commit_closure`] its own commit list,
653    /// [`Graph::limited_closure`] its own, [`Graph::tree_closure`] the
654    /// [`Emit`] it was handed, [`Reachability`] a visit counter. A single fetch
655    /// runs the first (or second) and then the third, so it could hold two full
656    /// budgets and be refused by neither — and step 5's `seen` set, which is
657    /// repository-sized and outlives both, was compared against nothing at all.
658    /// Under a `blob:none` filter that set grows with every blob in the
659    /// repository while the `Emit` being charged grows with none of them.
660    ///
661    /// So the check is against [`meter::live`], the ids the thread is actually
662    /// holding, minus what it was already holding when this scope opened. One
663    /// number, every accumulator in it by construction (one implementation): a new
664    /// repository-sized [`OidSet`] is charged the day it is written, without
665    /// anyone remembering to add a fifth comparison.
666    ///
667    /// # Nothing is inflated to charge it
668    ///
669    /// The bytes come from the id count and [`WALK_BYTES_PER_OBJECT`], a
670    /// measured rate. A budget that had to decompress an object to learn what to
671    /// charge for it would be re-creating the defect that made `gix-odb` inflate
672    /// every object just to read its compressed length, which cost 51.3% of
673    /// clone CPU.
674    ///
675    /// [`WALK_BYTES_PER_OBJECT`]: crate::WALK_BYTES_PER_OBJECT
676    fn charge_walk(&self) -> Result<()> {
677        let held = meter::live().saturating_sub(self.entered_at);
678        self.charge_budget(held)?;
679        let bytes = held.saturating_mul(crate::WALK_BYTES_PER_OBJECT);
680        if let Some(limit) = self.walk_bytes {
681            if bytes > limit {
682                return Err(Error::WalkTooLarge { held: bytes, limit });
683            }
684        }
685        // The shared purse, topped up in chunks. `reserved` only ever grows
686        // within one traversal: handing bytes back as a walk's `seen` set
687        // shrinks would let a second transfer take the room this one is about
688        // to need again, which is the shape that deadlocks a pool rather than
689        // bounding it. It all comes back at `Drop`.
690        if let Some(pool) = &self.pool {
691            if bytes >= crate::WalkPool::FLOOR_BYTES && bytes > self.reserved.get() {
692                let want = bytes.next_multiple_of(crate::WalkPool::CHUNK_BYTES);
693                pool.try_reserve(want - self.reserved.get())?;
694                self.reserved.set(want);
695            }
696        }
697        Ok(())
698    }
699
700    /// `--fetch-budget`, and the **one** comparison that applies it.
701    ///
702    /// Written once because it now has two callers that count two different
703    /// things — [`Self::charge_walk`] the ids a walker is holding,
704    /// [`Self::charge_answer`] the size of an answer nobody walked for — and an
705    /// operator's ceiling that meant `>=` on one path and `>` on the other
706    /// would be a flag whose meaning depended on which store was underneath.
707    fn charge_budget(&self, held: usize) -> Result<()> {
708        if let Some(limit) = self.budget {
709            if held >= limit {
710                return Err(Error::WalkBudget { limit });
711            }
712        }
713        Ok(())
714    }
715
716    /// The operator's object cap, charged against an answer that arrived
717    /// **whole**.
718    ///
719    /// # `--fetch-budget` was accepted and ignored by every store gunnar ships
720    ///
721    /// [`Self::charge_walk`] reads [`meter::live`], which counts ids a *walker*
722    /// is holding. On the path [`crate::reachable::select`] takes there is no
723    /// walker: the store does the set algebra over its own reachability index
724    /// and hands back the finished closure, so `meter::live()` is still zero
725    /// when `select_objects` returns at step 0 and the budget is never
726    /// compared against anything. That step is not a rare shape — since the storage
727    /// contract landed the
728    /// znippy cold tier is the storage engine the appliance runs, and it
729    /// answers every unfiltered, non-shallow clone this way. So an operator
730    /// setting `--fetch-budget 1` got a whole clone and exit 0.
731    ///
732    /// [`crate::reachable`] already had to make exactly this argument once, for
733    /// [`crate::Limits::object_bytes`]: a ceiling that quietly stops applying
734    /// on the fast path is worse than no ceiling, because every test of it
735    /// stays green. This is the second one, found the same way.
736    ///
737    /// # Why this charges the budget and not [`crate::Limits::walk_bytes`]
738    ///
739    /// They are ceilings on two different things and only one of them is an
740    /// operator's stated intent. `budget` is *"refuse a clone bigger than N
741    /// objects"*, and N objects is exactly what the store just named — the
742    /// count is exact here, where the walk can only ever approximate it.
743    /// `walk_bytes` and the [`crate::WalkPool`] bound what a *traversal* holds,
744    /// and this path runs no traversal; charging them here would start refusing
745    /// clones the appliance serves today, on a default that is on. That is a
746    /// shipping decision and it is not this function's to take — but the answer
747    /// path does hold a repository-sized `Vec<ObjectId>` that nothing charges,
748    /// and that gap is real and unowned.
749    pub(crate) fn charge_answer(&self, objects: usize) -> Result<()> {
750        self.charge_budget(objects)
751    }
752
753    /// Is this object here?
754    ///
755    /// # It returns a `Result` now, and that is not tidiness
756    ///
757    /// `ObjectStore::contains` was infallible `bool`; `GitOps::has` is
758    /// `Result<bool>`. The tempting adaptation is `.unwrap_or(false)`, and at
759    /// three of this method's call sites it would be a **silent under-send**:
760    /// `Missing::Skip` drops an object it is told is absent, so a backend fault
761    /// would quietly narrow a pack and exit zero. That is the shape a measured
762    /// assertion is about. The fault propagates instead, and the one place
763    /// where a negative really is the answer keeps its own method —
764    /// [`Graph::contains_have`].
765    pub(crate) fn contains(&self, id: &gix_hash::oid) -> Result<bool> {
766        self.store.has(id.as_bytes()).map_err(backend)
767    }
768
769    /// [`Graph::contains`] for a **`have`**, where the client not having been
770    /// heard of is the answer rather than a reason to go and look at the disk
771    /// again.
772    ///
773    /// Only negotiation may call this. A `want`, a delta base and the walk's own
774    /// `Missing::Skip` filter all keep [`Graph::contains`], because for them a
775    /// stale `false` is a refused fetch rather than a wider pack.
776    ///
777    /// # What was lost, and it can only cost syscalls
778    ///
779    /// `ObjectStore` had two questions here, `contains` and
780    /// `contains_without_rescan`, and the difference was whether a miss was
781    /// allowed to trigger a re-scan of the backing store first. git draws the
782    /// same line (`OBJECT_INFO_QUICK` against `HAS_OBJECT_RECHECK_PACKED`).
783    /// `GitOps` has one `has`, so the distinction is gone and every question
784    /// gets whatever the backend does. The **answers** are unchanged; only the
785    /// syscalls a negotiation may spend re-deriving a `false` it already had
786    /// are, and nothing in this crate can observe that. It is recorded here
787    /// rather than dropped because it is a measured optimisation that a future
788    /// `GitServe` method could restore, not a rule that stopped being true.
789    pub(crate) fn contains_have(&self, id: &gix_hash::oid) -> bool {
790        has_for_negotiation(self.store, id)
791    }
792
793    /// The object database underneath, for the callers that need to ask it
794    /// something a graph walk cannot — a bulk `sizes`, a `select`.
795    pub(crate) fn store(&self) -> &'a dyn GitServe {
796        self.store
797    }
798
799    /// Load an object's content into `buf`, returning its kind.
800    ///
801    /// # 🔴 The serial closure walk lives through this function. Read before changing it.
802    ///
803    /// Every commit in the repository used to be read and decoded **twice** —
804    /// once for its parents and once for the root tree the first decode had
805    /// already parsed and discarded. Removing that measured **1.295x on
806    /// `walk_1m`** and **1.025x on `clone_large`**, instructions retired, both
807    /// sides, and the guard that holds it is `commit_reads_per_commit` in
808    /// `nornir-bench`: it must read **1.000**, never 2.000. What keeps it there
809    /// is that [`Graph::commit_links`] makes exactly **one** call to this
810    /// function and returns the tree *and* the parents out of that one decode.
811    /// Retargeting the store underneath is exactly where a second call gets
812    /// added by accident.
813    ///
814    /// # The buffer, and what the frozen signature costs it
815    ///
816    /// `buf` belongs to the caller so that a walk of a million commits reuses
817    /// one allocation; the allocator was **14.2%** of `gunnar.walk_1m`'s
818    /// profile on 2026-08-03, which is why it is a parameter at all.
819    /// `GitServe::read`, as frozen, returns an owned `Vec<u8>`, so the
820    /// allocation happens inside the backend and the caller's is dropped. The
821    /// move below (`*buf = bytes`) is the least-bad adaptation — it is a move,
822    /// not a copy, so no object's bytes are duplicated — but it is one malloc
823    /// and one free per object where there used to be neither. **This is a
824    /// signature question, not a bug here**: a defaulted `read_into` on
825    /// `GitServe`, costing an implementer nothing, restores the reuse. It is
826    /// reported rather than invented, because the contract is frozen and
827    /// changing it is a two-implementation decision.
828    ///
829    /// The magnitude of that regression is an **unmeasured hypothesis**: the
830    /// bench target does not build and nothing has been run.
831    fn load(&self, id: &ObjectId, what: &str, buf: &mut Vec<u8>) -> Result<object::Kind> {
832        match self.store.read(id.as_bytes()).map_err(backend)? {
833            Some((kind, bytes)) => {
834                *buf = bytes;
835                resolved_kind(id, kind)
836            }
837            None => Err(Error::missing(*id, what.to_owned())),
838        }
839    }
840
841    fn corrupt(&self, id: &ObjectId, err: impl std::fmt::Display) -> Error {
842        Error::CorruptObject {
843            oid: *id,
844            reason: err.to_string(),
845        }
846    }
847
848    /// The object's kind, or `None` if it is not here.
849    ///
850    /// A header read, never an object read: `GitServe::header` answers kind and
851    /// post-resolution size *without* the payload, which is the whole reason it
852    /// is a separate method from `read`.
853    pub(crate) fn kind(&self, id: &ObjectId) -> Result<Option<object::Kind>> {
854        match self.store.header(id.as_bytes()).map_err(backend)? {
855            Some((kind, _size)) => Ok(Some(resolved_kind(id, kind)?)),
856            None => Ok(None),
857        }
858    }
859
860    /// Follow a tag chain to the first non-tag object.
861    ///
862    /// The intermediate tag objects come back too: a `want` naming an
863    /// annotated tag has to put the tag itself in the pack, not only what it
864    /// points at.
865    pub(crate) fn peel(&self, id: &ObjectId) -> Result<Peeled> {
866        let mut tags = Vec::new();
867        let mut current = *id;
868        let mut buf = Vec::new();
869        for depth in 0..MAX_TAG_DEPTH {
870            // **The label is the first thing an operator reads.** Every read
871            // here used to be called `"peeling a tag"`, including the first
872            // one — so a push whose tip was simply absent came back as
873            // `missing necessary objects: <oid> is not in this repository
874            // (peeling a tag)` with no tag anywhere in the push, the
875            // repository or the pack. bench `bare_lifecycle` refused a
876            // force-push rewind with exactly that sentence and it was read as
877            // a tag-peeling defect in the connectivity check; the protocol
878            // half is clean under every tag shape a frontier can hold
879            // (`tests/force_update.rs`), and what the sentence was actually
880            // reporting was an object the store did not have.
881            //
882            // Only a read that follows a tag's target is tag peeling. The
883            // first read is whatever the caller handed in.
884            let what = if depth == 0 {
885                "resolving an object"
886            } else {
887                "peeling a tag"
888            };
889            // **Kind is a HEADER question; the inflated read is only for a
890            // tag's BODY.** `header` answers kind off the index
891            // (`GitServe::header`) without pulling the object through the
892            // exploded table. The exploded read below is genuinely needed only
893            // to parse a tag's *target* out of its body — so for the terminal
894            // object of a peel (a commit, in the overwhelming common case of
895            // one annotated tag over a commit) `read` was pure waste: a full
896            // inflated pull, and on a many-ref repository a 64 MiB Arrow batch
897            // decode that thrashes `ExplodedTable::CACHE`.
898            // `advertise_from` peels every annotated tag on every advertisement
899            // (`ls-refs`, clone, fetch, replication), so this halves the
900            // exploded reads that whole path costs — reads per advertised tag
901            // fall from two (the tag *and* the commit it names) to one (the tag
902            // alone). The before/after µs/ref were measured.
903            let Some(kind) = self.kind(&current)? else {
904                return Err(Error::missing(current, what.to_owned()));
905            };
906            if kind != object::Kind::Tag {
907                return Ok(Peeled {
908                    id: current,
909                    kind,
910                    tags,
911                });
912            }
913            // It IS a tag: the body is now genuinely needed, to read the target
914            // it names. `load` re-confirms the kind it just derived — the two
915            // must agree (`serve.rs::header_resolves_every_delta_to_its_real_kind`).
916            let loaded = self.load(&current, "peeling a tag", &mut buf)?;
917            debug_assert_eq!(loaded, object::Kind::Tag);
918            let target = object::tag_target(&buf, self.hash_kind)
919                .map_err(|err| self.corrupt(&current, err))?;
920            tags.push(current);
921            current = target;
922        }
923        Err(self.corrupt(id, format!("tag chain deeper than {MAX_TAG_DEPTH}")))
924    }
925
926    /// A commit's tree and parents, and nothing else.
927    ///
928    /// [`Graph::commit`] answers a superset of this by way of
929    /// `CommitRef::from_bytes`, which decodes the **whole** object: both
930    /// signatures, both dates, the encoding, the extra headers and the message.
931    /// The closure walks want two fields off the front of the object and never
932    /// look at the rest, and `committed_at` — the one field of [`CommitFacts`]
933    /// that is not here — is read only by negotiation, which asks about a
934    /// handful of commits rather than about every commit in the repository.
935    ///
936    /// `CommitRefIter` decodes only as far as it is driven, and git's own
937    /// commit grammar puts `tree` first and every `parent` next, so stopping at
938    /// the first non-parent token leaves the committer, the encoding, the extra
939    /// headers and the message unparsed. It does **not** make the walk
940    /// signature-free: the iterator reports "no more parents" by decoding the
941    /// *author*, so one signature and one date parse per commit remain. Halving
942    /// that work is what is on offer, not removing it — `gix_actor` and
943    /// `gix_date` together were 2.8% of `gunnar.walk_1m`'s profile on
944    /// 2026-08-03.
945    ///
946    /// Both buffers belong to the caller and are reused across commits. This
947    /// runs once per commit in the repository, and a `Vec::new()` per call is a
948    /// malloc, a grow and a free per commit; the allocator was 14.2% of that
949    /// same profile.
950    pub(crate) fn commit_links(
951        &self,
952        id: &ObjectId,
953        buf: &mut Vec<u8>,
954        parents: &mut Vec<ObjectId>,
955    ) -> Result<ObjectId> {
956        let kind = self.load(id, "reading a commit", buf)?;
957        if kind != object::Kind::Commit {
958            return Err(self.corrupt(id, format!("expected a commit, found a {kind}")));
959        }
960        // `tree`, then `parent`*, then `author`: `commit_links` stops at the
961        // first line that is not a parent, so the committer, the encoding, the
962        // extra headers and the message are never looked at.
963        object::commit_links(buf, self.hash_kind, parents).map_err(|err| self.corrupt(id, err))
964    }
965
966    /// A commit's parents, tree and committer timestamp.
967    pub(crate) fn commit(&self, id: &ObjectId) -> Result<CommitFacts> {
968        let mut buf = Vec::new();
969        self.commit_facts(id, &mut buf)
970    }
971
972    /// [`Graph::commit`] with the caller's buffer, for a walk that reads
973    /// thousands of commits.
974    ///
975    /// **One pass of `CommitRefIter`, stopping at the committer.** This used to
976    /// be `CommitRef::from_bytes`, which decodes the *whole* object — both
977    /// signatures, both dates, the encoding, every extra header and the entire
978    /// message — to hand back three fields. `commit_links` above is this same
979    /// reader with the date left off, and it already had to decode the author
980    /// to learn that the parents were done; the committer is the very next
981    /// token after that, so the date is nearly free and the message never gets
982    /// looked at. One decoder rather than two (one implementation).
983    ///
984    /// The date is `SignatureRef::seconds`, which parses the leading integer
985    /// and ignores the timezone, where the old reader ran the strict
986    /// `gix_date` header parser and fell back to `0`. A well-formed commit
987    /// gives the identical number either way; a malformed one now yields its
988    /// seconds instead of the epoch, which is git's own leniency
989    /// (`parse_commit_buffer` does the same) and is strictly better for a walk
990    /// that orders by that number.
991    fn commit_facts(&self, id: &ObjectId, buf: &mut Vec<u8>) -> Result<CommitFacts> {
992        let mut parents = Vec::new();
993        let (tree, committed_at) = self.commit_facts_into(id, buf, &mut parents)?;
994        Ok(CommitFacts {
995            tree,
996            parents,
997            committed_at,
998        })
999    }
1000
1001    /// [`Graph::commit_facts`] into the **caller's** buffers, returning
1002    /// `(tree, committed_at)`.
1003    ///
1004    /// Same decoder, same single pass; the only difference is who owns the two
1005    /// `Vec`s. That matters wherever this is called once per commit over a
1006    /// list, for the reason [`Graph::commit_links`]'s own doc gives — a
1007    /// `Vec::new()` per call is a malloc, a grow and a free per commit, and the
1008    /// allocator was 14.2% of `gunnar.walk_1m`'s profile on 2026-08-03.
1009    ///
1010    /// # The measurement that put this here, **and what it did not explain**
1011    ///
1012    /// `negotiate::HaveMemo::resolve` used to call [`Graph::commit`] and read
1013    /// only `committed_at`, so `CommitFacts::parents` was dead and the compiler
1014    /// was free to delete the vector and the loop that fills it. The moment
1015    /// negotiation began *using* the parents (git's `THEY_HAVE` marking, see
1016    /// [`crate::negotiate::Negotiation::ack`]) that stopped, and the difference
1017    /// showed up as **+7.6 ms of server CPU** on a fetch across a 500-commit
1018    /// divergence with 1 008 `have`s — MEASURED 2026-08-04,
1019    /// `bench/examples/fetchbytes-probe.rs`, 29.0 → 36.6 ms median of five,
1020    /// isolated by keeping the parent read and removing only its consumer.
1021    ///
1022    /// **This entry point did not recover it, and that is recorded rather than
1023    /// quietly dropped.** The obvious reading of that 7.6 ms was the allocation
1024    /// [`Graph::commit_links`]'s doc names — a malloc, a grow and a free per
1025    /// commit. Hoisting both buffers out of the loop, which is exactly what
1026    /// this method exists to allow, moved the number by **nothing** (36.6 →
1027    /// 37.4 ms, inside the band). So the cost is the parent *decode*, not the
1028    /// allocation behind it, and what remains unexplained is why decoding
1029    /// ~1 000 parent lines costs 7.6 ms. It is kept because one buffer pair per
1030    /// negotiation is right regardless of what else is slow, not because it was
1031    /// measured to be worth 7.6 ms — it was measured not to be.
1032    ///
1033    /// The cost is confined to the **walk**: on the bitmap path, which is what
1034    /// every repacked repository serves from, the same change measured 0.0231 →
1035    /// 0.0217 s, i.e. no regression at all.
1036    fn commit_facts_into(
1037        &self,
1038        id: &ObjectId,
1039        buf: &mut Vec<u8>,
1040        parents: &mut Vec<ObjectId>,
1041    ) -> Result<(ObjectId, i64)> {
1042        let kind = self.load(id, "reading a commit", buf)?;
1043        if kind != object::Kind::Commit {
1044            return Err(self.corrupt(id, format!("expected a commit, found a {kind}")));
1045        }
1046        // `tree`, then `parent`*, then `author`, then `committer`. Everything
1047        // after that is the encoding, the extra headers and the message, and
1048        // none of it is ours to decode.
1049        //
1050        // A commit with an unparsable committer date is not a reason to fail a
1051        // clone; it only costs the walk its ordering for that one commit, and
1052        // `SLOP` is what absorbs exactly that. `commit_facts` answers `0` there
1053        // rather than an error, which is the same thing the
1054        // `gix_object::CommitRefIter` this replaced did by never yielding a
1055        // `Committer` token.
1056        object::commit_facts(buf, self.hash_kind, parents).map_err(|err| self.corrupt(id, err))
1057    }
1058
1059    /// [`Graph::commit_facts_into`], for callers outside this module.
1060    pub(crate) fn commit_into(
1061        &self,
1062        id: &ObjectId,
1063        buf: &mut Vec<u8>,
1064        parents: &mut Vec<ObjectId>,
1065    ) -> Result<i64> {
1066        self.commit_facts_into(id, buf, parents).map(|(_, at)| at)
1067    }
1068
1069    /// Commits reachable from `tips`, in breadth-first order.
1070    ///
1071    /// * `stop`: commits that are neither included nor traversed. The client's
1072    ///   side of the graph.
1073    /// * `boundary`: commits that *are* included but whose parents are not
1074    ///   traversed at all. A client's declared shallow frontier.
1075    /// * `restrict`: when set, only parents inside this set are followed.
1076    ///
1077    /// The last two are not the same thing, and conflating them is a bug worth
1078    /// naming: `deepen-not refs/heads/side` cuts a **merge** off from *one* of
1079    /// its parents, and treating that merge as a boundary drops the other
1080    /// parent's whole line as well. A restriction set says which parents
1081    /// survive; a boundary says none do.
1082    pub(crate) fn commit_closure(
1083        &self,
1084        tips: &[ObjectId],
1085        stop: &HashSet<ObjectId>,
1086        boundary: &HashSet<ObjectId>,
1087        restrict: Option<&HashSet<ObjectId>>,
1088        missing: Missing,
1089    ) -> Result<CommitClosure> {
1090        let mut seen = OidSet::new();
1091        let mut out = CommitClosure::default();
1092        let mut queue: VecDeque<ObjectId> = VecDeque::new();
1093        // Bounded by the number of distinct stopped-at parents, which is at
1094        // most the frontier between what the client has and what it wants —
1095        // not by the repository. A plain `HashSet` on purpose: `OidSet` meters
1096        // the repository-sized sets the rule is about, and metering this one
1097        // would blur the number that matters.
1098        let mut edges: HashSet<ObjectId> = HashSet::new();
1099        // Both live for the whole walk: see `commit_links`.
1100        let mut buf = Vec::new();
1101        let mut parents = Vec::new();
1102
1103        for tip in tips {
1104            if !stop.contains(tip) && seen.insert(*tip) {
1105                queue.push_back(*tip);
1106            }
1107        }
1108
1109        while let Some(id) = queue.pop_front() {
1110            self.charge_walk()?;
1111            if missing == Missing::Skip && !self.contains(&id)? {
1112                continue;
1113            }
1114            let tree = match self.commit_links(&id, &mut buf, &mut parents) {
1115                Ok(tree) => tree,
1116                Err(Error::ObjectMissing { .. }) if missing == Missing::Skip => continue,
1117                Err(err) => return Err(err),
1118            };
1119            // Pushed together, and nowhere else, which is what makes
1120            // `CommitClosure`'s pairing true by construction rather than by
1121            // agreement between two loops.
1122            out.ids.push(id);
1123            out.trees.push(tree);
1124            if boundary.contains(&id) {
1125                continue;
1126            }
1127            for parent in parents.drain(..) {
1128                if stop.contains(&parent) {
1129                    // An **edge**: a commit the client holds, sitting directly
1130                    // behind one it does not. `edges` is deduplicated because
1131                    // a widely-merged parent is stopped at once per child, and
1132                    // step 5 would otherwise walk its tree that many times.
1133                    if edges.insert(parent) {
1134                        out.edges.push(parent);
1135                    }
1136                    continue;
1137                }
1138                if restrict.is_some_and(|allowed| !allowed.contains(&parent)) {
1139                    continue;
1140                }
1141                if seen.insert(parent) {
1142                    queue.push_back(parent);
1143                }
1144            }
1145        }
1146        Ok(out)
1147    }
1148
1149    /// The commits to send, worked out **without** first enumerating the
1150    /// client's whole history.
1151    ///
1152    /// This is git's `limit_list()` (`revision.c`), and it replaces the pair of
1153    /// [`Graph::commit_closure`] calls that [`select_objects`] used to open
1154    /// with. Those two are correct and they are also `O(client history)`: the
1155    /// first walks every commit the client holds purely to build a stop set for
1156    /// the second, so a client one day behind on a hundred-thousand-commit
1157    /// repository costs a hundred thousand commit reads to serve twenty.
1158    ///
1159    /// # How one walk does what two did
1160    ///
1161    /// Both sides go into **one queue ordered by committer date**, newest
1162    /// first, each commit carrying git's `UNINTERESTING` bit. Popping newest
1163    /// first means the two histories are drained together rather than one after
1164    /// the other, and the moment nothing interesting is left in the queue the
1165    /// walk is done: everything still queued is reachable from a `have`, so
1166    /// dropping it cannot drop a commit that belongs in the pack. That
1167    /// termination is [`still_interesting`](Limiter::still_interesting), and
1168    /// [`SLOP`] is the slack it gives the ordering.
1169    ///
1170    /// # Why it cannot send less
1171    ///
1172    /// The walk stops only at a point where `everybody_uninteresting` has just
1173    /// returned true, so every id it abandons is marked `UNINTERESTING`. A
1174    /// commit is marked `UNINTERESTING` only by propagation from a `have`, and
1175    /// a commit reachable from a `have` is one the client demonstrably holds.
1176    /// So an abandoned commit is a held commit, and its ancestors are held too
1177    /// — which is why abandoning them unread is safe. In the other direction
1178    /// skew beyond `SLOP` can leave a held commit *un*marked and send it again:
1179    /// a superset, the same trade [`Exclusion::Edges`] already makes, and the
1180    /// same one git makes.
1181    ///
1182    /// `haves` are taken as the peeled `have` commits, not their closure —
1183    /// computing that closure is the cost this exists to remove.
1184    pub(crate) fn limited_closure(
1185        &self,
1186        tips: &[ObjectId],
1187        haves: &[ObjectId],
1188        boundary: &HashSet<ObjectId>,
1189        have_boundary: &HashSet<ObjectId>,
1190        restrict: Option<&HashSet<ObjectId>>,
1191        missing: Missing,
1192    ) -> Result<LimitedClosure> {
1193        let mut walk = Limiter::default();
1194        let mut buf = Vec::new();
1195
1196        // The `have`s first, so that a commit which is both a `want` and a
1197        // `have` is already uninteresting when the want reaches it — the same
1198        // answer the two-pass walk gives by testing every tip against its stop
1199        // set before queueing it.
1200        for id in haves {
1201            walk.admit(self, *id, true, Missing::Skip, &mut buf)?;
1202        }
1203        for id in tips {
1204            walk.admit(self, *id, false, missing, &mut buf)?;
1205        }
1206
1207        let mut out = LimitedClosure::default();
1208        let mut slop = SLOP;
1209        // git's `date`, starting at `TIME_MAX`: the committer date of the last
1210        // commit this walk kept.
1211        let mut date = i64::MAX;
1212        let mut cache: Option<ObjectId> = None;
1213
1214        while let Some(Pending { id, .. }) = walk.queue.pop() {
1215            if cache == Some(id) {
1216                cache = None;
1217            }
1218            self.charge_walk()?;
1219            let uninteresting = walk.is_uninteresting(&id);
1220            let facts = walk
1221                .parsed
1222                .get(&id)
1223                .expect("only a parsed commit is queued");
1224            let (tree, commit_date) = (facts.tree, facts.committed_at);
1225            // Cloned because `admit` takes `&mut walk`. Bounded by one commit's
1226            // parent count, not by the repository.
1227            let parents = facts.parents.clone();
1228
1229            if uninteresting {
1230                out.uninteresting.push(id);
1231                // A shallow client's graft: it holds this commit and says so,
1232                // but not what is behind it. Walking past it here counts the
1233                // whole repository as already-delivered and packs nothing.
1234                if !have_boundary.contains(&id) {
1235                    for parent in &parents {
1236                        walk.admit(self, *parent, true, Missing::Skip, &mut buf)?;
1237                        if have_boundary.contains(parent) {
1238                            continue;
1239                        }
1240                        // git's `process_parents`: having just marked the
1241                        // parent, push the mark on down through whatever this
1242                        // walk has already read. It reads nothing itself.
1243                        walk.mark_parents_uninteresting(parent, have_boundary);
1244                    }
1245                }
1246                slop = walk.still_interesting(date, slop, &mut cache);
1247                if slop > 0 {
1248                    continue;
1249                }
1250                break;
1251            }
1252
1253            if !boundary.contains(&id) {
1254                for parent in &parents {
1255                    if restrict.is_some_and(|allowed| !allowed.contains(parent)) {
1256                        continue;
1257                    }
1258                    walk.admit(self, *parent, false, missing, &mut buf)?;
1259                }
1260            }
1261            date = commit_date;
1262            out.ids.push(id);
1263            out.trees.push(tree);
1264        }
1265
1266        // git's `mark_edges_uninteresting` (`list-objects.c`): the commits the
1267        // client holds that sit directly behind one it is being sent.
1268        //
1269        // Read off the finished walk rather than recorded inside it, because a
1270        // parent is not always *known* to be uninteresting at the moment its
1271        // child is popped — the mark can arrive later, from another branch of
1272        // the same walk — and an edge set collected too early would miss
1273        // exactly the merge parents whose trees carry the shared content. It
1274        // costs no reads: every parent list is already in hand.
1275        let mut edges: HashSet<ObjectId> = HashSet::new();
1276        for id in &out.ids {
1277            if boundary.contains(id) {
1278                continue;
1279            }
1280            let Some(facts) = walk.parsed.get(id) else {
1281                continue;
1282            };
1283            for parent in &facts.parents {
1284                if !walk.is_uninteresting(parent) {
1285                    continue;
1286                }
1287                if edges.insert(*parent) {
1288                    out.edges.push(*parent);
1289                }
1290            }
1291        }
1292        Ok(out)
1293    }
1294
1295    /// Add everything under `root` to `out`, skipping anything already in
1296    /// `seen` or `exclude`, and honouring `filter`.
1297    ///
1298    /// `depth` starts at 1 for a commit's own root tree, which is how git
1299    /// numbers `tree:<depth>`: `tree:0` yields no trees at all and `tree:1`
1300    /// yields the root tree only.
1301    ///
1302    /// `max_blob_bytes` is the fetch path's object ceiling
1303    /// ([`crate::Limits::object_bytes`]) and is `None` on every walk that is not
1304    /// about to *send* what it finds. Only blobs are checked, and that is not a
1305    /// shortcut: a commit and a tree are bounded by their own grammar, a blob is
1306    /// bounded by nothing, and the walk already knows an entry is a blob from
1307    /// its tree entry's mode without reading anything.
1308    #[allow(clippy::too_many_arguments)]
1309    pub(crate) fn tree_closure<S: Seen + ?Sized>(
1310        &self,
1311        root: ObjectId,
1312        seen: &mut S,
1313        exclude: &HashSet<ObjectId>,
1314        filter: Option<&Filter>,
1315        missing: Missing,
1316        max_blob_bytes: Option<u64>,
1317        out: &mut Emit<'_>,
1318    ) -> Result<()> {
1319        let mut stack = vec![(root, 1usize)];
1320        let mut buf = Vec::new();
1321        // Hoisted out of the loop for the same reason `commit_links` takes its
1322        // buffers from the caller: these are two mallocs and two frees **per
1323        // tree in the repository**, and the allocator was 14.2% of
1324        // `gunnar.walk_1m`'s profile on 2026-08-03. They are cleared, not
1325        // reallocated, on every entry.
1326        let mut children: Vec<(ObjectId, usize)> = Vec::new();
1327        let mut blobs: Vec<ObjectId> = Vec::new();
1328        while let Some((id, depth)) = stack.pop() {
1329            self.charge_walk()?;
1330            if exclude.contains(&id) || !seen.insert(&id) {
1331                continue;
1332            }
1333            if missing == Missing::Skip && !self.contains(&id)? {
1334                continue;
1335            }
1336            let keep_tree = filter.is_none_or(|f| f.keeps_tree(depth));
1337            let kind = match self.load(&id, "reading a tree", &mut buf) {
1338                Ok(kind) => kind,
1339                Err(Error::ObjectMissing { .. }) if missing == Missing::Skip => continue,
1340                Err(err) => return Err(err),
1341            };
1342            if kind != object::Kind::Tree {
1343                return Err(self.corrupt(&id, format!("expected a tree, found a {kind}")));
1344            }
1345            if keep_tree {
1346                out.push(id);
1347            }
1348            let entries = object::TreeIter::from_bytes(&buf, self.hash_kind);
1349            children.clear();
1350            blobs.clear();
1351            for entry in entries {
1352                let entry = entry.map_err(|err| self.corrupt(&id, err))?;
1353                match entry.kind {
1354                    object::EntryKind::Tree => children.push((entry.oid, depth + 1)),
1355                    // A gitlink names a commit in *another* repository. It is
1356                    // not ours to have, to send, or to demand on a push.
1357                    object::EntryKind::Commit => {}
1358                    _ => blobs.push(entry.oid),
1359                }
1360            }
1361            for blob in blobs.drain(..) {
1362                // Inside the blob loop and not only at the top of the tree
1363                // loop: one tree's entries are bounded by nothing gunnar chose,
1364                // so charging per tree lets a single wide tree overshoot the
1365                // ceiling by its whole width. It costs a `Cell` read against a
1366                // header read.
1367                self.charge_walk()?;
1368                if exclude.contains(&blob) || !seen.insert(&blob) {
1369                    continue;
1370                }
1371                if !self.keeps_blob(&blob, filter, missing, max_blob_bytes)? {
1372                    continue;
1373                }
1374                out.push(blob);
1375            }
1376            stack.append(&mut children);
1377        }
1378        Ok(())
1379    }
1380
1381    /// Does this blob go in the pack — and is gunnar willing to hold it?
1382    ///
1383    /// Two questions, **one header read**, because they are asked about the
1384    /// same object at the same moment and reading its header twice would be
1385    /// the second copy one implementation is about. A `filter` that already
1386    /// declines every blob short-circuits before the read: an object that is
1387    /// not going in the pack is not one gunnar will materialise, so its size
1388    /// is nobody's problem.
1389    ///
1390    /// # The ceiling costs a `header` where a `contains` used to do
1391    ///
1392    /// With `max_blob_bytes` set this asks the store for a header rather than
1393    /// for existence. It is a **replacement**, not an addition — the same index
1394    /// probe, plus the entry-header decode gix needs to answer the size — and it
1395    /// buys the only thing that makes the ceiling a guard rather than a report:
1396    /// the size is known *before* any content is read. A ceiling checked after
1397    /// `Find::try_find` has filled a buffer cannot stop the allocation it exists
1398    /// to stop.
1399    fn keeps_blob(
1400        &self,
1401        id: &ObjectId,
1402        filter: Option<&Filter>,
1403        missing: Missing,
1404        max_blob_bytes: Option<u64>,
1405    ) -> Result<bool> {
1406        // What the filter decides without looking at the object at all.
1407        let filter_limit = match filter {
1408            None => None,
1409            Some(filter) => match filter.blob_limit() {
1410                Some(limit) => Some(limit),
1411                None if filter.keeps_any_blob() => None,
1412                None => return Ok(false),
1413            },
1414        };
1415
1416        if filter_limit.is_none() && max_blob_bytes.is_none() {
1417            if missing == Missing::Fail && !self.contains(id)? {
1418                return Err(Error::missing(*id, "packing a blob"));
1419            }
1420            return Ok(true);
1421        }
1422
1423        let Some((_kind, size)) = self.store.header(id.as_bytes()).map_err(backend)? else {
1424            return match missing {
1425                Missing::Skip => Ok(false),
1426                Missing::Fail => Err(Error::missing(*id, "sizing a blob")),
1427            };
1428        };
1429        if let Some(limit) = max_blob_bytes {
1430            if size > limit {
1431                return Err(Error::ObjectTooLarge {
1432                    oid: *id,
1433                    size,
1434                    limit,
1435                });
1436            }
1437        }
1438        // `<`, not `<=`, because that is what git does and a partial clone is
1439        // judged against git.
1440        //
1441        // `list-objects-filter.c`'s `filter_blobs_limit` keeps a blob when
1442        // `object_length < filter_data->max_bytes`, so `blob:limit=1024` omits a
1443        // blob of **exactly** 1024 bytes. gunnar kept it, and the disagreement
1444        // is observable with two commands.
1445        //
1446        // MEASURED 2026-08-05 — one repository, three blobs of 1023, 1024
1447        // and 1025 bytes, `--filter=blob:limit=1024`, the same stock git 2.53.0
1448        // as the client on both arms:
1449        //
1450        // | server | blobs delivered |
1451        // |---|---|
1452        // | stock `git-upload-pack` | 1023 |
1453        // | gunnar, before | 1023, **1024** |
1454        // | gunnar, after | 1023 |
1455        //
1456        // One blob, so it looks small; what it costs is the invariant. A
1457        // partial-clone client sizes its lazy-fetch expectations from git's
1458        // rule, and `tests/real_git_surface.rs` had written gunnar's `<=` down
1459        // as the expected answer, so the suite agreed with the defect instead
1460        // of catching it.
1461        Ok(filter_limit.is_none_or(|limit| size < limit))
1462    }
1463}
1464
1465/// What [`Graph::peel`] found at the end of a tag chain.
1466#[derive(Debug, Clone)]
1467pub(crate) struct Peeled {
1468    /// The first non-tag object.
1469    pub id: ObjectId,
1470    /// Its kind.
1471    pub kind: object::Kind,
1472    /// Every tag object walked through, outermost first.
1473    pub tags: Vec<ObjectId>,
1474}
1475
1476/// The parts of a commit gunnar's traversal needs.
1477#[derive(Debug, Clone)]
1478pub(crate) struct CommitFacts {
1479    pub tree: ObjectId,
1480    pub parents: Vec<ObjectId>,
1481    pub committed_at: i64,
1482}
1483
1484/// What [`Graph::commit_closure`] found: the commits, and each one's root tree.
1485///
1486/// The trees ride along because **every** caller of `commit_closure` walks each
1487/// commit's tree the moment it gets the list back, and asking the store for the
1488/// tree oid again means reading and decoding that commit a *second* time. On
1489/// the `fs` backend that second read is a pack-index probe, a zlib inflate and
1490/// a delta chain — the odb handles are built with `refresh_never()` and no
1491/// object cache, so nothing memoises it. `commit_links` already has the oid in
1492/// hand; dropping it and re-reading the object for it is the twin one
1493/// implementation is about, one field wide.
1494///
1495/// The price is one commit-sized `Vec<ObjectId>` beside the one that was
1496/// already being returned — 20 bytes per commit at sha1, 32 at sha256. It is
1497/// **not** metered by [`meter`], which watches the repository-sized *object*
1498/// sets the one-set rule is about, and it is not one of those: it is bounded by the
1499/// commit count, not the object count.
1500#[derive(Debug, Default)]
1501pub(crate) struct CommitClosure {
1502    /// The commits, in breadth-first order.
1503    pub ids: Vec<ObjectId>,
1504    /// `trees[i]` is `ids[i]`'s root tree. Same length, same order, and true by
1505    /// construction: the one loop that pushes either pushes both.
1506    pub trees: Vec<ObjectId>,
1507    /// Commits in `stop` that this walk ran straight into: a parent of an
1508    /// included commit that the client already holds.
1509    ///
1510    /// This is git's `mark_edges_uninteresting` (`list-objects.c`) edge set,
1511    /// and it falls out of a walk that was happening anyway — the `stop` test
1512    /// is already there, this only records which id failed it. See
1513    /// [`Exclusion::Edges`] for what it buys and what it costs.
1514    ///
1515    /// Empty unless the walk was given a non-empty `stop`, so a clone (no
1516    /// `have`s) never allocates.
1517    pub edges: Vec<ObjectId>,
1518}
1519
1520/// What [`Graph::limited_closure`] found.
1521///
1522/// [`Self::ids`] and [`Self::trees`] are [`CommitClosure`]'s, with the same
1523/// pairing property and for the same reason; [`Self::edges`] is the same edge
1524/// set. The one field that is new is [`Self::uninteresting`], and what it is
1525/// *not* is the point: the commits the walk actually looked at on the client's
1526/// side, which is a frontier, where the two-pass walk's equivalent was the
1527/// client's entire history.
1528#[derive(Debug, Default)]
1529pub(crate) struct LimitedClosure {
1530    /// The commits to send, newest committer date first.
1531    pub ids: Vec<ObjectId>,
1532    /// `trees[i]` is `ids[i]`'s root tree.
1533    pub trees: Vec<ObjectId>,
1534    /// Client-held commits sitting directly behind a commit being sent.
1535    pub edges: Vec<ObjectId>,
1536    /// Client-held commits this walk popped. A frontier, not a history.
1537    pub uninteresting: Vec<ObjectId>,
1538}
1539
1540/// git's `SLOP` (`revision.c`): how many further commits the limiting walk
1541/// looks at once everything left in its queue is uninteresting.
1542///
1543/// Five, git's number, for git's reason. Committer dates are wall-clock
1544/// readings taken on other people's machines, so a queue ordered by them can
1545/// hand back a commit that is *older* than its own child. This is the slack
1546/// that ordering gets before the walk calls the frontier closed. Beyond it the
1547/// walk sends objects the client already had, which is slow; it can never send
1548/// fewer, which is what would be broken. See [`Graph::limited_closure`].
1549const SLOP: usize = 5;
1550
1551/// The bits git keeps on `struct object` while it limits a walk.
1552///
1553/// git's `ADDED` is deliberately absent: it guards against processing one
1554/// commit's parents twice, and here [`flag::SEEN`] already admits an id to the
1555/// queue at most once, so it could never fire.
1556mod flag {
1557    /// Queued, and therefore already read.
1558    pub const SEEN: u8 = 1 << 0;
1559    /// Reachable from something the client already has.
1560    pub const UNINTERESTING: u8 = 1 << 1;
1561    /// Referenced, absent, and tolerated. git re-reads such a commit and leans
1562    /// on `parse_commit_gently`'s per-object memo; gunnar's store has none, so
1563    /// the "we looked, it is not here" answer is remembered here instead.
1564    pub const ABSENT: u8 = 1 << 2;
1565}
1566
1567/// One entry of the limiting walk's date-ordered queue.
1568///
1569/// `Ord` is the committer date first, so [`BinaryHeap::pop`] hands back the
1570/// newest commit — the same end git's `commit_list_insert_by_date` list is read
1571/// from. The insertion sequence breaks ties, reversed so the *earlier*
1572/// insertion wins, because git's list puts an equal date after the ones already
1573/// there and a heap would otherwise settle ties by whatever the object ids
1574/// happen to compare as. That would make the order commits are emitted in, and
1575/// therefore the order of objects in the pack, depend on the hash of their
1576/// contents.
1577#[derive(Debug, PartialEq, Eq)]
1578struct Pending {
1579    date: i64,
1580    seq: std::cmp::Reverse<u64>,
1581    id: ObjectId,
1582}
1583
1584impl Ord for Pending {
1585    fn cmp(&self, other: &Self) -> std::cmp::Ordering {
1586        // `seq` is unique per entry, so this is a total order and never reports
1587        // `Equal` for two different commits.
1588        self.date
1589            .cmp(&other.date)
1590            .then_with(|| self.seq.cmp(&other.seq))
1591    }
1592}
1593
1594impl PartialOrd for Pending {
1595    fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> {
1596        Some(self.cmp(other))
1597    }
1598}
1599
1600/// The state of one limiting walk: the fields git keeps on `struct commit` and
1601/// `struct object`, which gunnar has nowhere to hang.
1602///
1603/// All three maps are bounded by the commits the walk **reads**, which is the
1604/// frontier between the two histories rather than either of them. They are
1605/// plain [`HashMap`]s rather than [`OidSet`]s for the reason `commit_closure`'s
1606/// `edges` set is: [`meter`] watches the repository-sized sets the one-set rule is
1607/// about, and metering a frontier-sized one would blur the number that matters.
1608#[derive(Default)]
1609struct Limiter {
1610    flags: HashMap<ObjectId, u8>,
1611    parsed: HashMap<ObjectId, CommitFacts>,
1612    queue: BinaryHeap<Pending>,
1613    seq: u64,
1614}
1615
1616impl Limiter {
1617    /// git's `repo_parse_commit_gently` followed by
1618    /// `commit_list_insert_by_date`: mark `id`, read it if this is the first
1619    /// time it has been named, and queue it.
1620    ///
1621    /// The mark is applied **before** the read and before the already-seen
1622    /// test, which is what lets a commit already queued as interesting be
1623    /// turned uninteresting by a later arrival from the client's side.
1624    fn admit(
1625        &mut self,
1626        graph: &Graph<'_>,
1627        id: ObjectId,
1628        uninteresting: bool,
1629        missing: Missing,
1630        buf: &mut Vec<u8>,
1631    ) -> Result<()> {
1632        {
1633            let flags = self.flags.entry(id).or_insert(0);
1634            if uninteresting {
1635                *flags |= flag::UNINTERESTING;
1636            }
1637            if *flags & (flag::SEEN | flag::ABSENT) != 0 {
1638                return Ok(());
1639            }
1640        }
1641        let facts = match graph.commit_facts(&id, buf) {
1642            Ok(facts) => facts,
1643            Err(Error::ObjectMissing { .. }) if missing == Missing::Skip => {
1644                *self.flags.entry(id).or_insert(0) |= flag::ABSENT;
1645                return Ok(());
1646            }
1647            Err(err) => return Err(err),
1648        };
1649        *self.flags.entry(id).or_insert(0) |= flag::SEEN;
1650        self.queue.push(Pending {
1651            date: facts.committed_at,
1652            seq: std::cmp::Reverse(self.seq),
1653            id,
1654        });
1655        self.seq += 1;
1656        self.parsed.insert(id, facts);
1657        Ok(())
1658    }
1659
1660    /// git's `mark_parents_uninteresting` (`revision.c`).
1661    ///
1662    /// **It reads nothing.** It descends only through commits this walk has
1663    /// already parsed and stops at the first one it has not, which is what
1664    /// keeps it bounded by the frontier rather than by the history behind it —
1665    /// and is why the walk it belongs to can be cheap at all. It exists for the
1666    /// case where a commit was queued as interesting, so its parents are in
1667    /// hand and marked interesting too, and only afterwards turns out to be
1668    /// reachable from a `have`.
1669    fn mark_parents_uninteresting(&mut self, start: &ObjectId, have_boundary: &HashSet<ObjectId>) {
1670        let Some(facts) = self.parsed.get(start) else {
1671            return;
1672        };
1673        let mut stack: Vec<ObjectId> = facts.parents.clone();
1674        while let Some(mut id) = stack.pop() {
1675            loop {
1676                let flags = self.flags.entry(id).or_insert(0);
1677                if *flags & flag::UNINTERESTING != 0 {
1678                    break;
1679                }
1680                *flags |= flag::UNINTERESTING;
1681                if have_boundary.contains(&id) {
1682                    break;
1683                }
1684                let Some(facts) = self.parsed.get(&id) else {
1685                    break;
1686                };
1687                let Some((first, rest)) = facts.parents.split_first() else {
1688                    break;
1689                };
1690                stack.extend_from_slice(rest);
1691                id = *first;
1692            }
1693        }
1694    }
1695
1696    fn is_uninteresting(&self, id: &ObjectId) -> bool {
1697        self.flags
1698            .get(id)
1699            .is_some_and(|flags| flags & flag::UNINTERESTING != 0)
1700    }
1701
1702    /// git's `everybody_uninteresting`, with git's `interesting_cache` memo.
1703    ///
1704    /// The scan is over the whole queue. Remembering the one interesting entry
1705    /// it found makes every later call O(1) until that entry is popped, which
1706    /// matters because the caller runs on every uninteresting commit.
1707    fn everybody_uninteresting(&self, cache: &mut Option<ObjectId>) -> bool {
1708        if let Some(id) = *cache {
1709            if !self.is_uninteresting(&id) {
1710                return false;
1711            }
1712            *cache = None;
1713        }
1714        for pending in &self.queue {
1715            if self.is_uninteresting(&pending.id) {
1716                continue;
1717            }
1718            *cache = Some(pending.id);
1719            return false;
1720        }
1721        true
1722    }
1723
1724    /// git's `still_interesting`. The slop left; zero means stop.
1725    ///
1726    /// `date` is the committer date of the last commit the walk kept.
1727    fn still_interesting(&self, date: i64, slop: usize, cache: &mut Option<ObjectId>) -> usize {
1728        let Some(head) = self.queue.peek() else {
1729            return 0;
1730        };
1731        // Something queued is at least as new as the last commit kept, so the
1732        // frontier is not behind the walk yet.
1733        if date <= head.date {
1734            return SLOP;
1735        }
1736        if !self.everybody_uninteresting(cache) {
1737            return SLOP;
1738        }
1739        slop.saturating_sub(1)
1740    }
1741}
1742
1743/// The result of choosing what goes in a pack.
1744#[derive(Debug, Default)]
1745pub(crate) struct Selection {
1746    /// Every object to send, tags first, then commits, then trees and blobs.
1747    pub objects: Vec<ObjectId>,
1748    /// The commits included, in traversal order.
1749    pub commits: Vec<ObjectId>,
1750    /// Every object this walk proved the **client already holds**. Nothing
1751    /// else.
1752    ///
1753    /// This is step 6's `seen` set with the objects this transfer carries taken
1754    /// back out again — see [`Carried`]. the rule that there is one
1755    /// repository-sized set and no second copy still holds: the set is moved
1756    /// out rather than cloned, and the subtraction shrinks it.
1757    ///
1758    /// # The paragraph that used to be here, and the object that refuted it
1759    ///
1760    /// It read: a base `write_pack_copy_thin` cannot re-point is *either*
1761    /// outside the transfer — and then the client holds it — *or* inside the
1762    /// transfer in a different source pack, "and then naming it is an
1763    /// `OBJ_REF_DELTA` against an object in the same pack, which
1764    /// `tests/ofs_delta_off.rs` already holds stock git to. Both are sound."
1765    ///
1766    /// Stock git is sound on it. **gitoxide is not**, and gitoxide is the
1767    /// receiver `gunnar-client` resolves a fetch with — and every other
1768    /// gitoxide-based client, and gunnar's own `receive-pack`.
1769    /// `gix_pack::data::input::LookupRefDeltaObjectsIter` reads every
1770    /// `OBJ_REF_DELTA` in a thin pack as naming an object the *receiver already
1771    /// has*: it looks in the local object database and in the bases it has
1772    /// already spliced in, and **nowhere else** — never inside the pack it is
1773    /// reading. A base that is in the transfer and has not been spliced in is
1774    /// `input::Error::NotFound`, and the fetch dies with
1775    /// `The object <base> could not be decoded or wasn't found`.
1776    ///
1777    /// Reproduced on `gunnar.year`, 2026-08-05, and then offline from the
1778    /// run's own directories: tree
1779    /// `1d062a8c1d97fd2d04222bb9a0d5c309fde57796` went into the pack as a delta
1780    /// on a tree the client held; the root tree of the *next* commit went in as
1781    /// a delta on `1d062a8c`; gitoxide refused the whole pack while
1782    /// `git index-pack --strict --fix-thin` completed the same bytes. The
1783    /// server holds `1d062a8c` **twice** — whole in the pack whose push
1784    /// introduced its dependant, and as a delta in the pack whose push
1785    /// introduced it — so `gix_pack`'s copier cannot re-point the second delta
1786    /// and names its base by object id. That double storage is what
1787    /// `index-pack --fix-thin` leaves behind on any repository pushed to more
1788    /// than once and not `gc`'d, which is exactly the repository with no
1789    /// `.bitmap`: this path and no other.
1790    ///
1791    /// So the rule is the narrow one. **A thin delta may name a base only if
1792    /// the receiver already holds it.** An object this transfer is carrying is
1793    /// not held until the pack has been resolved, and the receiver resolves
1794    /// thin bases *before* it resolves the pack.
1795    ///
1796    /// # What it is **not**
1797    ///
1798    /// It is not a promise about a shallow client. A `have` from one names a
1799    /// commit whose ancestry that client may not hold, and steps 2 and 5
1800    /// deliberately tolerate those gaps rather than fail the fetch. The caller
1801    /// is what must not offer this as a voucher in that case; see
1802    /// `upload_pack::send_pack`.
1803    ///
1804    /// # Two spellings, one question
1805    ///
1806    /// The walk hands over its exclusion closure. The bitmap path has no such
1807    /// set and must not build one — enumerating the client's history as object
1808    /// ids is the `O(client history)` cost the bitmap exists to remove — so it
1809    /// hands over the `have` bitmap and the positions that name it. Both answer
1810    /// [`ClientHas::contains`] in constant time, which is the only thing the
1811    /// one consumer asks, and since 2026-08-05 both answer the **same**
1812    /// question: does the receiver hold this already.
1813    pub client_has: ClientHas,
1814    /// Whether a `.bitmap` answered this, rather than the walk below it.
1815    ///
1816    /// # Why this is a field and not a log line
1817    ///
1818    /// Because a guard needs an instrument. The bitmap path declines request
1819    /// shapes it cannot answer exactly, and the list is long enough that
1820    /// getting one entry wrong turns the whole feature off **silently**: every
1821    /// test still passes, because the walk still answers and the walk is
1822    /// correct. That is exactly what happened on 2026-08-04 — `max_blob_bytes`
1823    /// is `Some` by default (`Limits::object_bytes`), the bitmap path declined
1824    /// on it, and four end-to-end cases against stock git went green while
1825    /// measuring the walk. A deliberately broken reader was the only thing that
1826    /// showed it, and only because it left the unit tests red and the
1827    /// end-to-end ones green.
1828    pub answered_from_bitmap: bool,
1829    /// How many commits the bitmap path had to **open**, because they had no
1830    /// stored bitmap of their own.
1831    ///
1832    /// The second thing about this path no other guard can see. ORing a stored
1833    /// bitmap in and then walking its parents anyway is *semantically
1834    /// identical* — a bitmap is closed under reachability, so the extra work
1835    /// changes no answer — and it turns a bounded walk into a full one. Proven
1836    /// 2026-08-04: with the termination deliberately left out, every one of the
1837    /// 9 set-equality tests and all 25 end-to-end cases against stock git
1838    /// stayed green. Nothing but a count can fail on it.
1839    ///
1840    /// Zero for a fetch whose every want already has a bitmap, which is every
1841    /// fetch against a freshly repacked repository.
1842    pub bitmap_commits_walked: usize,
1843}
1844
1845/// What the receiver is known **already to hold**, in whichever shape the
1846/// selection produced.
1847///
1848/// One question — *may a thin delta name this base?* — asked of the exclusion
1849/// closure the selection produced. It used to have a second arm carrying bits
1850/// over a git `.bitmap`'s positions; that path is gone with the format that
1851/// needed it, and reachability is answered by `GitServe::select` in object
1852/// ids. The answer is `true` only
1853/// for an object the receiver had **before** this transfer: see
1854/// [`Selection::client_has`] for the object that proved "and the ones this
1855/// transfer carries" is a different, wrong answer.
1856#[derive(Debug)]
1857pub(crate) enum ClientHas {
1858    /// The walk's exclusion closure, moved out rather than dropped.
1859    Ids(OidSet),
1860}
1861
1862impl Default for ClientHas {
1863    fn default() -> Self {
1864        ClientHas::Ids(OidSet::new())
1865    }
1866}
1867
1868impl ClientHas {
1869    /// **`contains` is gone, and it is not an oversight.**
1870    ///
1871    /// It answered *"may a thin delta name `id` as its base?"* on this side of
1872    /// the seam, for a pack writer that lived here. The pack writer is the
1873    /// engine's now, so the question is asked where the answer is used:
1874    /// `GitServe::emit_pack` takes the negotiated common **tips** and derives
1875    /// the voucher from its own reachability. Keeping a second, caller-side
1876    /// answer would be two authorities on which bases are safe, and the unsafe
1877    /// direction is a *failed* fetch rather than a larger one.
1878    ///
1879    /// **As of 2026-08-10 the znippy arm does not read `have` at all** — it
1880    /// closes every delta base into the pack rather than offering any as
1881    /// external — so the voucher is currently derived by nobody and costs bytes
1882    /// rather than correctness. That is recorded in that engine's source as an
1883    /// honest unused parameter, and it does not change the argument above:
1884    /// when the narrowing pass is written, `have` is where it reads from, and
1885    /// it must stay one authority rather than two.
1886    ///
1887    /// What survives is [`ClientHas::len`], because the server logs how much
1888    /// this transfer was allowed to point a delta at, and a regression in
1889    /// either selection path shows up as that number moving while
1890    /// `recompressed` climbs beside it.
1891    ///
1892    /// How many objects this vouches for.
1893    pub(crate) fn len(&self) -> usize {
1894        match self {
1895            ClientHas::Ids(ids) => ids.len(),
1896        }
1897    }
1898}
1899
1900/// Which of the client's commits step 5 opens up to find objects it already
1901/// holds.
1902///
1903/// # The measurement this exists for
1904///
1905/// Taken 2026-08-04, one fetch of 20 commits out of 1500 against the
1906/// `scale-deep-1500c-256k-20d` fixture (a 377.2 MiB single pack, one
1907/// incompressible 256 KiB blob per commit), load average 7.70:
1908///
1909/// ```text
1910///                     commit reads   tree reads   pack faulted in
1911///   Full                    1 504        3 000     377.23 MiB  (99.96%)
1912///   stock git 2.53.0            —            —      10.17 MiB  ( 2.69%)
1913/// ```
1914///
1915/// The 0.78 MiB those 4 504 reads actually inflate is not the cost. `git
1916/// repack` writes a pack in *commit* order — each commit, then its trees, then
1917/// its blobs — so the 3 000 trees are spread evenly across all 377 MiB, 0.2
1918/// MiB of contiguous commits at the front and everything else interleaved with
1919/// a quarter-megabyte blob. Every one of those touches is an mmap fault, and
1920/// the desktop's `md1` carries `read_ahead_kb = 2048`, so each fault pulls in 2 MiB.
1921/// 3 000 touches over 189 two-megabyte windows hits every window: the whole
1922/// pack. The amplification is **readahead over a scattered walk**, not
1923/// decompression.
1924///
1925/// [`Exclusion::Full`] does what `git rev-list --objects --not` does with
1926/// `--objects-edge-aggressive`; [`Exclusion::Edges`] does what `pack-objects`
1927/// does by default.
1928#[derive(Debug, Default, Clone, Copy, PartialEq, Eq)]
1929pub(crate) enum Exclusion {
1930    /// Walk the trees of **every** commit the client holds.
1931    ///
1932    /// Exact: an object the client has is never sent twice. Also
1933    /// `O(client history)` tree reads, which is the row above.
1934    ///
1935    /// Kept, and reachable, because it is the red switch this optimisation is
1936    /// checked against: `GUNNAR_EXCLUSION=full` restores it, and a test that
1937    /// cannot restore the old behaviour cannot show that the new one changed
1938    /// anything.
1939    Full,
1940    /// Walk the trees only of the client's own tips and of the **edge** — the
1941    /// commits it holds that sit directly behind a commit it is being sent.
1942    ///
1943    /// This is `mark_edges_uninteresting()` in git's `list-objects.c`, and it
1944    /// is what `git upload-pack` uses for every fetch unless
1945    /// `--objects-edge-aggressive` is asked for.
1946    ///
1947    /// **It is a superset, not an equality, and that is the trade.** An object
1948    /// reachable only from deep in the client's history and *also* reachable
1949    /// from a commit being sent is no longer recognised as already-held, so it
1950    /// goes in the pack again. Sending an object the client has is slow;
1951    /// sending one it lacks is a broken fetch, and nothing here can do that —
1952    /// the *commit* stop set is still the client's full closure, so no commit,
1953    /// tree or blob that belongs in the pack is ever dropped. Git made the same
1954    /// call for the same reason.
1955    ///
1956    /// On the fixture above the superset is empty: 76 objects either way.
1957    #[default]
1958    Edges,
1959}
1960
1961impl Exclusion {
1962    /// The serving default, with `GUNNAR_EXCLUSION=full` as the way back.
1963    ///
1964    /// Read every time rather than cached in a `OnceLock`, because a switch
1965    /// that only takes effect if it was set before the first fetch is a switch
1966    /// that reads green on a warm process and red on a cold one. It is one
1967    /// `getenv` per fetch against a walk that reads thousands of objects.
1968    ///
1969    /// Anything other than `full` is `Edges`, including a typo. That is
1970    /// deliberate: this is a rollback lever, and the failure mode of an
1971    /// unrecognised value must be *the shipping behaviour*, not a third one.
1972    pub(crate) fn from_env() -> Self {
1973        match std::env::var("GUNNAR_EXCLUSION").as_deref() {
1974            Ok("full") => Exclusion::Full,
1975            _ => Exclusion::Edges,
1976        }
1977    }
1978}
1979
1980/// How [`select_objects`] works out which commits to send.
1981///
1982/// # The measurement this exists for
1983///
1984/// `gunnar.fetch_readside` , 2026-08-04, one fetch of 20 commits out of
1985/// 1500 on `scale-deep-1500c-256k-20d`. [`Exclusion::Edges`] had just taken the
1986/// tree reads from 3 000 to 40, and this is what was left:
1987///
1988/// ```text
1989///   store reads   1 546
1990///     of which    1 506  commits — step 3, the client's whole history
1991///                    40  trees   — the frontier
1992/// ```
1993///
1994/// 97.4% of every read a fetch still made was one walk enumerating history the
1995/// client had already told the server it holds, purely to build a stop set. On
1996/// a fixture with 1500 commits that is 1 506 reads; on a hundred-thousand-commit
1997/// repository it is a hundred thousand, for the same twenty commits of work.
1998#[derive(Debug, Default, Clone, Copy, PartialEq, Eq)]
1999pub(crate) enum Limiting {
2000    /// Two walks: the client's whole commit history, then the commits to send
2001    /// against that history as a stop set.
2002    ///
2003    /// Exact, `O(client history)`, and **required** by [`Exclusion::Full`],
2004    /// which needs the root tree of every commit the client holds and can only
2005    /// get it from a walk that visited every one of them. Also the rollback
2006    /// lever: `GUNNAR_LIMIT=two-pass`.
2007    TwoPass,
2008    /// One walk with both sides in it, ordered by committer date, stopping when
2009    /// the frontier is behind it. git's `limit_list`, and
2010    /// [`Graph::limited_closure`].
2011    #[default]
2012    Merged,
2013}
2014
2015impl Limiting {
2016    /// The serving default, with `GUNNAR_LIMIT=two-pass` as the way back.
2017    ///
2018    /// Read every time, and any unrecognised value is the shipping behaviour,
2019    /// for the two reasons [`Exclusion::from_env`] gives.
2020    pub(crate) fn from_env() -> Self {
2021        match std::env::var("GUNNAR_LIMIT").as_deref() {
2022            Ok("two-pass") => Limiting::TwoPass,
2023            _ => Limiting::Merged,
2024        }
2025    }
2026}
2027
2028/// Whether the serve path may answer from a `.bitmap`.
2029///
2030/// The rollback lever, in the same shape as [`Exclusion`] and [`Limiting`]:
2031/// `GUNNAR_BITMAP=off` puts every request back on the walk, which is what every
2032/// request did before 2026-08-04. It is also how the tests get both answers out
2033/// of one server to compare, so the lever is exercised by the suite rather than
2034/// only by an operator having a bad day.
2035#[derive(Debug, Default, Clone, Copy, PartialEq, Eq)]
2036pub(crate) enum Bitmaps {
2037    /// Read the `.bitmap` when the store has one and the request shape allows.
2038    #[default]
2039    Read,
2040    /// Walk, always.
2041    Off,
2042}
2043
2044impl Bitmaps {
2045    /// The serving default, with `GUNNAR_BITMAP=off` as the way back.
2046    ///
2047    /// Read every time, and any unrecognised value is the shipping behaviour,
2048    /// for the two reasons [`Exclusion::from_env`] gives.
2049    pub(crate) fn from_env() -> Self {
2050        match std::env::var("GUNNAR_BITMAP").as_deref() {
2051            Ok("off") => Bitmaps::Off,
2052            _ => Bitmaps::Read,
2053        }
2054    }
2055}
2056
2057/// Options for [`select_objects`].
2058#[derive(Debug, Default, Clone)]
2059pub(crate) struct SelectOptions<'f> {
2060    /// The client's shallow frontier: included, but not traversed past.
2061    pub boundary: HashSet<ObjectId>,
2062    /// Where the client's *own* history stops, bounding step 3's walk over the
2063    /// `excludes`.
2064    ///
2065    /// Equal to `boundary` for every request that is not a deepening, and
2066    /// deliberately not folded into it for the ones that are: a deepening
2067    /// walks past the client's cut on purpose, while the walk that decides
2068    /// what the client already holds never may. See
2069    /// [`crate::shallow::ShallowOutcome::client_boundary`].
2070    pub have_boundary: HashSet<ObjectId>,
2071    /// A partial-clone filter, if the client sent one.
2072    pub filter: Option<&'f Filter>,
2073    /// When set, the commit walk may only visit commits in this set, which is
2074    /// the answer a deepening request computed.
2075    pub restrict: Option<&'f HashSet<ObjectId>>,
2076    /// Whether a referenced-but-absent object is an error.
2077    pub missing: Missing,
2078    /// How much of the client's history step 5 opens up. See [`Exclusion`].
2079    pub exclusion: Exclusion,
2080    /// How much of the client's history steps 3 and 4 walk. See [`Limiting`].
2081    ///
2082    /// [`Exclusion::Full`] forces [`Limiting::TwoPass`] whatever this says,
2083    /// because the full exclusion needs the root tree of every commit the
2084    /// client holds and only the two-pass walk ever visits them. The two knobs
2085    /// are separate all the same: they name different halves of the same
2086    /// request, and folding them into one would make the exactness lever and
2087    /// the traversal lever impossible to roll back independently.
2088    pub limiting: Limiting,
2089    /// The ceiling on a single blob this walk may select — the fetch path's
2090    /// [`crate::Limits::object_bytes`].
2091    ///
2092    /// **`None` by default, and that is not the exception to "a guard that is
2093    /// off by default is not a guard".** This struct is crate-private and has
2094    /// three callers. Only one of them is about to *materialise* what it
2095    /// selects, and only that one may set the field; the exclusion walk and
2096    /// `WantPolicy`'s reachability closure decide membership and read no
2097    /// content, so a ceiling there would refuse fetches over an object nobody
2098    /// was going to hold. The default that is on lives where the server is
2099    /// configured — [`crate::upload_pack::Config::limits`].
2100    pub max_blob_bytes: Option<u64>,
2101    /// Whether this selection may be answered from a `.bitmap`. See
2102    /// [`Bitmaps`], and [`crate::bitmap::select`] for what it does and for the
2103    /// request shapes it declines.
2104    pub bitmaps: Bitmaps,
2105}
2106
2107impl SelectOptions<'_> {
2108    /// The options a **fetch is actually served with**, before the
2109    /// request-specific fields are filled in.
2110    ///
2111    /// One definition, used by `upload_pack::send_pack` and by the tests (one
2112    /// implementation). Not a convenience: a test that builds these by hand is
2113    /// a test of a configuration the server never runs in, and the ceiling
2114    /// below is the proof — `Limits::object_bytes` is `Some` by default,
2115    /// `..Default::default()` makes it `None`, and every bitmap test written
2116    /// that way was measuring
2117    /// the walk.
2118    pub(crate) fn for_serving(limits: &Limits) -> Self {
2119        SelectOptions {
2120            missing: Missing::Fail,
2121            exclusion: Exclusion::from_env(),
2122            limiting: Limiting::from_env(),
2123            bitmaps: Bitmaps::from_env(),
2124            max_blob_bytes: limits.object_bytes.map(|n| n as u64),
2125            ..Default::default()
2126        }
2127    }
2128
2129    /// The options a **mirror push** is built with.
2130    ///
2131    /// [`Self::for_serving`] with the client-facing blob ceiling left off, and
2132    /// for one reason: a mirror is not a client. `Limits::object_bytes` refuses
2133    /// to *hand out* an object bigger than an operator allows a fetch to pull;
2134    /// applying it here would mean a repository holding one large blob could
2135    /// never obtain a second copy at all, which is the opposite of what the
2136    /// ceiling is for.
2137    ///
2138    /// The three rollback levers — `GUNNAR_EXCLUSION`, `GUNNAR_LIMIT`,
2139    /// `GUNNAR_BITMAP` — are read exactly as the serve path reads them, so an
2140    /// operator putting the server back on the walk puts replication back on it
2141    /// too rather than leaving one of the two paths on a setting nobody chose.
2142    pub(crate) fn for_mirroring() -> Self {
2143        SelectOptions {
2144            missing: Missing::Fail,
2145            exclusion: Exclusion::from_env(),
2146            limiting: Limiting::from_env(),
2147            bitmaps: Bitmaps::from_env(),
2148            max_blob_bytes: None,
2149            ..Default::default()
2150        }
2151    }
2152}
2153
2154/// **The objects one push must carry**: everything reachable from `want` that
2155/// is not reachable from `have`, in this store's ids.
2156///
2157/// # Why replication comes through here rather than calling the store
2158///
2159/// 🔴 MEASURED on the reference appliance 2026-08-31. A resync of 25 repositories left
2160/// six failing, every one of them on an annotated tag, every one refused by the
2161/// standby's own `receive-pack` with
2162/// `missing necessary objects: <oid> is not in this repository (peeling a
2163/// tag)` — where `<oid>` is the commit the tag names and the primary holds.
2164///
2165/// The mirror built its object list from
2166/// [`GitOps::reachable`](gunnar_store::git_storage_trait::GitOps::reachable),
2167/// handing it the ref tips **unpeeled**. For an annotated tag that tip is the
2168/// tag *object*, and the shipped storage engine contributes a non-commit tip's
2169/// own id and stops: the tag went into the pack and the commit under it did
2170/// not. It only showed where the tagged commit was on no branch — `skade`'s
2171/// `v0.9.10`, `edda`'s twenty-five tags on `2c7f9919…` — because otherwise a
2172/// branch tip in the same push carried it by accident.
2173///
2174/// A clone of those same repositories has always worked, and this function is
2175/// why: [`select_objects`] peels its tips (step 1) and puts the tag objects it
2176/// walked through into the answer, and so does the store-answered fast path in
2177/// [`crate::reachable::select`]. Replication asking the store directly was a
2178/// second definition of *"what does this ref tip reach"*, and the two
2179/// definitions disagreed about exactly one edge. There is one now (one implementation).
2180///
2181/// `have` is tolerant by construction: a tip the far end advertised and this
2182/// store does not hold contributes nothing rather than failing, which is what a
2183/// standby whose history diverged looks like. A `want` this store does not hold
2184/// is an error, because that pack cannot be built.
2185///
2186/// # No object budget and no byte ceiling, deliberately
2187///
2188/// [`Graph::with_budget`] and [`Graph::with_walk_limit`] are what an operator
2189/// aims at a *client*: a fetch that would cost more than this machine will
2190/// spend is refused, and the client is told to ask for less. A mirror cannot
2191/// ask for less — there is no smaller request that still leaves the standby
2192/// holding the repository — so a ceiling here would not bound the work, it
2193/// would only make one repository permanently unreplicable. That is what
2194/// `GitOps::reachable` did before this function existed, over the same objects,
2195/// so this changes nothing about what the replicator may consume.
2196pub fn objects_to_send(
2197    store: &dyn GitServe,
2198    hash_kind: Kind,
2199    want: &[&[u8]],
2200    have: &[&[u8]],
2201) -> Result<Vec<Vec<u8>>> {
2202    let tips: Vec<ObjectId> = want.iter().map(|raw| oid(raw)).collect::<Result<_>>()?;
2203    let excludes: Vec<ObjectId> = have.iter().map(|raw| oid(raw)).collect::<Result<_>>()?;
2204    let graph = Graph::new(store, hash_kind);
2205    let selection = select_objects(&graph, &tips, &excludes, &SelectOptions::for_mirroring())?;
2206    Ok(selection
2207        .objects
2208        .into_iter()
2209        .map(|id| id.as_slice().to_vec())
2210        .collect())
2211}
2212
2213/// Everything reachable from `tips` that is not reachable from `excludes`.
2214///
2215/// This is `git rev-list --objects <tips> --not <excludes>`. Three callers use
2216/// it: to build a fetch's pack, to check that a push's new tips close, and,
2217/// with an empty `excludes`, to enumerate a whole repository for a clone.
2218pub(crate) fn select_objects(
2219    graph: &Graph<'_>,
2220    tips: &[ObjectId],
2221    excludes: &[ObjectId],
2222    options: &SelectOptions<'_>,
2223) -> Result<Selection> {
2224    // ── 0. the bitmap, when there is one and it can answer this ─────────────
2225    //
2226    //    Before anything else, because everything after it is the walk this
2227    //    replaces. It declines rather than approximates: a request shape the
2228    //    bitmap cannot answer exactly — a filter, a shallow boundary, a
2229    //    deepening, an object ceiling — falls through to the code below and is
2230    //    served exactly as it was before this existed. See [`crate::bitmap`].
2231    if options.bitmaps == Bitmaps::Read {
2232        let _s = crate::stage::enter("select.bitmap");
2233        // ── the escape hatch moved, and it is still the whole design ────────
2234        //
2235        // This used to be two questions: `ObjectStore::reachable()`, an
2236        // `Option<Arc<dyn ReachIndex>>` answering *"do you have a reachability
2237        // index at all"*, and then `ReachIndex::select`, answering *"can you
2238        // answer THIS request exactly"*. `GitServe` has one method and it is
2239        // the second one — `select` is required, and `Ok(None)` is how a
2240        // backend declines. A backend with no index answers `Ok(None)` always.
2241        //
2242        // **`Ok(None)` must stay reachable.** The Arrow arm returns it when its
2243        // reachability projection is empty, which a clean restart produces
2244        // today; without it a backend whose index does not cover a tip would
2245        // contribute only the tip and the clone would be short by everything
2246        // beneath it — while exiting zero. Losing the option type is how that
2247        // becomes unrepresentable.
2248        if let Some(selection) = crate::reachable::select(graph, tips, excludes, options)? {
2249            // ── the budget applies to the fast path too ────────────────────
2250            //
2251            // It did not, and that made `--fetch-budget` a control that does
2252            // nothing on the shipped engine: the znippy arm answers `select`
2253            // out of its reachability index, so the walk below — the only place
2254            // `charge_walk` runs — never happens, and a budget of **1** served
2255            // a whole clone. `tests/wire_config.rs` saw it as *"a budget of 1
2256            // served a clone; the traversal budget never reached the walk"*.
2257            //
2258            // "Objects visited" is the walk's quantity; the index's is
2259            // "objects selected", and it is the honest analogue — it is what
2260            // the server is about to materialise, which is what the budget's
2261            // own doc says it stands in front of. So the ceiling binds on both
2262            // paths and means the same thing on both: **this fetch is bigger
2263            // than you allowed.**
2264            //
2265            // Deliberately `>` and not `>=`: a budget of N must serve a fetch
2266            // of exactly N objects. The walk's check is `>=` because it fires
2267            // *before* visiting the next object rather than after counting the
2268            // last one; both refuse the same N+1 request.
2269            if let Some(limit) = graph.budget {
2270                if selection.objects.len() > limit {
2271                    return Err(Error::WalkBudget { limit });
2272                }
2273            }
2274            return Ok(selection);
2275        }
2276    }
2277
2278    let mut selection = Selection::default();
2279    let mut emitted = OidSet::new();
2280    let _peel = crate::stage::enter("select.peel");
2281
2282    // 1. Peel the tips. A `want` may name a tag, a tree, or even a blob.
2283    let mut commit_tips = Vec::new();
2284    let mut direct: Vec<(ObjectId, object::Kind)> = Vec::new();
2285    for tip in tips {
2286        let peeled = graph.peel(tip)?;
2287        for tag in peeled.tags {
2288            if emitted.insert(tag) {
2289                selection.objects.push(tag);
2290            }
2291        }
2292        match peeled.kind {
2293            object::Kind::Commit => commit_tips.push(peeled.id),
2294            other => direct.push((peeled.id, other)),
2295        }
2296    }
2297
2298    // 2. The same for the excludes, tolerating gaps: a `have` from a shallow
2299    //    client can name a commit whose ancestry that client does not hold.
2300    let mut exclude_commits = Vec::new();
2301    for id in excludes {
2302        if !graph.contains(id)? {
2303            continue;
2304        }
2305        match graph.peel(id) {
2306            Ok(peeled) if peeled.kind == object::Kind::Commit => exclude_commits.push(peeled.id),
2307            Ok(_) => {}
2308            Err(Error::ObjectMissing { .. }) => {}
2309            Err(err) => return Err(err),
2310        }
2311    }
2312
2313    // 3 & 4. What the client already has, and what it is about to be sent.
2314    //
2315    //    Two questions, and [`Limiting`] decides whether they take two walks or
2316    //    one. Both run before step 5 rather than after it, which is the order
2317    //    this used to be in, so that step 5 can *take* `excluded_commits`
2318    //    instead of copying it. Nothing here reads the object closure, only the
2319    //    commit graph, so the steps commute.
2320    //
2321    //    `Exclusion::Full` needs the root tree of every commit the client
2322    //    holds, and only the two-pass walk ever visits them, so it pins the
2323    //    choice. `excluded_trees` is that list, and it is empty under the
2324    //    merged walk because nothing there wants it.
2325    let limiting = match options.exclusion {
2326        Exclusion::Full => Limiting::TwoPass,
2327        Exclusion::Edges => options.limiting,
2328    };
2329    let mut excluded_trees: Vec<ObjectId> = Vec::new();
2330    crate::stage::end(_peel);
2331    let _walk = crate::stage::enter("select.commit_walk");
2332    let (commits, excluded_commits) = match limiting {
2333        Limiting::TwoPass => {
2334            let held = graph.commit_closure(
2335                &exclude_commits,
2336                &HashSet::new(),
2337                // Bounded by where the client said *its* history stops, not by
2338                // what this request decided to send. A shallow client offers
2339                // its grafted tip as a `have`; walking past that graft here
2340                // counts the whole repository as already-delivered and packs
2341                // nothing.
2342                &options.have_boundary,
2343                // The client's own history is not limited by what this request
2344                // decided to deepen to.
2345                None,
2346                Missing::Skip,
2347            )?;
2348            let excluded: OidSet = held.ids.iter().copied().collect();
2349            excluded_trees = held.trees;
2350            let commits = graph.commit_closure(
2351                &commit_tips,
2352                &excluded,
2353                &options.boundary,
2354                options.restrict,
2355                options.missing,
2356            )?;
2357            (commits, excluded)
2358        }
2359        Limiting::Merged => {
2360            let limited = graph.limited_closure(
2361                &commit_tips,
2362                &exclude_commits,
2363                &options.boundary,
2364                &options.have_boundary,
2365                options.restrict,
2366                options.missing,
2367            )?;
2368            // A frontier, where `TwoPass` puts the client's whole history here.
2369            // It seeds step 6's `seen`, where the only ids it can decide are
2370            // commits, and `tree_closure` never walks into one.
2371            let excluded: OidSet = limited.uninteresting.iter().copied().collect();
2372            let commits = CommitClosure {
2373                ids: limited.ids,
2374                trees: limited.trees,
2375                edges: limited.edges,
2376            };
2377            (commits, excluded)
2378        }
2379    };
2380    let CommitClosure {
2381        ids: commit_ids,
2382        trees: commit_trees,
2383        // Where the client's history meets what it is being sent. Free either
2384        // way: both walks already had to decide which parents the client holds.
2385        edges: commit_edges,
2386    } = commits;
2387    for commit in &commit_ids {
2388        if emitted.insert(*commit) {
2389            selection.objects.push(*commit);
2390        }
2391    }
2392    selection.commits = commit_ids;
2393    crate::stage::end(_walk);
2394
2395    // 5. ...and as a set of every object under those commits. This is the
2396    //    expensive half, and the one the `git` package's reachability bitmaps replace.
2397    //
2398    //    **One set, and only one**. It starts as the excluded commits,
2399    //    moved rather than copied; the tree walk's `seen` set *is* this set, so
2400    //    a visited id is recorded once; and the walk tallies what it found
2401    //    rather than handing back a `Vec` of ids this function already holds.
2402    //    All three of those used to be separate repository-sized allocations.
2403    //    **Which** of the client's commits get opened up is [`Exclusion`], and
2404    //    it is the difference between reading 3 000 trees and reading 42.
2405    let mut excluded_objects = excluded_commits;
2406    {
2407        let _s = crate::stage::enter("select.exclude_trees");
2408        let mut tally = Emit::Tally(0);
2409        let mut buf = Vec::new();
2410        let mut parents = Vec::new();
2411        // `Full` already has every root tree in hand — the walk read them off
2412        // the commits it decoded; see `CommitClosure`. `Edges` names a handful
2413        // of commits instead and pays one `commit_links` each, which is a read
2414        // of an object sitting in the contiguous commit run at the front of the
2415        // pack, not another scattered fault in the middle of it.
2416        let roots: Vec<ObjectId> = match options.exclusion {
2417            Exclusion::Full => excluded_trees,
2418            Exclusion::Edges => {
2419                // The client's own tips, then the edge. Git's
2420                // `mark_edges_uninteresting` walks both: the tips because they
2421                // are what the client actually said it holds, the edge because
2422                // that is where the two histories meet and where the shared
2423                // content is.
2424                let mut roots = Vec::with_capacity(exclude_commits.len() + commit_edges.len());
2425                for id in exclude_commits.iter().chain(commit_edges.iter()) {
2426                    match graph.commit_links(id, &mut buf, &mut parents) {
2427                        Ok(tree) => roots.push(tree),
2428                        // The same tolerance step 2 already applies to a
2429                        // `have`: a shallow client can name a commit whose
2430                        // ancestry it does not hold, and refusing the fetch
2431                        // over it would be a regression on a path that used to
2432                        // skip it silently.
2433                        Err(Error::ObjectMissing { .. }) => {}
2434                        Err(err) => return Err(err),
2435                    }
2436                }
2437                roots
2438            }
2439        };
2440        for tree in &roots {
2441            graph.tree_closure(
2442                *tree,
2443                &mut excluded_objects,
2444                &HashSet::new(),
2445                None,
2446                Missing::Skip,
2447                // The client already has these; gunnar is not about to hold one.
2448                None,
2449                &mut tally,
2450            )?;
2451        }
2452    }
2453
2454    // 6. Their trees and blobs, plus anything a want pointed at directly.
2455    //
2456    //    The exclusion set is handed straight over to be step 6's `seen` set.
2457    //    It used to be cloned here, and that clone was the defect. Merging the two
2458    //    is the same predicate written once, not a shortcut: `tree_closure`
2459    //    skips an id that is in `exclude` **or** already in `seen`, so a `seen`
2460    //    that starts out holding everything `exclude` held decides every id
2461    //    identically. The `exclude` argument is left for `verify_connectivity`,
2462    //    the one caller that must keep the two apart.
2463    let _tree = crate::stage::enter("select.tree_closure");
2464    let mut seen = excluded_objects;
2465    // ── a NAMED want under a filter is never "already held" (FORGE-SHALLOW) ──
2466    //
2467    // The exclusion above closes over the client's `have` tips, and for a
2468    // partial clone that closure is a lie in the dangerous direction: a
2469    // `blob:none` client's tip reaches every blob under it and it was sent none
2470    // of them. Its lazy fetch (`git fetch --filter=blob:none origin <blob>`)
2471    // names exactly such a blob, and with the blob in `seen` the arm below
2472    // skipped it and the answer was an EMPTY pack (`did not send all necessary
2473    // objects`; git.gunnar.rs, edda, 2026-10-28). Stock `git upload-pack`
2474    // sends it. Without a filter the client's tips do vouch for everything
2475    // under them, so this is the filtered case only.
2476    if options.filter.is_some() {
2477        for (id, _) in &direct {
2478            seen.remove(id);
2479        }
2480    }
2481    // Every id this step puts into `seen`, so the two questions that set is
2482    // answering can be separated again at the end. See [`Carried`], and
2483    // `Selection::client_has` for what goes wrong when they are not.
2484    let mut carried: Vec<ObjectId> = Vec::with_capacity(emitted.len());
2485    let mut payload = Vec::new();
2486    let mut keep = Emit::Keep(&mut payload);
2487    {
2488        let mut visited = Carried {
2489            seen: &mut seen,
2490            carried: &mut carried,
2491        };
2492        for id in emitted.iter() {
2493            visited.insert(id);
2494        }
2495        // The named wants FIRST. The commit-tree walk below inserts every object
2496        // it visits into `seen`, including the ones a filter then declines to
2497        // send, so a want naming a blob under one of the wanted commits would
2498        // be skipped as "visited" if it ran second. What a want names is sent.
2499        for (id, kind) in direct {
2500            match kind {
2501                object::Kind::Tree => graph.tree_closure(
2502                    id,
2503                    &mut visited,
2504                    &HashSet::new(),
2505                    options.filter,
2506                    options.missing,
2507                    options.max_blob_bytes,
2508                    &mut keep,
2509                )?,
2510                // A `want` naming a blob outright — a partial clone's lazy fetch
2511                // does exactly this. It never reaches `tree_closure`, so the
2512                // ceiling has to be applied here as well or the one shape that
2513                // asks for a single enormous object by name is the one shape that
2514                // is not checked.
2515                _ => {
2516                    if let (Some(limit), object::Kind::Blob) = (options.max_blob_bytes, kind) {
2517                        if let Some((_kind, size)) =
2518                            graph.store.header(id.as_bytes()).map_err(backend)?
2519                        {
2520                            if size > limit {
2521                                return Err(Error::ObjectTooLarge {
2522                                    oid: id,
2523                                    size,
2524                                    limit,
2525                                });
2526                            }
2527                        }
2528                    }
2529                    if visited.insert(&id) {
2530                        keep.push(id);
2531                    }
2532                }
2533            }
2534        }
2535        for tree in &commit_trees {
2536            graph.tree_closure(
2537                *tree,
2538                &mut visited,
2539                &HashSet::new(),
2540                options.filter,
2541                options.missing,
2542                options.max_blob_bytes,
2543                &mut keep,
2544            )?;
2545        }
2546    }
2547    selection.objects.extend(payload);
2548    // ── the voucher is what the client HOLDS ────────────────────────────────
2549    //
2550    // `seen` is now the client's closure **plus** everything this transfer
2551    // visited, and only the first half may vouch for a thin delta's base. So
2552    // the second half goes back out. It is `O(objects in this transfer)` and
2553    // frees memory rather than taking any, which is why this is a subtraction
2554    // rather than the snapshot-before-step-6 that the one-set rule forbids.
2555    //
2556    // `carried` is the exact list, not `selection.objects`: a walk under a
2557    // partial-clone filter *visits* trees and blobs it then declines to send,
2558    // and those are neither held nor carried. Using the sent list would leave
2559    // them vouched for.
2560    for id in &carried {
2561        seen.remove(id);
2562    }
2563    selection.client_has = ClientHas::Ids(seen);
2564    crate::stage::end(_tree);
2565
2566    Ok(selection)
2567}
2568
2569/// Every object reachable from `tips`, as **one** set and no vector.
2570///
2571/// This answers `upload-pack`'s [`WantPolicy::ReachableFromRefs`] question: *is
2572/// this `want` reachable from some reference?* It used to be answered by
2573/// `select_objects(graph, tips, &[], …).objects.into_iter().collect()`, and
2574/// that phrasing holds the whole repository **twice** at its peak — a `Vec` of
2575/// every id, alive while the `HashSet` built out of it fills up. At ten million
2576/// sha256 ids that is a 320 MB vector plus a ~500 MB set, ~820 MB of peak for a
2577/// question that never wanted the order and never wanted a second copy. It is
2578/// The same shape, one caller further out.
2579///
2580/// Here the tree walk's `seen` set **is** the answer: it tallies rather than
2581/// emits, so nothing is built beside it. The set that comes back is the only
2582/// repository-sized allocation the call ever makes.
2583///
2584/// A tip that does not resolve is **skipped**, not fatal. A dangling reference
2585/// is a problem for `gc`; refusing every non-tip `want` in the repository
2586/// because one reference is broken is not this function's call to make — the
2587/// same judgement `Frontier::new` already makes about the same values.
2588///
2589/// [`WantPolicy::ReachableFromRefs`]: crate::WantPolicy::ReachableFromRefs
2590pub(crate) fn reachable_closure(graph: &Graph<'_>, tips: &[ObjectId]) -> Result<OidSet> {
2591    let mut seen = OidSet::new();
2592    let mut commit_tips = Vec::new();
2593    let mut direct_trees = Vec::new();
2594
2595    for tip in tips {
2596        if !graph.contains(tip)? {
2597            continue;
2598        }
2599        let peeled = match graph.peel(tip) {
2600            Ok(peeled) => peeled,
2601            Err(Error::ObjectMissing { .. }) => continue,
2602            Err(err) => return Err(err),
2603        };
2604        for tag in peeled.tags {
2605            seen.insert(tag);
2606        }
2607        match peeled.kind {
2608            object::Kind::Commit => commit_tips.push(peeled.id),
2609            object::Kind::Tree => direct_trees.push(peeled.id),
2610            _ => {
2611                seen.insert(peeled.id);
2612            }
2613        }
2614    }
2615
2616    let commits = graph.commit_closure(
2617        &commit_tips,
2618        &HashSet::new(),
2619        &HashSet::new(),
2620        None,
2621        // A repository with a gap in it is a problem for `gc`, not a reason to
2622        // refuse every fetch.
2623        Missing::Skip,
2624    )?;
2625    let mut tally = Emit::Tally(0);
2626    for (commit, tree) in commits.ids.iter().zip(&commits.trees) {
2627        seen.insert(*commit);
2628        graph.tree_closure(
2629            *tree,
2630            &mut seen,
2631            &HashSet::new(),
2632            None,
2633            Missing::Skip,
2634            None,
2635            &mut tally,
2636        )?;
2637    }
2638    for tree in direct_trees {
2639        graph.tree_closure(
2640            tree,
2641            &mut seen,
2642            &HashSet::new(),
2643            None,
2644            Missing::Skip,
2645            None,
2646            &mut tally,
2647        )?;
2648    }
2649    Ok(seen)
2650}
2651
2652/// What the repository already vouched for, before a push.
2653///
2654/// Built **once per push**, not once per reference command: a client pushing a
2655/// thousand refs in one go used to rebuild this a thousand times.
2656pub(crate) struct Frontier {
2657    /// Reference values that peel to a commit. The connectivity walk stops
2658    /// here: a commit a reference already points at passed this same check
2659    /// before that reference was allowed to move, so its closure is complete by
2660    /// induction.
2661    commits: HashSet<ObjectId>,
2662    /// Root trees of those commits, plus any reference value that peels to a
2663    /// tree or a blob. Anything in here is present and complete, so the tree
2664    /// walk does not descend into it.
2665    present: HashSet<ObjectId>,
2666}
2667
2668impl Frontier {
2669    /// `values` is every reference target the repository held before the push.
2670    ///
2671    /// Absent and unreadable values are skipped rather than fatal: a repository
2672    /// with a dangling reference is broken, but refusing every push until
2673    /// someone fixes it is not this function's call to make.
2674    pub(crate) fn new(graph: &Graph<'_>, values: &[ObjectId]) -> Result<Self> {
2675        let mut commits = HashSet::new();
2676        let mut present = HashSet::new();
2677        for id in values {
2678            if !graph.contains(id)? {
2679                continue;
2680            }
2681            match graph.peel(id) {
2682                Ok(peeled) if peeled.kind == object::Kind::Commit => {
2683                    commits.insert(peeled.id);
2684                }
2685                Ok(peeled) => {
2686                    present.insert(peeled.id);
2687                }
2688                Err(Error::ObjectMissing { .. }) => {}
2689                Err(err) => return Err(err),
2690            }
2691        }
2692        for commit in &commits {
2693            present.insert(graph.commit(commit)?.tree);
2694        }
2695        Ok(Frontier { commits, present })
2696    }
2697}
2698
2699/// Assert that every object reachable from `tips` is present, stopping at the
2700/// frontier.
2701///
2702/// This is receive-pack's connectivity check, and it walks **from the new tips
2703/// down**. It never computes the frontier's object closure, which is the whole
2704/// point: the frontier is every reference the repository already had, so its
2705/// closure is the entire repository, and paying for it once per pushed
2706/// reference made push latency a function of repository size.
2707///
2708/// # What this actually costs (the previous claim here was wrong)
2709///
2710/// The commit walk stops at any commit a reference already pointed at, so the
2711/// ordinary case — a push onto an existing branch — visits exactly the commits
2712/// being pushed. Their trees and blobs then get walked, with the frontier's
2713/// root trees excluded and every subtree visited at most once across the whole
2714/// push. So the honest bound is:
2715///
2716/// > `O(new commits + the tree closure of those commits)`
2717///
2718/// which is one snapshot's worth of trees, not the repository's history. It is
2719/// **not** `O(newly pushed history)` alone, and this is the part the old
2720/// comment claimed and never had:
2721///
2722/// * a blob is existence-checked, not inflated, so the tree side is cheap per
2723///   object but is still proportional to the number of paths in the snapshot;
2724/// * a push that bases a new branch on an old commit that is not itself a
2725///   reference value has nothing to stop the commit walk before the root, so it
2726///   degenerates to `O(history below the base)`.
2727///
2728/// Both remainders need reachability data gunnar does not have: the `git`
2729/// package's
2730/// `__gunnar_reach__` bitmaps, or the set of object ids the incoming pack
2731/// actually introduced, either of which turns "is this already here and
2732/// complete" into a lookup. Until one exists, this comment states the cost the
2733/// code has rather than the cost the design wants.
2734pub(crate) fn verify_connectivity(
2735    graph: &Graph<'_>,
2736    tips: &[ObjectId],
2737    frontier: &Frontier,
2738) -> Result<usize> {
2739    // 1. Peel the new tips. `peel` loads every object it walks through, so a
2740    //    tag chain is proven present by getting this far.
2741    //
2742    //    `tags` is bounded by `tips.len() * MAX_TAG_DEPTH` — by what the client
2743    //    sent, not by the repository — so it is a plain `HashSet` for the same
2744    //    reason `commit_closure`'s `edges` is. It is the only set here that
2745    //    exists purely for the returned count.
2746    let mut tags: HashSet<ObjectId> = HashSet::new();
2747    let mut commit_tips = Vec::new();
2748    let mut direct: Vec<(ObjectId, object::Kind)> = Vec::new();
2749    for tip in tips {
2750        let peeled = graph.peel(tip)?;
2751        tags.extend(peeled.tags);
2752        match peeled.kind {
2753            object::Kind::Commit => commit_tips.push(peeled.id),
2754            other => direct.push((peeled.id, other)),
2755        }
2756    }
2757
2758    // 2. The commits being pushed. `Missing::Fail` is what makes this a check:
2759    //    a commit named by a parent and absent from the store stops the push.
2760    let commits = graph.commit_closure(
2761        &commit_tips,
2762        &frontier.commits,
2763        &HashSet::new(),
2764        None,
2765        Missing::Fail,
2766    )?;
2767
2768    // 3. Their trees and blobs. One `seen` set for the whole push, so a subtree
2769    //    shared by ten pushed commits is walked once.
2770    //
2771    //    **`Tally`, not `Keep`.** The only caller — `receive_pack::apply` —
2772    //    throws the returned count's *identity* away and logs the number, so
2773    //    materialising a `Vec<ObjectId>` of every object in the push bought
2774    //    nothing and cost a second repository-sized allocation beside `seen`
2775    //    (the one-set rule, on the push side). There used to be a third: a `counted`
2776    //    `OidSet` deduplicating the three groups against each other, which
2777    //    cannot fire — a tag, a commit and a tree are different objects with
2778    //    different ids, `commit_closure` deduplicates its own output, and
2779    //    `seen` is shared across every `tree_closure` call below, so each group
2780    //    is already distinct and the groups are disjoint by kind. Three
2781    //    repository-sized structures on this path are now one.
2782    let mut seen = OidSet::new();
2783    let mut keep = Emit::Tally(0);
2784    for tree in &commits.trees {
2785        graph.tree_closure(
2786            *tree,
2787            &mut seen,
2788            &frontier.present,
2789            None,
2790            Missing::Fail,
2791            // A push is checked for completeness, not packed. Nothing here is
2792            // materialised, so the serve path's object ceiling does not apply —
2793            // the *receive* path's ceiling is `gunnar_pack::IndexOptions`, and
2794            // it has already run by the time this does.
2795            None,
2796            &mut keep,
2797        )?;
2798    }
2799    for (id, kind) in direct {
2800        match kind {
2801            object::Kind::Tree => graph.tree_closure(
2802                id,
2803                &mut seen,
2804                &frontier.present,
2805                None,
2806                Missing::Fail,
2807                None,
2808                &mut keep,
2809            )?,
2810            // A reference pointing straight at a blob. `peel` already read it.
2811            _ => {
2812                if seen.insert(id) {
2813                    keep.push(id);
2814                }
2815            }
2816        }
2817    }
2818
2819    Ok(tags.len() + commits.ids.len() + keep.len())
2820}
2821
2822/// Memoised "does this commit have an ancestor the client said it has".
2823///
2824/// This is git's `can_all_from_reach_with_flag`, which is what decides whether
2825/// the server may stop negotiating. Memoising across wants is what keeps it
2826/// `O(V+E)` for the whole request instead of `O(V+E)` per want. On a
2827/// merge-heavy repository that is milliseconds against minutes.
2828pub(crate) struct Reachability<'g, 'a> {
2829    graph: &'g Graph<'a>,
2830    common: OidSet,
2831    known: HashMap<ObjectId, bool>,
2832    cutoff: i64,
2833    visited: usize,
2834    budget: Option<usize>,
2835}
2836
2837impl<'g, 'a> Reachability<'g, 'a> {
2838    pub(crate) fn new(graph: &'g Graph<'a>, common: &[ObjectId], cutoff: i64) -> Self {
2839        Reachability {
2840            graph,
2841            common: common.iter().copied().collect(),
2842            known: HashMap::new(),
2843            cutoff,
2844            visited: 0,
2845            budget: graph.budget,
2846        }
2847    }
2848
2849    /// Can `start` reach any commit the client claimed?
2850    ///
2851    /// Commits older than the cutoff are answered `false` without being walked
2852    /// through: the client's oldest `have` bounds how far back a useful cut
2853    /// point can be, and without that bound a divergent history walks to the
2854    /// root of the repository on every round.
2855    ///
2856    /// # One read per commit, and why the obvious shape is two
2857    ///
2858    /// This used to be an iterative **post-order**: a commit was left on the
2859    /// stack under its unresolved parents and looked at again once they were
2860    /// known. That is a correct way to fold a DAG, and it reads every commit on
2861    /// the path **twice**: once going down, once on the way back up, because
2862    /// the second look has to re-parse the commit to get at its parents again.
2863    /// On a divergent negotiation that doubling was a third of the entire
2864    /// suite's most expensive arm.
2865    ///
2866    /// It is not needed, because the fold being computed is an *or*. A plain
2867    /// downward walk answers `true` the instant it touches a common commit, and
2868    /// if it exhausts without touching one then **every commit it visited** is
2869    /// proven not to reach a common. One sweep, one read each, and a stronger
2870    /// memo than the post-order produced. The `true` case only memoises `start`
2871    /// itself: the walk stops early, so nothing else on the frontier has been
2872    /// settled and claiming otherwise would be a lie the next `want` acts on.
2873    ///
2874    /// One consequence worth stating rather than discovering: `visited` now
2875    /// counts each commit **once**, where the post-order counted the ones it
2876    /// read twice twice. So the budget is reached after roughly twice as many
2877    /// commits as before. That is the count being *correct* rather than the
2878    /// ceiling being raised; the budget bounds objects visited, and it was
2879    /// previously charging some of them double. It does mean the effective
2880    /// ceiling on a divergent walk moved, so judge `DEFAULT_BUDGET` against the
2881    /// honest number.
2882    pub(crate) fn reaches_common(&mut self, start: ObjectId) -> Result<bool> {
2883        if self.common.is_empty() {
2884            return Ok(false);
2885        }
2886        if let Some(&answer) = self.known.get(&start) {
2887            return Ok(answer);
2888        }
2889
2890        // Breadth-first rather than depth-first so that a common commit a few
2891        // steps away is found in a few steps, whatever the depth of the branch
2892        // it happens to hang off.
2893        let mut queue: VecDeque<ObjectId> = VecDeque::new();
2894        let mut queued = OidSet::new();
2895        // Visited, and unless the walk stops early, proven `false`.
2896        let mut exhausted: Vec<ObjectId> = Vec::new();
2897        queue.push_back(start);
2898        queued.insert(start);
2899
2900        let mut found = false;
2901        while let Some(id) = queue.pop_front() {
2902            if self.common.contains(&id) {
2903                found = true;
2904                break;
2905            }
2906            match self.known.get(&id) {
2907                Some(true) => {
2908                    found = true;
2909                    break;
2910                }
2911                // Already proven not to reach a common: its ancestry adds
2912                // nothing, and re-walking it is what the memo exists to stop.
2913                Some(false) => continue,
2914                None => {}
2915            }
2916            self.visited += 1;
2917            if let Some(limit) = self.budget {
2918                if self.visited > limit {
2919                    return Err(Error::WalkBudget { limit });
2920                }
2921            }
2922            // `visited` is cumulative and `queued` is live, so they are two
2923            // different quantities and both need saying. This one is the shared
2924            // charge every other walk answers to, which is what puts a
2925            // negotiation's `queued` set under the same bytes ceiling as a
2926            // clone's `seen` set instead of under none.
2927            self.graph.charge_walk()?;
2928            if !self.graph.contains(&id)? {
2929                self.known.insert(id, false);
2930                continue;
2931            }
2932            let facts = self.graph.commit(&id)?;
2933            if facts.committed_at < self.cutoff {
2934                self.known.insert(id, false);
2935                continue;
2936            }
2937            exhausted.push(id);
2938            for parent in facts.parents {
2939                if queued.insert(parent) {
2940                    queue.push_back(parent);
2941                }
2942            }
2943        }
2944
2945        if found {
2946            self.known.insert(start, true);
2947        } else {
2948            // The queue drained, so every commit on this frontier had its whole
2949            // in-cutoff ancestry looked at and none of it was common.
2950            for id in exhausted {
2951                self.known.insert(id, false);
2952            }
2953        }
2954        Ok(found)
2955    }
2956}
2957
2958#[cfg(test)]
2959mod tests {
2960
2961    /// [`SelectOptions`] with the **bitmap short-circuit off**.
2962    ///
2963    /// # Why every walk test in this module needs it, as of 2026-08-10
2964    ///
2965    /// `SelectOptions::default()` leaves [`Bitmaps::Read`] on, and step 0 then
2966    /// asks [`GitServe::select`] before any walk happens. The old fixture store
2967    /// could not answer — `ObjectStore::reachable()` handed back `None`,
2968    /// because `MemObjectStore` had no reachability index — so the question was
2969    /// free and every test below fell through to the walk it was written about.
2970    ///
2971    /// [`MemGitServe::select`] **does** answer: it is a genuine reachability
2972    /// walk over an in-memory store, and it declines only for an empty store or
2973    /// an unknown `want`. So with the default options these tests measure the
2974    /// short-circuit and not the traversal — seen exactly that way when they
2975    /// first compiled again: *"the two-pass walk touched the store 3 times over
2976    /// a client history of 300 commits"*, and a fetch whose peak was 0 ids.
2977    ///
2978    /// `Bitmaps::Off` is the documented *"walk, always"*, the same switch
2979    /// `GUNNAR_BITMAP=off` sets, so this forces the subject of the test rather
2980    /// than working around anything. What step 0 itself does is
2981    /// `crate::reachable`'s to assert, and it does.
2982    fn walked() -> SelectOptions<'static> {
2983        SelectOptions {
2984            bitmaps: Bitmaps::Off,
2985            ..SelectOptions::default()
2986        }
2987    }
2988    use super::*;
2989    use gix_hash::Kind;
2990
2991    // The byte-level suites' fixture builder, reused rather than re-written
2992    // (one implementation). Declared in `lib.rs`; see the `#[path]` there.
2993    use crate::common::Fixture;
2994
2995    /// A fetch holds the client's object closure **once**.
2996    ///
2997    /// # What is measured, and why it is not a wall clock or an RSS reading
2998    ///
2999    /// The bug this guards is invisible to every other kind of assertion. It
3000    /// changes no byte on the wire, no object in the pack and no exit status;
3001    /// it only makes the server hold a second copy of a repository-sized set of
3002    /// object ids. So the instrument is a count of the ids the walkers hold at
3003    /// once ([`meter`]), which is the applied output of the thing that went
3004    /// wrong. Peak resident set is the same fact one layer down, but a
3005    /// `MemObjectStore` fixture *is* memory, with the objects outweighing the
3006    /// sets about four to one, so RSS on a fixture this size could not resolve
3007    /// the difference. That measurement belongs on a real repository, and it
3008    /// lives in the `bench/` harness.
3009    ///
3010    /// # The two bounds, and what each one catches
3011    ///
3012    /// **The ceiling** is `one closure + a scrap`. Before the fix the peak was
3013    /// the closure **twice** — `let mut seen = excluded_objects.clone();` —
3014    /// plus a `Vec` of the same ids and a separate `seen` set inside the
3015    /// exclusion walk. Seen RED: with the clone restored this asserts
3016    /// 2.0 × closure against a bound of 1.02 ×.
3017    ///
3018    /// **The floor** is what stops the guard being hollow, and it is not
3019    /// decoration: a `select_objects` that never built the exclusion closure at
3020    /// all would hold almost nothing, pass any ceiling, and be a far worse bug,
3021    /// because it would send the client the whole repository on every
3022    /// incremental fetch. `peak >= closure` says the closure really was built.
3023    /// The selection assertion below is the same claim from the other side.
3024    #[test]
3025    fn a_fetch_holds_the_clients_object_closure_once() {
3026        // 300 commits, each with its own tree and its own blob, so the closure
3027        // is ~900 ids. Far enough above the handful of scrap ids that the two
3028        // bounds cannot overlap.
3029        const HISTORY: usize = 300;
3030
3031        let fixture = Fixture::build(Kind::Sha1);
3032        let client_tip = fixture.extend(fixture.four, HISTORY, "client");
3033        let server_tip = fixture.extend(client_tip, 1, "server");
3034
3035        let graph = Graph::new(&fixture.git, fixture.kind);
3036        // `Full`, deliberately and explicitly: this test's subject is the
3037        // *whole* client closure being held exactly once, and under the
3038        // shipping [`Exclusion::Edges`] there is no whole client closure to
3039        // hold. `Edges` gets its own half at the bottom, and the two together
3040        // say more than either did alone.
3041        let options = SelectOptions {
3042            exclusion: Exclusion::Full,
3043            ..walked()
3044        };
3045
3046        // How big one copy of the client's closure is, measured by the same
3047        // walker rather than predicted: a clone with no excludes selects
3048        // exactly the set an incremental fetch has to exclude.
3049        let closure = select_objects(&graph, &[client_tip], &[], &options)
3050            .expect("the client's own closure")
3051            .objects
3052            .len();
3053        assert!(
3054            closure > 3 * HISTORY,
3055            "the fixture is not the shape this test needs: {HISTORY} commits \
3056             should close over more than {} objects, not {closure}",
3057            3 * HISTORY
3058        );
3059
3060        meter::reset_peak();
3061        assert_eq!(
3062            meter::live(),
3063            0,
3064            "a set from the sizing call above outlived it"
3065        );
3066        let selection = select_objects(&graph, &[server_tip], &[client_tip], &options)
3067            .expect("the incremental fetch");
3068        let peak = meter::peak();
3069
3070        // One new commit, its tree and its blob. Nothing the client already
3071        // had. Without this the memory bounds could be satisfied by a walk that
3072        // excluded far too much.
3073        assert_eq!(
3074            selection.objects.len(),
3075            3,
3076            "an incremental fetch one commit behind selected {} objects",
3077            selection.objects.len()
3078        );
3079
3080        assert!(
3081            peak >= closure,
3082            "the walk peaked at {peak} live object ids against a client closure \
3083             of {closure}: the exclusion set was never built, so this fetch is \
3084             about to send objects the client already has"
3085        );
3086        let ceiling = closure + 4 * selection.objects.len() + 16;
3087        assert!(
3088            peak <= ceiling,
3089            "the walk held {peak} object ids at once for a client closure of \
3090             {closure}, {:.2}× one copy, ceiling {ceiling}. The exclusion \
3091             closure is being materialised more than once; at ten \
3092             million objects each extra copy is ~450 MB per in-flight fetch",
3093            peak as f64 / closure as f64
3094        );
3095
3096        // ── the closure is now HANDED OUT, and that has a lifetime cost ──────
3097        //
3098        // The thin voucher needs the client's closure after the walk, to vouch for the
3099        // base of a thin delta, so `Selection::client_has` carries it to the
3100        // pack writer instead of dropping it. The peak asserted above is
3101        // unchanged — it is the same set moved, not a second copy — but it now
3102        // lives as long as the `Selection` does, which means it coexists with
3103        // the writer's own tables rather than dying before them. That is a
3104        // real entry in `limits.rs`'s ledger and it is asserted here rather
3105        // than described.
3106        let full_selected = selection.objects.len();
3107        let vouched = selection.client_has.len();
3108        assert!(
3109            vouched >= closure,
3110            "the walk handed back {vouched} ids to vouch with against a client \
3111             closure of {closure}: the set being carried is not the closure, so \
3112             a thin delta's base would be refused for want of evidence that was \
3113             computed and thrown away"
3114        );
3115        assert_eq!(
3116            meter::live(),
3117            vouched,
3118            "the ids alive after the walk are not exactly the set it handed \
3119             back, so something else is being retained too"
3120        );
3121        drop(selection);
3122        assert_eq!(
3123            meter::live(),
3124            0,
3125            "the client's closure outlived the Selection that owns it"
3126        );
3127
3128        // And now the shipping mode. `Edges` opens up the client's tips and
3129        // the edge and nothing else, so the set it holds is bounded by the
3130        // frontier rather than by the client's history — which is the entire
3131        // read-side claim of the serial walk, asserted here as *live ids*
3132        // rather than as a timing.
3133        meter::reset_peak();
3134        let edge_selection = select_objects(&graph, &[server_tip], &[client_tip], &walked())
3135            .expect("the incremental fetch, edge exclusion");
3136        let edge_peak = meter::peak();
3137
3138        // The pack must not have grown. `Edges` is allowed to send a *superset*
3139        // and this fixture is the shape where the superset is empty; if that
3140        // ever stops being true here, the trade has moved and this is where it
3141        // shows up rather than in a byte count nobody reads.
3142        assert_eq!(
3143            edge_selection.objects.len(),
3144            full_selected,
3145            "edge exclusion selected {} objects where the full closure selected \
3146             {full_selected}: the frontier walk is missing objects the client \
3147             demonstrably has",
3148            edge_selection.objects.len()
3149        );
3150        // And the set carried to the writer shrinks with it. This is what
3151        // bounds the thin voucher's new lifetime cost in the SHIPPING
3152        // configuration: the voucher is frontier-sized, not history-sized, so
3153        // a fetch does not hold the client's whole history alive while it
3154        // writes the pack.
3155        assert!(
3156            edge_selection.client_has.len() < vouched,
3157            "the voucher carried under edge exclusion is {} ids against the full \
3158             closure's {vouched}, so the frontier walk is handing the pack writer \
3159             the client's whole history after all",
3160            edge_selection.client_has.len()
3161        );
3162        // Seen RED by making `Exclusion::from_env`'s default `Full` and
3163        // `SelectOptions::default()` follow it: `edge_peak` then equals `peak`
3164        // and this fires with "held 915 ids ... against the full closure's 915".
3165        assert!(
3166            edge_peak < peak,
3167            "edge exclusion held {edge_peak} ids at once against the full \
3168             closure's {peak} over a client history of {HISTORY} commits. It is \
3169             walking the client's whole history after all, and the read-side \
3170             win is not there"
3171        );
3172    }
3173
3174    /// The traversal budget still bounds the **exclusion** walk.
3175    ///
3176    /// That walk stopped collecting its ids into a `Vec` when the extra copy
3177    /// was removed and tallies them instead, with the budget charged against
3178    /// whichever of the two the walk is using. A budget that silently stopped
3179    /// applying to the one walk sized by *the client's* history would be an
3180    /// unbounded client-driven allocation, which is the whole class this fix
3181    /// is in.
3182    ///
3183    /// # The first version of this test could not fail, and that is the point
3184    ///
3185    /// It used a budget of 10 against 200 commits, so the **commit** walk in
3186    /// step 3 tripped first and the assertion was green whatever the tree walk
3187    /// did. Proved by breaking `Emit::Tally`'s `len()` to return `0` and
3188    /// watching it stay green. The budget here is therefore chosen to sit in
3189    /// the gap between the two: comfortably above the commit count, and below
3190    /// the object count those commits close over. Seen RED with the same
3191    /// `len()` broken: "the exclusion walk ran to completion under a budget of
3192    /// 400 and selected 3 objects".
3193    #[test]
3194    fn the_budget_bounds_the_exclusion_walk_not_only_the_commit_walk() {
3195        // 300 + the fixture's own 5 commits, each with one tree and one blob:
3196        // ~305 commits to walk, ~610 objects under them.
3197        const HISTORY: usize = 300;
3198        const BUDGET: usize = 400;
3199
3200        let fixture = Fixture::build(Kind::Sha1);
3201        let client_tip = fixture.extend(fixture.four, HISTORY, "client");
3202        let server_tip = fixture.extend(client_tip, 1, "server");
3203
3204        // The commit walk alone must NOT reach the budget, or this test is the
3205        // hollow one it replaced. Measured against the same graph.
3206        let graph = Graph::new(&fixture.git, fixture.kind);
3207        let commits = graph
3208            .commit_closure(
3209                &[client_tip],
3210                &HashSet::new(),
3211                &HashSet::new(),
3212                None,
3213                Missing::Skip,
3214            )
3215            .expect("the client's commits");
3216        assert!(
3217            commits.ids.len() < BUDGET,
3218            "the {} commits alone reach a budget of {BUDGET}, so this test              cannot tell the two walks apart",
3219            commits.ids.len()
3220        );
3221
3222        let graph = Graph::new(&fixture.git, fixture.kind).with_budget(Some(BUDGET));
3223        let outcome = select_objects(
3224            &graph,
3225            &[server_tip],
3226            &[client_tip],
3227            &SelectOptions {
3228                exclusion: Exclusion::Full,
3229                ..walked()
3230            },
3231        );
3232
3233        match outcome {
3234            Err(Error::WalkBudget { limit }) => assert_eq!(limit, BUDGET),
3235            Err(other) => panic!("the budget should have stopped this walk, not {other}"),
3236            Ok(selection) => panic!(
3237                "the exclusion walk ran to completion under a budget of \
3238                 {BUDGET} and selected {} objects",
3239                selection.objects.len()
3240            ),
3241        }
3242
3243        // The other half of the same fact, and the reason the budget stopped
3244        // being reached in practice: under `Exclusion::Edges` this identical
3245        // fetch, on the identical graph, under the identical budget, does not
3246        // come near it. The budget still guards the walk — it is the walk that
3247        // got smaller.
3248        let edges = select_objects(&graph, &[server_tip], &[client_tip], &walked()).expect(
3249            "edge exclusion tripped a budget of 400 on a 305-commit history; \
3250             it is supposed to open up the frontier, not the history",
3251        );
3252        assert_eq!(
3253            edges.objects.len(),
3254            3,
3255            "edge exclusion selected {} objects where the full closure selects 3",
3256            edges.objects.len()
3257        );
3258    }
3259
3260    /// A fetch does not read the history the client just told it about.
3261    ///
3262    /// # What is measured
3263    ///
3264    /// Object **reads**, counted by the store itself. Not wall clock, which on
3265    /// a `MemObjectStore` is a `BTreeMap` lookup either way and would report a
3266    /// win of nothing; not the pack, which must not move at all. The claim of
3267    /// [`Limiting::Merged`] is that a fetch stops asking for commits the client
3268    /// already named, and a count of what the store was asked for is that claim
3269    /// as applied output.
3270    ///
3271    /// # The three bounds
3272    ///
3273    /// **The floor on the old walk.** `TwoPass` must read more commits than the
3274    /// client's history is long, or the fixture is not the shape that can tell
3275    /// the two apart and every ratio below is against nothing.
3276    ///
3277    /// **The ceiling on the new one.** `Merged` must read fewer than a tenth as
3278    /// many. Seen RED by handing both sides `Limiting::TwoPass`: "the merged
3279    /// walk read 916 objects against the two-pass walk's 916".
3280    ///
3281    /// **The pack must not move.** Same object *set*, both ways. The dangerous
3282    /// direction is a merged walk that stops too early and sends fewer objects
3283    /// than the client needs, which is a broken fetch rather than a slow one;
3284    /// a set comparison catches it in either direction. Seen RED by raising
3285    /// [`SLOP`]'s countdown into an unconditional `break` — "the merged walk
3286    /// selected 3 objects where the two-pass walk selected 18".
3287    #[test]
3288    fn a_fetch_does_not_read_the_history_the_client_already_told_it_about() {
3289        use crate::common::Counting;
3290
3291        const HISTORY: usize = 300;
3292        const DAY: usize = 5;
3293
3294        let fixture = Fixture::build(Kind::Sha1);
3295        // Committer dates ascending across BOTH calls, which `extend`'s default
3296        // does not give: it restarts its clock per call, so the commits a fetch
3297        // is about to send would be dated older than the entire history behind
3298        // them. That is a real shape — a rebased branch looks like it — but it
3299        // is the shape a date-ordered walk cannot shortcut, and pricing the
3300        // walk on it would be pricing its worst case as its behaviour.
3301        let client_tip = fixture.extend_dated(fixture.four, HISTORY, "client", |i| {
3302            1_700_200_000 + i as i64
3303        });
3304        let server_tip = fixture.extend_dated(client_tip, DAY, "server", |i| {
3305            1_700_200_000 + (HISTORY + i) as i64
3306        });
3307
3308        let measure = |limiting: Limiting| -> (usize, Vec<ObjectId>) {
3309            let store = Counting::new(fixture.git.inner.clone());
3310            let graph = Graph::new(&store, Kind::Sha1);
3311            let options = SelectOptions {
3312                limiting,
3313                ..walked()
3314            };
3315            let selection = select_objects(&graph, &[server_tip], &[client_tip], &options)
3316                .expect("the incremental fetch");
3317            let mut ids = selection.objects;
3318            ids.sort();
3319            (store.touches(), ids)
3320        };
3321
3322        let (two_pass_reads, two_pass_objects) = measure(Limiting::TwoPass);
3323        let (merged_reads, merged_objects) = measure(Limiting::Merged);
3324
3325        assert!(
3326            two_pass_reads > HISTORY,
3327            "the two-pass walk touched the store {two_pass_reads} times over a \
3328             client history of {HISTORY} commits, so it is not walking that \
3329             history and this test cannot tell the two walks apart"
3330        );
3331        assert_eq!(
3332            merged_objects,
3333            two_pass_objects,
3334            "the merged walk selected {} objects where the two-pass walk \
3335             selected {}. Fewer is a broken fetch, not a win",
3336            merged_objects.len(),
3337            two_pass_objects.len()
3338        );
3339        assert!(
3340            merged_objects.len() >= 3 * DAY,
3341            "an incremental fetch of {DAY} commits selected only {} objects, so \
3342             the comparison above is between two empty sets",
3343            merged_objects.len()
3344        );
3345        assert!(
3346            merged_reads * 10 < two_pass_reads,
3347            "the merged walk touched the store {merged_reads} times against the \
3348             two-pass walk's {two_pass_reads}, {:.2}x. It is still enumerating \
3349             the client's history, which is the whole cost this removes",
3350            two_pass_reads as f64 / merged_reads as f64
3351        );
3352    }
3353
3354    /// The merged walk and the two-pass walk pick the same objects, on the
3355    /// shapes where they could disagree.
3356    ///
3357    /// The measurement above runs one linear fetch. These are the shapes a
3358    /// date-ordered walk can get wrong and a breadth-first one cannot: a
3359    /// **merge** whose two parents are on different sides of the frontier, a
3360    /// `have` the client names that is also a `want`, a `have` that is an
3361    /// ancestor of another `have`, and a fetch whose wants are dated *older*
3362    /// than the history behind them, which is what `SLOP` exists for.
3363    ///
3364    /// Equality is asserted on the sorted id list, because the merged walk
3365    /// emits in committer-date order and the two-pass walk breadth-first, and
3366    /// the *set* is what a pack has to contain.
3367    ///
3368    /// Seen RED by dropping [`SLOP`] to `0`, which ends the walk on the first
3369    /// uninteresting pop: "skewed wants: merged selected 3, two-pass 15".
3370    #[test]
3371    fn the_two_walks_agree_on_the_shapes_that_could_split_them() {
3372        let fixture = Fixture::build(Kind::Sha1);
3373        let old_line = fixture.extend_dated(fixture.three, 8, "old", |i| 1_700_300_000 + i as i64);
3374        let new_line = fixture.extend_dated(fixture.two, 8, "new", |i| 1_700_400_000 + i as i64);
3375        // A merge of a recent line and a stale one: its second parent is older
3376        // than commits the client already has, which is exactly where a walk
3377        // ordered by date can pop things in a surprising order.
3378        let merged_tip = crate::common::commit(
3379            &fixture.git,
3380            fixture.kind,
3381            "merged.txt",
3382            "merged",
3383            &[new_line, old_line],
3384            1_700_400_100,
3385        );
3386        // Wants dated older than everything behind them: a rebased branch, and
3387        // the case `SLOP` absorbs.
3388        let skewed = fixture.extend_dated(new_line, 4, "skew", |_| 1_700_000_000);
3389
3390        // The `bool` is whether the fetch is expected to select anything at
3391        // all. Without it the "nothing to send" case would satisfy an equality
3392        // between two empty lists and quietly become the hollow half of this
3393        // guard — while still being worth asserting, because a merged walk that
3394        // sent the client its own tip back would show up here and nowhere else.
3395        let cases: [(&str, &[ObjectId], &[ObjectId], bool); 5] = [
3396            (
3397                "merge across the frontier",
3398                &[merged_tip],
3399                &[old_line],
3400                true,
3401            ),
3402            (
3403                "a want that is also a have",
3404                &[merged_tip],
3405                &[merged_tip],
3406                false,
3407            ),
3408            (
3409                "one have behind another",
3410                &[merged_tip],
3411                &[new_line, fixture.two],
3412                true,
3413            ),
3414            ("skewed wants", &[skewed], &[new_line], true),
3415            ("no haves at all", &[merged_tip], &[], true),
3416        ];
3417
3418        for (name, tips, haves, expect_objects) in cases {
3419            let pick = |limiting: Limiting| -> Vec<ObjectId> {
3420                let graph = Graph::new(&fixture.git, fixture.kind);
3421                let mut ids = select_objects(
3422                    &graph,
3423                    tips,
3424                    haves,
3425                    &SelectOptions {
3426                        limiting,
3427                        ..walked()
3428                    },
3429                )
3430                .unwrap_or_else(|err| panic!("{name}: {err}"))
3431                .objects;
3432                ids.sort();
3433                ids
3434            };
3435            let merged = pick(Limiting::Merged);
3436            let two_pass = pick(Limiting::TwoPass);
3437            assert_eq!(
3438                !two_pass.is_empty(),
3439                expect_objects,
3440                "{name}: the two-pass walk selected {} objects, which is not the \
3441                 shape this case is here to compare",
3442                two_pass.len()
3443            );
3444            assert_eq!(
3445                merged,
3446                two_pass,
3447                "{name}: merged selected {}, two-pass {}",
3448                merged.len(),
3449                two_pass.len()
3450            );
3451        }
3452    }
3453
3454    /// A `have` older than its own parent still stops the walk behind it.
3455    ///
3456    /// # The one thing the two comparisons above cannot see
3457    ///
3458    /// They assert that the merged walk and the two-pass walk pick the same
3459    /// objects. git's `mark_parents_uninteresting` is invisible to that,
3460    /// because the only shape where it changes anything is a shape where the
3461    /// two walks are *allowed* to differ — the merged one sends a documented
3462    /// superset there, so an equality assertion is either already failing or
3463    /// already blind. Proved: with the whole function replaced by an early
3464    /// `return`, all five tests in this module stay green.
3465    ///
3466    /// So this asserts the applied output directly, on the shape that needs it:
3467    ///
3468    /// ```text
3469    ///   A (t=1000) ── B (t=2000) ── W (t=3000)   the want
3470    ///                     └──────── U (t=1500)   the have, OLDER than B
3471    /// ```
3472    ///
3473    /// `W` is popped first and takes `B` for interesting, so `B`'s parent `A`
3474    /// is admitted interesting too. `U` arrives afterwards and marks `B`
3475    /// uninteresting — and `A` is only reached by pushing that mark on down
3476    /// through a commit already read, which is what the function does and the
3477    /// only thing that does it.
3478    ///
3479    /// Seen RED with that early `return` in place: "the merged walk sent 3
3480    /// commits, including A, which sits behind a commit the client holds".
3481    ///
3482    /// `B` itself **is** sent, and that is the trade rather than a bug: it was
3483    /// emitted before `U` was ever popped. The two-pass walk sends only `W`.
3484    /// The assertion records both numbers so the size of the superset is a
3485    /// stated fact instead of a surprise.
3486    #[test]
3487    fn a_have_older_than_its_own_parent_still_stops_the_walk_behind_it() {
3488        use crate::common::commit;
3489        use gunnar_store::mem::MemGitServe;
3490
3491        let store = MemGitServe::new(Kind::Sha1);
3492        let a = commit(&store, Kind::Sha1, "a.txt", "a", &[], 1_000);
3493        let b = commit(&store, Kind::Sha1, "b.txt", "b", &[a], 2_000);
3494        let u = commit(&store, Kind::Sha1, "u.txt", "u", &[b], 1_500);
3495        let w = commit(&store, Kind::Sha1, "w.txt", "w", &[b], 3_000);
3496
3497        let graph = Graph::new(&store, Kind::Sha1);
3498        let pick = |limiting: Limiting| -> Vec<ObjectId> {
3499            select_objects(
3500                &graph,
3501                &[w],
3502                &[u],
3503                &SelectOptions {
3504                    limiting,
3505                    ..walked()
3506                },
3507            )
3508            .expect("the skewed fetch")
3509            .commits
3510        };
3511
3512        let merged = pick(Limiting::Merged);
3513        assert!(
3514            !merged.contains(&a),
3515            "the merged walk sent {} commits, including A, which sits behind a \
3516             commit the client holds: the client's side of the graph stopped \
3517             propagating at the first commit already taken for interesting",
3518            merged.len()
3519        );
3520        assert_eq!(
3521            merged,
3522            vec![w, b],
3523            "the merged walk sent {merged:?}; W and B is the documented \
3524             superset on this shape, and anything else has moved it"
3525        );
3526        assert_eq!(
3527            pick(Limiting::TwoPass),
3528            vec![w],
3529            "the two-pass walk is the exact answer here and must stay it, or \
3530             the superset above is being measured against a moving baseline"
3531        );
3532    }
3533
3534    /// The edge is the parent the two histories actually share, not the tip the
3535    /// client named.
3536    ///
3537    /// # Why this was missing, and what it was missing from
3538    ///
3539    /// [`Exclusion::Edges`] opens up two things in step 5: the commits the
3540    /// client named as `have`s, and the **edge**. On a linear fetch those are
3541    /// the same commit, so every test in this crate that fetches a line forward
3542    /// is green with the edge set emptied — proved, by emptying it. The edge
3543    /// only does work when the two histories join *behind* what the client
3544    /// named:
3545    ///
3546    /// ```text
3547    ///   A ── C1 ── C2      the client's line; C2 is the `have`
3548    ///    └── S1            the server's;      S1 is the `want`
3549    /// ```
3550    ///
3551    /// `S1` carries the same file and the same content as `A`, so it shares
3552    /// `A`'s tree and blob object for object. `A` is two hops behind the `have`
3553    /// and is named by nobody. A step 5 that opened up only what the client
3554    /// named would put that tree and that blob in the pack again.
3555    ///
3556    /// Asserted for **both** walks, because both compute an edge set and this
3557    /// is the only test that can tell either of them is doing it.
3558    ///
3559    /// Seen RED, both ways: `Limiting::Merged`'s edge pass turned into
3560    /// `.take(0)` and `commit_closure`'s `out.edges.push(parent)` removed —
3561    /// "merged: a fetch sharing a tree and a blob with the client packed 3
3562    /// objects, not 1" and the same line for two-pass.
3563    #[test]
3564    fn the_exclusion_opens_up_the_parent_the_two_histories_share() {
3565        use crate::common::commit;
3566        use gunnar_store::mem::MemGitServe;
3567
3568        let store = MemGitServe::new(Kind::Sha1);
3569        let a = commit(&store, Kind::Sha1, "shared.txt", "shared", &[], 1_000);
3570        let c1 = commit(&store, Kind::Sha1, "c1.txt", "c1", &[a], 2_000);
3571        let c2 = commit(&store, Kind::Sha1, "c2.txt", "c2", &[c1], 3_000);
3572        // Same path, same content, so the same tree and the same blob as `A`.
3573        let s1 = commit(&store, Kind::Sha1, "shared.txt", "shared", &[a], 4_000);
3574        assert_ne!(
3575            s1, a,
3576            "the two commits must differ, only their content may not"
3577        );
3578
3579        let graph = Graph::new(&store, Kind::Sha1);
3580        for limiting in [Limiting::Merged, Limiting::TwoPass] {
3581            let selection = select_objects(
3582                &graph,
3583                &[s1],
3584                &[c2],
3585                &SelectOptions {
3586                    limiting,
3587                    ..walked()
3588                },
3589            )
3590            .expect("the branch fetch");
3591            assert_eq!(
3592                selection.objects,
3593                vec![s1],
3594                "{limiting:?}: a fetch sharing a tree and a blob with the client \
3595                 packed {} objects, not 1. The edge behind the client's named \
3596                 tip was never opened up, so content the client demonstrably \
3597                 holds is going back over the wire",
3598                selection.objects.len()
3599            );
3600        }
3601    }
3602
3603    /// `CommitClosure::trees[i]` really is `ids[i]`'s root tree.
3604    ///
3605    /// # Why this needs its own guard, when 500 other tests already pack packs
3606    ///
3607    /// Nothing downstream can tell a **mispaired** tree from a correct one. Every
3608    /// tree in `trees` is a real tree of a real commit in the same closure, so a
3609    /// walk that paired them by an off-by-one would still visit exactly the same
3610    /// set of trees, still emit a complete and self-consistent pack, and still
3611    /// satisfy every "the client got what it asked for" assertion in the suite —
3612    /// on a linear history, where the closure is one commit per tree, it would
3613    /// produce a **byte-identical** pack. The bug only becomes visible on a
3614    /// shape where the sets differ, and it would then look like a corrupt
3615    /// repository rather than like a walker bug. So the pairing is asserted
3616    /// here, element by element, against a **second and independent** read of
3617    /// each commit through `Graph::commit`, which is the full
3618    /// `CommitRef::from_bytes` decoder rather than the `CommitRefIter` one the
3619    /// walk now uses. That the two decoders agree is the other half of what is
3620    /// being checked.
3621    ///
3622    /// Seen RED, both ways: `out.trees.push(tree)` moved above `out.ids.push(id)`
3623    /// with a one-element skew fails on the first pair; returning
3624    /// `graph.commit(&id)?.tree` for the *previous* commit fails likewise. The
3625    /// budget test above and the one-set test both stay green under either edit.
3626    ///
3627    /// # The floor
3628    ///
3629    /// A `zip` over two empty vectors asserts nothing and passes. `HISTORY` is
3630    /// checked to have actually arrived, because that is the shape this guard
3631    /// would otherwise silently take.
3632    #[test]
3633    fn the_commit_walk_hands_back_each_commits_own_root_tree() {
3634        const HISTORY: usize = 40;
3635
3636        let fixture = Fixture::build(Kind::Sha1);
3637        let tip = fixture.extend(fixture.four, HISTORY, "p010");
3638        let graph = Graph::new(&fixture.git, fixture.kind);
3639
3640        let closure = graph
3641            .commit_closure(
3642                &[tip],
3643                &HashSet::new(),
3644                &HashSet::new(),
3645                None,
3646                Missing::Fail,
3647            )
3648            .expect("the commit closure");
3649
3650        assert_eq!(
3651            closure.ids.len(),
3652            closure.trees.len(),
3653            "the two halves of a CommitClosure must be the same length or the \
3654             pairing is not a pairing"
3655        );
3656        assert!(
3657            closure.ids.len() > HISTORY,
3658            "the fixture is not the shape this test needs: {HISTORY} commits on \
3659             top of the base should close over more than {HISTORY} commits, not \
3660             {}. A zip over a shorter list would assert nothing.",
3661            closure.ids.len()
3662        );
3663
3664        for (i, (id, tree)) in closure.ids.iter().zip(&closure.trees).enumerate() {
3665            let independent = graph
3666                .commit(id)
3667                .expect("re-reading a commit the walk just read")
3668                .tree;
3669            assert_eq!(
3670                *tree, independent,
3671                "commit {i} ({id}) was paired with tree {tree}, but its own root \
3672                 tree is {independent}"
3673            );
3674        }
3675    }
3676
3677    // ── the bytes ceiling ──────────────────────────────────────────────────
3678
3679    /// The number of ids a `walk_bytes` ceiling of `objects` objects is.
3680    fn ceiling_for(objects: usize) -> usize {
3681        objects * crate::WALK_BYTES_PER_OBJECT
3682    }
3683
3684    /// **A fetch is charged for what it HOLDS, not for what it sends**, and a
3685    /// `blob:none` filter is where the two come apart.
3686    ///
3687    /// # Why this shape and not a big clone
3688    ///
3689    /// The old ceiling compared `Emit::len()` — the objects the walk is about
3690    /// to send — against the budget. On a full clone that is nearly the whole
3691    /// set held and the difference does not show. Under `filter=blob:none` the
3692    /// walk still puts **every blob** into its `seen` set, because it has to
3693    /// know it has been there, and then declines to emit any of them. So the
3694    /// thing being counted stops growing while the thing being allocated keeps
3695    /// going, and a ceiling written against the first cannot see the second.
3696    ///
3697    /// # Both edges, and the floor
3698    ///
3699    /// | ceiling | must |
3700    /// |---|---|
3701    /// | the ids the walk EMITS | refuse — it is holding more than that |
3702    /// | the ids the walk HOLDS, and a scrap | serve |
3703    ///
3704    /// The second is not decoration. A ceiling that refused at both would be
3705    /// satisfied by a charge that is simply too high — or by a walk that cannot
3706    /// serve anything — and this test would then be green about a server that
3707    /// clones nothing.
3708    ///
3709    /// Seen RED with `charge_walk` reading zero instead of [`meter::live`]:
3710    /// *"a ceiling the size of what this fetch sends must refuse it"*.
3711    ///
3712    /// # What it does NOT guard, said plainly
3713    ///
3714    /// [`WALK_BYTES_PER_OBJECT`](crate::WALK_BYTES_PER_OBJECT). Doubling that
3715    /// constant leaves this test green, because `ceiling_for` sizes the
3716    /// ceiling from the same constant — deliberately, so the test is about the
3717    /// mechanism and survives a re-measurement, but it does mean nothing here
3718    /// can tell a well-chosen rate from a wrong one. The rate is a measurement
3719    /// and only a measurement can check it: `bench/examples/walkbudget-probe.rs`,
3720    /// and the table on the constant is what it produced.
3721    #[test]
3722    fn the_ceiling_charges_what_the_walk_holds_not_what_it_sends() {
3723        const HISTORY: usize = 300;
3724        let fixture = Fixture::build(Kind::Sha1);
3725        let tip = fixture.extend(fixture.four, HISTORY, "held");
3726
3727        let blobless = Filter::parse("blob:none").expect("blob:none parses");
3728        // The ceiling under test lives on the `Graph`, not here: it is a
3729        // property of the traversal scope, which is what has to release the
3730        // pool reservation when the request ends.
3731        let options = |filter| SelectOptions { filter, ..walked() };
3732
3733        // What the blob:none walk HOLDS, and what it SENDS. Both measured by
3734        // the same walker rather than predicted — `selection.objects.len()` is
3735        // not the first of them, which is the whole point of this test and was
3736        // worth finding out the hard way: on this fixture the walk sends 915
3737        // ids and holds 980, because `seen` carries the peeled tips and tags
3738        // the emitted vector never does.
3739        let (sent, held) = {
3740            let graph = Graph::new(&fixture.git, fixture.kind);
3741            let sent = select_objects(&graph, &[tip], &[], &options(Some(&blobless)))
3742                .expect("the blob:none clone")
3743                .objects
3744                .len();
3745            (sent, meter::peak())
3746        };
3747        assert!(
3748            held > sent + 100,
3749            "the fixture is not the shape this test needs: a blob:none clone \
3750             sent {sent} objects and held {held}, and the two have to be far \
3751             enough apart that a ceiling can sit between them"
3752        );
3753
3754        // ── the ceiling at what it SENDS must refuse ────────────────────────
3755        let graph =
3756            Graph::new(&fixture.git, fixture.kind).with_walk_limit(Some(ceiling_for(sent)), None);
3757        let refused = select_objects(&graph, &[tip], &[], &options(Some(&blobless)))
3758            .expect_err("a ceiling the size of what this fetch sends must refuse it");
3759        match refused {
3760            Error::WalkTooLarge { held: bytes, limit } => {
3761                assert_eq!(
3762                    limit,
3763                    ceiling_for(sent),
3764                    "the refusal named another ceiling"
3765                );
3766                assert!(
3767                    bytes > limit,
3768                    "the refusal said it was holding {bytes} bytes against a \
3769                     ceiling of {limit}, which is not over it"
3770                );
3771            }
3772            other => panic!(
3773                "a blob:none fetch over the byte ceiling failed with {other}, not WalkTooLarge — \
3774                 so the ceiling is not what stopped it and this test is measuring another fault"
3775            ),
3776        }
3777
3778        // ── and the ceiling at what it HOLDS must serve ─────────────────────
3779        let graph =
3780            Graph::new(&fixture.git, fixture.kind).with_walk_limit(Some(ceiling_for(held)), None);
3781        let served = select_objects(&graph, &[tip], &[], &options(Some(&blobless)))
3782            .expect("a ceiling above what this fetch holds must serve it");
3783        assert_eq!(
3784            served.objects.len(),
3785            sent,
3786            "the served arm sent {} objects where the unbounded one sent {sent}: a ceiling that \
3787             changes the ANSWER rather than refusing is worse than the fault being measured",
3788            served.objects.len()
3789        );
3790    }
3791
3792    /// The ceiling is **off** when `walk_bytes` is `None`, and that is the arm
3793    /// that keeps the one above honest: without it, a `Graph` that refused
3794    /// every walk would satisfy the refusal half and the serve half would only
3795    /// be saying the number was large.
3796    #[test]
3797    fn no_ceiling_means_no_ceiling() {
3798        let fixture = Fixture::build(Kind::Sha1);
3799        let tip = fixture.extend(fixture.four, 50, "off");
3800        let graph = Graph::new(&fixture.git, fixture.kind).with_walk_limit(None, None);
3801        let all = select_objects(&graph, &[tip], &[], &walked())
3802            .expect("an unbounded walk serves")
3803            .objects
3804            .len();
3805        assert!(
3806            all > 50,
3807            "the fixture holds {all} objects, too few to mean anything"
3808        );
3809    }
3810
3811    /// **The pool is what bounds the box**: two walks, one purse, and the
3812    /// second is refused because the first is holding it.
3813    ///
3814    /// # Why this is a different assertion from the one above
3815    ///
3816    /// A per-request ceiling that admits one large clone admits 256 of them,
3817    /// because `gunnar serve` counts transfers and not bytes. So the fact under
3818    /// test is not "a walk can be refused" — it is "a walk that would have been
3819    /// admitted on its own is refused because of what somebody else is holding",
3820    /// and that cannot be shown with one walk.
3821    ///
3822    /// Seen RED twice, and the two failures name different halves. With
3823    /// `Graph::charge_walk`'s pool arm disabled: *"the first walk reserved 0 of
3824    /// the 16777216 it holds, so there is nothing for the second to be refused
3825    /// by"*. With `Drop for Graph` releasing nothing: *"the pool did not get
3826    /// its bytes back, so this server refuses everything after enough
3827    /// clones"* — which is the failure mode where a pool closes for good.
3828    #[test]
3829    fn the_pool_refuses_a_walk_because_of_what_another_is_holding() {
3830        // Big enough to cross `WalkPool::FLOOR_BYTES`, and not one commit more.
3831        // The floor is 8 MiB, which is 21 845 ids at the charge rate, and
3832        // `extend` lays down a commit, a tree and a blob apiece. The shipped
3833        // constants are used rather than lowered for the test: a floor that is
3834        // only ever crossed by a number the test chose is not the floor the
3835        // server has.
3836        let history = crate::WalkPool::FLOOR_BYTES / crate::WALK_BYTES_PER_OBJECT / 3 + 500;
3837        let fixture = Fixture::build(Kind::Sha1);
3838        let tip = fixture.extend(fixture.four, history, "pool");
3839
3840        let held = {
3841            let graph = Graph::new(&fixture.git, fixture.kind);
3842            select_objects(&graph, &[tip], &[], &walked()).expect("the whole clone");
3843            meter::peak()
3844        };
3845        assert!(
3846            ceiling_for(held) >= crate::WalkPool::FLOOR_BYTES,
3847            "this fixture holds {held} ids, {} bytes, which is under the pool's floor of {} \
3848             — the walk would never touch the pool and every arm below would pass against \
3849             nothing",
3850            ceiling_for(held),
3851            crate::WalkPool::FLOOR_BYTES
3852        );
3853
3854        // A purse sized to exactly what one of these walks reserves: room for
3855        // the first and none for the second. Computed from the measurement
3856        // rather than typed, so the test cannot drift out of the shape it needs
3857        // when the fixture or the charge rate moves.
3858        let reservation = ceiling_for(held).next_multiple_of(crate::WalkPool::CHUNK_BYTES);
3859        let pool = crate::WalkPool::new(reservation);
3860
3861        let outer =
3862            Graph::new(&fixture.git, fixture.kind).with_walk_limit(None, Some(pool.clone()));
3863        let holding = select_objects(&outer, &[tip], &[], &walked())
3864            .expect("the first walk fits the empty pool");
3865        assert!(!holding.objects.is_empty());
3866        assert_eq!(
3867            pool.in_flight_bytes(),
3868            reservation,
3869            "the first walk reserved {} of the {reservation} it holds, so there is nothing for \
3870             the second to be refused by",
3871            pool.in_flight_bytes()
3872        );
3873
3874        // ── the second walk, on its own perfectly reasonable ────────────────
3875        let inner =
3876            Graph::new(&fixture.git, fixture.kind).with_walk_limit(None, Some(pool.clone()));
3877        let refused =
3878            select_objects(&inner, &[tip], &[], &walked()).expect_err("a full pool must refuse");
3879        match refused {
3880            Error::WalkPoolFull { limit, .. } => assert_eq!(limit, reservation),
3881            other => panic!(
3882                "a walk against a full pool failed with {other}, not WalkPoolFull — a client \
3883                 cannot tell a busy server from a request it must never send again"
3884            ),
3885        }
3886        assert_eq!(pool.refusals(), 1, "the refusal was not counted");
3887        drop(inner);
3888
3889        // ── and the purse comes back ────────────────────────────────────────
3890        drop(outer);
3891        drop(holding);
3892        assert_eq!(
3893            pool.in_flight_bytes(),
3894            0,
3895            "the pool did not get its bytes back, so this server refuses everything after enough \
3896             clones"
3897        );
3898        let again =
3899            Graph::new(&fixture.git, fixture.kind).with_walk_limit(None, Some(pool.clone()));
3900        select_objects(&again, &[tip], &[], &walked())
3901            .expect("the same walk must serve once the first has finished");
3902    }
3903
3904    /// A release the pool never handed out cannot wrap it shut.
3905    ///
3906    /// `saturating_sub` on a `usize` that went negative would be `usize::MAX`,
3907    /// and a pool whose `in_flight` is `usize::MAX` refuses every transfer
3908    /// forever. Seen RED with a plain `fetch_sub`.
3909    #[test]
3910    fn walk_pool_never_goes_negative() {
3911        let pool = crate::WalkPool::new(1024);
3912        pool.release(4096);
3913        assert_eq!(pool.in_flight_bytes(), 0);
3914        pool.try_reserve(1024)
3915            .expect("a pool at zero has all of itself");
3916        assert_eq!(pool.in_flight_bytes(), 1024);
3917    }
3918
3919    /// A pool sized to this machine is not sized to a constant.
3920    #[test]
3921    fn a_pool_for_this_machine_reads_the_machine() {
3922        let pool = crate::WalkPool::for_this_machine();
3923        assert!(
3924            pool.limit_bytes() > 0,
3925            "a pool of zero bytes refuses every transfer"
3926        );
3927        if let Ok(text) = std::fs::read_to_string("/proc/meminfo") {
3928            let kb: usize = text
3929                .lines()
3930                .find_map(|l| l.strip_prefix("MemTotal:")?.trim().strip_suffix(" kB"))
3931                .and_then(|v| v.trim().parse().ok())
3932                .expect("MemTotal is in /proc/meminfo");
3933            assert_eq!(
3934                pool.limit_bytes(),
3935                kb * 1024 / 2,
3936                "the pool is not half of this machine's memory, so it is a constant wearing a \
3937                 measurement's name"
3938            );
3939        }
3940    }
3941}