znippy_plugin_git/exploded_cache.rs
1//! **The exploded tier is a CACHE, not a tier — and this is the bound.**
2//!
3//! The owner, 2026-08-30, on being shown the lazy-explode design: *"does lazy
4//! explode mean it eventually takes the disk anyway?"* — and the honest answer
5//! is **yes, unless the tier is capped**, because a client that browses a whole
6//! history reads every object once and a write-back-on-first-read tier would
7//! then hold exactly what the eager one held. MEASURED on the rig 2026-08-30:
8//! the 8.12 GiB `torvalds/linux` pack drove the eager tier to **141.7 GiB**
9//! into `/data`, ~17× the pack. *"We cannot have one repo take all 300 GB
10//! disk."*
11//!
12//! So the tier has a **byte budget** and evicts. Nothing here is data: every
13//! row is re-derivable from the verbatim pack bytes by
14//! [`crate::resolve::materialize_at`], so an eviction is never a loss and never
15//! an error a client can see — it is a later read paying ~0.5–2 ms to inflate
16//! its own delta chain again.
17//!
18//! # Two pools, and the split is the whole design
19//!
20//! ```text
21//! objects.exploded the EAGER pool — what the indexer writes at push
22//! time. Under the shipped policy that is commits
23//! and tags: what `graph`/`reach`/tag-peel need, so
24//! visibility and pull checks never stall.
25//! NEVER evicted — it is the "commits last" rule,
26//! held by construction rather than by a comparator.
27//!
28//! objects.exploded.c1 the LAZY pool — first-read write-backs, appended
29//! objects.exploded.c2 in read order, one file per segment.
30//! … EVICTION IS ONE `unlink` of the oldest segment.
31//! ```
32//!
33//! Both pools are counted against the budget and both are named on the
34//! dashboard; only the lazy one is evictable. If the eager pool alone exceeds
35//! the budget the lazy pool is evicted to nothing and the eager pool is still
36//! kept — a store that cannot answer `reach` is broken, and a store that reads
37//! slowly is not.
38//!
39//! # Why segments, and why eviction is an `unlink`
40//!
41//! [`ExplodedArchive`] is one append-only Arrow IPC stream: `[schema][batch]…`,
42//! nothing ever rewritten. That is what makes it 1.0× its payload instead of
43//! redb's 12×, and it is also what makes *deleting a row in the middle* cost a
44//! whole-file rewrite — up to 17 GB, which the module docs already refuse to
45//! smuggle into `gc`. Cutting the pool into segment files turns "reclaim the
46//! coldest bytes" into `remove_file`: O(1), no rewrite, no read amplification,
47//! and a crash during it leaves a pool that is merely shorter.
48//!
49//! # The recency argument, stated honestly
50//!
51//! Segments are appended in **first-read order**, so dropping the oldest drops
52//! the objects least recently *admitted*. That is not the same as least
53//! recently *used* — an object read at minute 1 and again at minute 90 sits in
54//! an old segment. It is not promoted, and it does not need to be: when its
55//! segment is dropped, the next read of it misses, re-inflates its chain, and
56//! **writes it back into the newest segment**. Re-entry-on-miss is what makes
57//! this LRU-shaped, at the cost of one chain inflate per hot object per eviction
58//! cycle rather than a copy per hit. So it is FIFO over segments with
59//! re-entry-on-miss, and the thing an LRU is wanted for — a working set that
60//! stays resident under a scan of everything else — holds, because the working
61//! set is what keeps re-entering.
62//!
63//! # What a full-history read now costs on disk
64//!
65//! The plateau, not the climb. A reader that walks every object of a
66//! Linux-sized repository writes back every object it inflates; the pool grows
67//! to the budget and then stops, because each rotation past it drops the oldest
68//! segment. That is the number lane Q reports as its headline, and
69//! `the_tier_plateaus_at_its_budget_under_a_full_scan` is the test that asserts
70//! it rather than the sentence.
71
72use std::path::{Path, PathBuf};
73use std::sync::Mutex;
74use std::sync::atomic::{AtomicU64, Ordering};
75
76use anyhow::Result;
77
78use crate::exploded::ExplodedStats;
79use crate::exploded_arrow::{ExplodePolicy, ExplodedArchive};
80use crate::object::GitObjectKind;
81
82/// One lazy segment is closed and a new one started once it passes this many
83/// bytes. Eviction granularity is one segment, so this is the quantum a budget
84/// is spent and reclaimed in: 1/16 of the budget, floored so a small budget
85/// still gets whole batches and ceilinged so a huge one does not accumulate
86/// thousands of open indexes.
87const MIN_SEGMENT: u64 = 16 << 20;
88const MAX_SEGMENT: u64 = 1 << 30;
89const SEGMENTS_PER_BUDGET: u64 = 16;
90
91/// The lazy pool buffers this much before it writes. Far under
92/// [`crate::exploded_arrow::FLUSH_BYTES`] on purpose: that 64 MiB buffer is
93/// sized for an indexer draining a pack, and a **reader's** write-back holding
94/// 64 MiB per store is memory the box did not agree to spend.
95pub const CACHE_FLUSH_BYTES: u64 = 4 << 20;
96
97// ════════════════════════════════════════════════════════════════════════════
98// A VALIDATION READ IS NOT A READER
99// ════════════════════════════════════════════════════════════════════════════
100
101thread_local! {
102 /// Depth, not a flag: guards nest, and the innermost one ending must not
103 /// re-open the cache for an outer scope that is still validating.
104 static SUPPRESS_DEPTH: std::cell::Cell<u32> = const { std::cell::Cell::new(0) };
105}
106
107/// **Declare that reads on this thread are validation, not a reader's.**
108///
109/// `receive-pack` reads every commit and tree it was sent — once — to prove the
110/// push closes, and then never reads them again. Those reads look identical to
111/// a client's from inside the store, so the store cannot tell them apart and
112/// the caller must say.
113///
114/// MEASURED on `torvalds/linux`, oden 2026-09-03: the lazy pool is **empty
115/// until the instant indexing completes** and then grows to
116/// **6 778 047 872 B** (13 segments) before the push returns — 31 % of the
117/// whole store and the largest non-pack thing in it, written for a reader that
118/// does not exist. Full table in
119/// `gunnar/.nornir/storage-levers-2026-09-03.md`.
120///
121/// # Scope
122///
123/// **Thread-local, and that is the point.** A `receive-pack` connection does
124/// its ingest, its index and its connectivity walk on one thread and serves
125/// nobody; the background absorber runs elsewhere and is untouched, and so is
126/// every serving connection. Nothing global is switched off, so a store that is
127/// simultaneously answering a clone still warms its cache from that clone.
128///
129/// The eager pool is not affected either: the indexer writes that one directly,
130/// not through [`ExplodedCache::write_back`].
131///
132/// # It is a guard, not a pair of calls
133///
134/// Restored on drop, including on unwind — a panic inside a connectivity walk
135/// must not leave a thread that never caches again.
136#[must_use = "the suppression lasts only as long as the guard is held"]
137pub fn suppress_write_back() -> SuppressWriteBack {
138 SUPPRESS_DEPTH.with(|d| d.set(d.get().saturating_add(1)));
139 SuppressWriteBack { _private: () }
140}
141
142/// Whether [`suppress_write_back`] is in force on this thread.
143pub fn write_back_suppressed() -> bool {
144 SUPPRESS_DEPTH.with(|d| d.get()) > 0
145}
146
147/// The guard [`suppress_write_back`] returns. See it for what this is for.
148pub struct SuppressWriteBack {
149 _private: (),
150}
151
152impl Drop for SuppressWriteBack {
153 fn drop(&mut self) {
154 SUPPRESS_DEPTH.with(|d| d.set(d.get().saturating_sub(1)));
155 }
156}
157
158// ════════════════════════════════════════════════════════════════════════════
159// The admission policy: DO NOT EXPLODE UNIFORMLY
160// ════════════════════════════════════════════════════════════════════════════
161
162/// **The cheapest thing a cache can do is not cache.**
163///
164/// The owner, 2026-08-30: *"can the index point at the original git object and
165/// decompress it directly instead of materialising everything?"* — and it can,
166/// because a packfile is not one stream. Every object is its own zlib stream at
167/// its own offset, so `oid → (offset, type, base, inflated size)` is enough to
168/// inflate exactly one object ([`crate::resolve::materialize_at`]). A **whole**
169/// object is then one `inflate` of a mapped byte range: microseconds, no chain,
170/// no delta application. Writing that into a cache costs a copy, a flush, an
171/// fsync and a slice of somebody's quota — to save microseconds it did not
172/// need saving.
173///
174/// So the cache admits on **rebuild cost**, not on being read:
175///
176/// # It admits on cost DENSITY, not on cost
177///
178/// The first cut admitted anything over 16 KiB "because the inflate itself is
179/// the cost". That is true and it is the wrong rule, because the budget is
180/// **shared**: what a cache should hold is what saves the most time *per byte
181/// held*. A 3 MiB whole blob costs one linear inflate — bandwidth-bound,
182/// microseconds a kilobyte — and would occupy the same 3 MiB as ~250 deep tree
183/// chains that each cost a chain walk. Admitting it evicts all of them.
184///
185/// | what the read paid | admitted? |
186/// |---|---|
187/// | over [`MAX_CACHED_BYTES`] inflated | **no** — a bad tenant of a shared budget whatever its depth; its direct read is bandwidth-bound anyway |
188/// | depth 0–[`CHEAP_DEPTH`] | **no** — one inflate of one mapped zlib stream, plus at most two delta applications. Caching it spends quota to save microseconds |
189/// | deeper than [`CHEAP_DEPTH`] | yes — every extra link is another inflate plus a delta application, and git allows chains to 50 |
190/// | a base outside this archive | yes — it went through a [`BaseSource`](crate::resolve::BaseSource), which is never the cheap path |
191///
192/// The thresholds are named, not tuned: `CHEAP_DEPTH` is 2 because git's own
193/// default `pack.depth` is 50 and a 0–2 chain is at most two delta applications
194/// over a mapped buffer; `MAX_CACHED_BYTES` is 1 MiB because that is where one
195/// object starts costing what hundreds of chain-heavy ones do.
196///
197/// **This is also the zero-copy rule.** A declined object is declined *before*
198/// anything is copied, so the commonest reads — and every large one — cost the
199/// cache exactly zero bytes. MEASURED by lane AA 2026-08-30: one 3 145 728 B
200/// object read through `gunnar-cli` allocated 6 340 693 B, exactly two copies,
201/// and the second was the cache taking a blob it should never have wanted.
202pub const CHEAP_DEPTH: u32 = 2;
203pub const MAX_CACHED_BYTES: u64 = 1 << 20;
204
205/// What it cost to produce an object, as the cache scores it.
206///
207/// `depth + 1` inflates and `depth` delta applications over `inflated` bytes.
208/// Deliberately a coarse integer and not a timing: a duration measured on the
209/// reader's thread carries that thread's scheduling, and two identical objects
210/// would then score differently depending on how busy the box was.
211pub fn rebuild_cost(depth: u32, inflated: u64, external_base: bool) -> u64 {
212 // Bytes inflated across the whole chain, roughly: every link inflates
213 // something of about the object's order, plus one delta application each.
214 let links = depth as u64 + 1;
215 let base = links * inflated.max(1);
216 if external_base { base.saturating_mul(4) } else { base }
217}
218
219/// Is this object worth a slice of the quota?
220pub fn worth_caching(depth: u32, inflated: u64, external_base: bool) -> bool {
221 inflated <= MAX_CACHED_BYTES && (external_base || depth > CHEAP_DEPTH)
222}
223
224/// Environment door for the budget. Named in [`crate::arms::ENV_EXPLODE_BUDGET`]
225/// and read through that crate's one counted door, so
226/// `every_environment_read_in_this_crate_is_named_in_all_env` can see it —
227/// a key spelled locally is a key that roster cannot check.
228pub use crate::arms::ENV_EXPLODE_BUDGET as ENV_BUDGET;
229
230/// **The default budget, and the arithmetic that makes it safe.**
231///
232/// `min(10 % of the filesystem holding the store, 8 GiB)` per repository.
233///
234/// On the shipped appliance — 300 GB `/data`, the shape the owner named — 10 %
235/// is 30 GB, so the 8 GiB ceiling binds and every repository is capped at
236/// 8 GiB. **Twenty** repositories of any size therefore cost at most 160 GB of
237/// exploded cache against 300 GB, leaving the packs themselves (the truth,
238/// which is never evictable) room; and a repository only reaches its cap if
239/// somebody actually reads all of it. The same appliance with the eager tier
240/// took 141.7 GB for **one** repository.
241///
242/// On a small box the percentage binds instead: a 64 GiB `/data` gives 6.4 GiB,
243/// a 20 GiB one gives 2 GiB. The floor is [`MIN_BUDGET`] so a tiny volume still
244/// gets a cache rather than thrashing at zero.
245pub const BUDGET_FRACTION_PERCENT: u64 = 10;
246pub const BUDGET_CEILING: u64 = 8 << 30;
247pub const MIN_BUDGET: u64 = 64 << 20;
248
249/// The default budget for a store rooted at `dir`, from that filesystem's size.
250pub fn default_budget(dir: &Path) -> u64 {
251 let total = filesystem_bytes(dir).unwrap_or(0);
252 if total == 0 {
253 return BUDGET_CEILING;
254 }
255 (total / 100 * BUDGET_FRACTION_PERCENT)
256 .min(BUDGET_CEILING)
257 .max(MIN_BUDGET)
258}
259
260/// The process-wide line for a store rooted at `dir`: the same tenth of the
261/// filesystem as [`default_budget`], **uncapped** — the cap is per store, the
262/// tenth is per box. Every cache on one filesystem computes the same figure;
263/// caches on different filesystems each hold the sum to their own, which
264/// over-counts across mounts and is the conservative direction. `0` (no
265/// filesystem figure) turns the global check off rather than inventing one.
266pub fn default_global_line(dir: &Path) -> u64 {
267 filesystem_bytes(dir).unwrap_or(0) / 100 * BUDGET_FRACTION_PERCENT
268}
269
270/// **The disk line**: once the filesystem holding a store is this full, the
271/// lazy pool gives its OLDEST segments back, whatever its own budget says.
272///
273/// Rickard, 2026-09-28: *"add a deletion of the old objects in case 80 % disk
274/// is taken"*. The budget and the global line are both fractions of the
275/// filesystem's SIZE, so a disk filled by packs — the truth, never evictable —
276/// never moved either of them: the cache kept its full 10 % while pushes were
277/// refused at the floor. 80 % is also where the appliance's grow batcher
278/// queues a growth (monetize `grow_high_water_percent`), so the cache is
279/// handing space back at the moment the disk is asked to grow, not after.
280///
281/// Oldest first, not cheapest first: under pressure the question is "what has
282/// nobody needed for longest", and the rebuild cost of an object nobody reads
283/// is paid by nobody. The eager pool is still never touched. `0` = off.
284pub const DISK_LINE_PERCENT: u64 = 80;
285/// **Hysteresis under the disk line** (owner 2026-09-28): once the line trips,
286/// the pressure holds until the disk is this many points BELOW it (80 → 75).
287/// Without it a disk that stays over 80 % because of packs — not cache — made
288/// every tick drop every lazy segment but the live one, in every store, while
289/// write-back kept refilling them: the cache rewrote and threw away up to a
290/// segment per store per tick.
291pub const DISK_LINE_HYSTERESIS_PERCENT: u64 = 5;
292
293/// How full the filesystem holding `dir` is, in percent, the way `df` counts
294/// it: used / (used + available to an unprivileged writer). `None` when
295/// `statvfs` has no answer — and then the disk line does nothing, because a
296/// reading that did not happen is not a full disk.
297/// Pure: the latched pressure after a reading of `used` % against `line` %.
298/// It trips at `used >= line` and releases only at
299/// `used < line - DISK_LINE_HYSTERESIS_PERCENT`; in between it keeps `was`.
300pub fn pressure_after(was: bool, used: u64, line: u64) -> bool {
301 if used >= line {
302 true
303 } else if used < line.saturating_sub(DISK_LINE_HYSTERESIS_PERCENT) {
304 false
305 } else {
306 was
307 }
308}
309
310pub fn disk_used_percent(dir: &Path) -> Option<u64> {
311 use std::os::unix::ffi::OsStrExt;
312 let mut c = Vec::with_capacity(dir.as_os_str().len() + 1);
313 c.extend_from_slice(dir.as_os_str().as_bytes());
314 c.push(0);
315 let mut s: libc::statvfs = unsafe { std::mem::zeroed() };
316 let rc = unsafe { libc::statvfs(c.as_ptr().cast(), &mut s) };
317 if rc != 0 {
318 return None;
319 }
320 let used = (s.f_blocks as u64).saturating_sub(s.f_bfree as u64);
321 let seen = used + s.f_bavail as u64;
322 (seen > 0).then(|| used * 100 / seen)
323}
324
325/// Total bytes of the filesystem holding `dir`, via `statvfs`.
326fn filesystem_bytes(dir: &Path) -> Option<u64> {
327 use std::os::unix::ffi::OsStrExt;
328 let mut c = Vec::with_capacity(dir.as_os_str().len() + 1);
329 c.extend_from_slice(dir.as_os_str().as_bytes());
330 c.push(0);
331 let mut s: libc::statvfs = unsafe { std::mem::zeroed() };
332 let rc = unsafe { libc::statvfs(c.as_ptr().cast(), &mut s) };
333 (rc == 0).then(|| s.f_blocks as u64 * s.f_frsize as u64)
334}
335
336/// The budget a store opens with: the environment door if it names one, the
337/// filesystem-derived default otherwise.
338pub fn budget_from_env(dir: &Path) -> Result<u64> {
339 match crate::arms::read_env(crate::arms::ENV_EXPLODE_BUDGET) {
340 Some(v) => parse_budget(&v),
341 None => Ok(budget_default().unwrap_or_else(|| default_budget(dir))),
342 }
343}
344
345/// `u64::MAX` is "no embedder default" and not a budget: `0` already MEANS
346/// unlimited ([`parse_budget`]), so the sentinel cannot be zero without making
347/// "unbounded" unsayable from code.
348const NO_BUDGET_DEFAULT: u64 = u64::MAX;
349
350/// What [`budget_from_env`] answers when [`crate::arms::ENV_EXPLODE_BUDGET`] is unset:
351/// the embedder's figure, or the filesystem-derived [`default_budget`] when it set none.
352static BUDGET_DEFAULT: AtomicU64 = AtomicU64::new(NO_BUDGET_DEFAULT);
353
354/// The embedder's process-wide budget, if one was set.
355pub fn budget_default() -> Option<u64> {
356 match BUDGET_DEFAULT.load(Ordering::Relaxed) {
357 NO_BUDGET_DEFAULT => None,
358 n => Some(n),
359 }
360}
361
362/// **Set the process-wide default explode budget** for every store opened AFTER this call
363/// (an explicit [`crate::arms::ENV_EXPLODE_BUDGET`] still wins). `None` restores the
364/// filesystem-derived [`default_budget`]. The twin of
365/// [`crate::arms::set_graph_cache_default`], for an embedder that learns its role at runtime
366/// and cannot set the environment — edition 2024 makes `set_var` unsafe, and gunnar forbids
367/// `unsafe` — such as gunnar's twin, which caps the pool at 16 MiB so a later policy change
368/// cannot walk back over its measured memory bound.
369pub fn set_explode_budget_default(bytes: Option<u64>) {
370 BUDGET_DEFAULT.store(bytes.unwrap_or(NO_BUDGET_DEFAULT), Ordering::Relaxed);
371}
372
373/// `12G`, `512M`, `1048576`, `0`, `unlimited`. An unparseable value is an
374/// ERROR and never a silent default — a typo in a quota is a bill that is
375/// wrong, and nobody notices until it is.
376pub fn parse_budget(s: &str) -> Result<u64> {
377 let t = s.trim().to_ascii_lowercase();
378 if t == "unlimited" || t == "none" || t == "off" {
379 return Ok(0);
380 }
381 let (digits, mult) = match t.as_bytes().last() {
382 Some(b'k') => (&t[..t.len() - 1], 1u64 << 10),
383 Some(b'm') => (&t[..t.len() - 1], 1u64 << 20),
384 Some(b'g') => (&t[..t.len() - 1], 1u64 << 30),
385 Some(b't') => (&t[..t.len() - 1], 1u64 << 40),
386 _ => (t.as_str(), 1),
387 };
388 let n: u64 = digits
389 .trim()
390 .parse()
391 .map_err(|_| anyhow::anyhow!("unreadable explode budget {s:?}; expected 0, 512M, 8G, …"))?;
392 n.checked_mul(mult)
393 .ok_or_else(|| anyhow::anyhow!("explode budget {s:?} overflows"))
394}
395
396/// What the tier is holding and what it is allowed to hold — the row a
397/// dashboard prints and `Admin.StorageInfo` carries.
398#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
399pub struct CacheUse {
400 /// Bytes in the eager pool (commits and tags under the shipped policy).
401 pub eager_bytes: u64,
402 /// Bytes in the lazy pool, across every live segment.
403 pub lazy_bytes: u64,
404 /// The cap. `0` is unlimited.
405 pub budget: u64,
406 /// Live lazy segments.
407 pub segments: u64,
408 /// Segments unlinked since this store was opened.
409 pub evicted_segments: u64,
410 /// Bytes those segments held. **Applied output**: what actually left the
411 /// filesystem, measured before the unlink, not a projection.
412 pub evicted_bytes: u64,
413 /// Rows those segments held.
414 pub evicted_rows: u64,
415 /// Objects written back by a first read.
416 pub written_back: u64,
417 /// **Reads served by inflating the object directly out of the pack and
418 /// caching NOTHING** — the cheap ones the admission policy declined.
419 /// Applied output: the counter says the policy ran, which a size alone
420 /// could not.
421 pub direct_reads: u64,
422 /// **Write-backs declined because the caller declared the read a
423 /// validation read** ([`suppress_write_back`]) — a `receive-pack`'s
424 /// connectivity walk, not a reader.
425 ///
426 /// Applied output, and the only way to tell the suppression ran from a
427 /// store that simply had nothing to cache: a pool that stays empty proves
428 /// nothing on its own, and this counter moving proves the gate fired.
429 pub suppressed_write_backs: u64,
430}
431
432impl CacheUse {
433 pub fn used(&self) -> u64 {
434 self.eager_bytes + self.lazy_bytes
435 }
436 /// `3.1G/8G` — what the dashboard cell prints.
437 pub fn band(&self) -> String {
438 if self.budget == 0 {
439 format!("{}/unlimited", human(self.used()))
440 } else {
441 format!("{}/{}", human(self.used()), human(self.budget))
442 }
443 }
444 /// Over budget right now — true only while the eager pool alone exceeds it,
445 /// because the lazy pool is evicted back under after every rotation.
446 pub fn over(&self) -> bool {
447 self.budget != 0 && self.used() > self.budget
448 }
449}
450
451/// `3.1G`, `812M`, `40K` — one significant fraction, the same shape the
452/// dashboard's other byte cells use.
453pub fn human(bytes: u64) -> String {
454 const UNITS: [(u64, &str); 4] = [(1 << 40, "T"), (1 << 30, "G"), (1 << 20, "M"), (1 << 10, "K")];
455 for (scale, suffix) in UNITS {
456 if bytes >= scale {
457 let v = bytes as f64 / scale as f64;
458 return if v < 10.0 {
459 format!("{v:.1}{suffix}")
460 } else {
461 format!("{v:.0}{suffix}")
462 };
463 }
464 }
465 format!("{bytes}B")
466}
467
468/// What one live lazy segment is holding, for cost-weighted eviction.
469#[derive(Debug, Clone, Copy, Default)]
470struct SegStat {
471 rows: u64,
472 /// Summed [`rebuild_cost`] of the rows written into it.
473 cost: u64,
474 /// Monotonic tick of the last write into it — the recency half of the rank.
475 touched: u64,
476}
477
478/// **The two pools, one door.** Everything that used to call an
479/// [`ExplodedArchive`] calls this instead; the eager pool *is* an
480/// `ExplodedArchive` and behaves exactly as it did.
481pub struct ExplodedCache {
482 eager: ExplodedArchive,
483 /// Oldest first. `Mutex` and not `RwLock`: it is touched once per rotation
484 /// and once per read miss, never per object.
485 lazy: Mutex<Vec<std::sync::Arc<ExplodedArchive>>>,
486 base: PathBuf,
487 policy: ExplodePolicy,
488 budget: AtomicU64,
489 /// **The process-wide line**, in bytes, that the SUM of every registered
490 /// cache's disk use is held under; `0` = no global line. `budget` above is
491 /// per store — 10 % of the filesystem, capped at 8 GiB — and MEASURED on
492 /// 2026-09-05 that meant N repositories read hard could hold N × 8 GiB:
493 /// ten busy tenants on a 1 TB appliance, 80 GB of cache. This is the same
494 /// 10 % of the filesystem, uncapped, applied once to the box: a cache whose
495 /// own use is above its fair share of it (`line / live caches`) evicts its
496 /// cheapest segment on its tick even while under its own budget, so the
497 /// biggest pools shrink first and the sum converges under the line.
498 global_line: AtomicU64,
499 /// [`DISK_LINE_PERCENT`], changeable live; `0` = off.
500 disk_line_percent: AtomicU64,
501 /// Latched disk pressure: set at the line, cleared only below
502 /// `line − DISK_LINE_HYSTERESIS_PERCENT`. While set, eviction runs
503 /// oldest-first AND write-back admits nothing (see [`Self::write_back`]).
504 disk_pressure: std::sync::atomic::AtomicBool,
505 /// Write-backs refused because the disk line was latched.
506 pressure_refused_write_backs: AtomicU64,
507 /// A test's disk reading in place of `statvfs`; [`Self::NO_DISK_READING`] = read
508 /// the real filesystem. Starts at `0` under test (see `open`).
509 #[cfg(test)]
510 disk_used_override: AtomicU64,
511 segment_bytes: AtomicU64,
512 next_seg: AtomicU64,
513 written_back: AtomicU64,
514 direct_reads: AtomicU64,
515 /// See [`CacheUse::suppressed_write_backs`].
516 suppressed_write_backs: AtomicU64,
517 /// Per live segment, parallel to `lazy`: rows, and the summed rebuild cost
518 /// of what is in it. What cost-weighted eviction ranks on.
519 seg_cost: Mutex<Vec<SegStat>>,
520 evicted_segments: AtomicU64,
521 evicted_bytes: AtomicU64,
522 evicted_rows: AtomicU64,
523 /// Bytes buffered in the newest lazy segment since its last flush. The
524 /// flusher reads it; a reader adds to it and never blocks on it.
525 dirty: AtomicU64,
526 /// Where this cache's store lives, for the registry's rows.
527 root: String,
528 /// A maintenance thread is running. **The whole of "nobody waits"**: the
529 /// reader that fills the buffer starts one and walks away, and a second
530 /// reader that arrives while it runs does nothing at all.
531 maintaining: std::sync::atomic::AtomicBool,
532}
533
534impl ExplodedCache {
535 /// "No test reading": the disk line reads the real filesystem.
536 #[cfg(test)]
537 const NO_DISK_READING: u64 = u64::MAX;
538
539 /// Open both pools at `base` (`…/objects.exploded`), adopting whatever
540 /// lazy segments a previous process left beside it.
541 pub fn open(base: &Path, policy: ExplodePolicy, budget: u64) -> Result<Self> {
542 let eager = ExplodedArchive::open_with_policy(base, policy)?;
543 let mut found: Vec<(u64, PathBuf)> = Vec::new();
544 if let Some(dir) = base.parent()
545 && let Ok(rd) = std::fs::read_dir(dir)
546 {
547 let stem = base
548 .file_name()
549 .map(|s| s.to_string_lossy().into_owned())
550 .unwrap_or_default();
551 let prefix = format!("{stem}.c");
552 for e in rd.flatten() {
553 let name = e.file_name().to_string_lossy().into_owned();
554 if let Some(n) = name.strip_prefix(&prefix)
555 && let Ok(n) = n.parse::<u64>()
556 {
557 found.push((n, e.path()));
558 }
559 }
560 }
561 found.sort_unstable_by_key(|(n, _)| *n);
562 let next = found.last().map(|(n, _)| n + 1).unwrap_or(1);
563 // A segment adopted from a previous process has no cost history — the
564 // scores are per-process and are not written down. It is scored by its
565 // position instead, oldest first, which is what plain FIFO would do
566 // and is the honest reading of "we do not know".
567 let seg_stats: Vec<SegStat> = (0..found.len())
568 .map(|i| SegStat { rows: 0, cost: 0, touched: i as u64 })
569 .collect();
570 let mut lazy = Vec::with_capacity(found.len());
571 for (_, p) in found {
572 // The lazy pool takes EVERY kind: it is where an object the eager
573 // policy declined comes to rest, so its own policy is `Full`.
574 lazy.push(std::sync::Arc::new(ExplodedArchive::open_with_policy(
575 &p,
576 ExplodePolicy::Full,
577 )?));
578 }
579 Ok(Self {
580 eager,
581 lazy: Mutex::new(lazy),
582 base: base.to_path_buf(),
583 policy,
584 budget: AtomicU64::new(budget),
585 global_line: AtomicU64::new(default_global_line(base)),
586 disk_line_percent: AtomicU64::new(DISK_LINE_PERCENT),
587 disk_pressure: std::sync::atomic::AtomicBool::new(false),
588 pressure_refused_write_backs: AtomicU64::new(0),
589 // Hermetic by default: a test box at 85 % would otherwise run every
590 // eviction test in this crate — and every `GitStore` test — under the
591 // disk line, turning "cheapest first" into "oldest first" and a
592 // store's lazy pool into one segment, depending on whose disk it is.
593 // A test that wants the real `statvfs` sets `NO_DISK_READING`.
594 #[cfg(test)]
595 disk_used_override: AtomicU64::new(0),
596 segment_bytes: AtomicU64::new(segment_for(budget)),
597 next_seg: AtomicU64::new(next),
598 written_back: AtomicU64::new(0),
599 direct_reads: AtomicU64::new(0),
600 suppressed_write_backs: AtomicU64::new(0),
601 seg_cost: Mutex::new(seg_stats),
602 evicted_segments: AtomicU64::new(0),
603 evicted_bytes: AtomicU64::new(0),
604 evicted_rows: AtomicU64::new(0),
605 dirty: AtomicU64::new(0),
606 root: base.parent().unwrap_or(base).display().to_string(),
607 maintaining: std::sync::atomic::AtomicBool::new(false),
608 })
609 }
610
611 /// Register this cache so a dashboard tick can read every open store's
612 /// band without holding a store lock — the same `Weak` registry
613 /// [`crate::progress`] keeps, for the same reason and with the same sweep.
614 pub fn registered(self: std::sync::Arc<Self>) -> std::sync::Arc<Self> {
615 registry()
616 .lock()
617 .unwrap_or_else(|e| e.into_inner())
618 .push(std::sync::Arc::downgrade(&self));
619 self
620 }
621
622 pub fn root(&self) -> &str {
623 &self.root
624 }
625
626 /// The quota, changed live — what the control plane sets when an account's
627 /// plan changes. Takes effect at the next rotation; it never refuses a
628 /// write and never deletes on the caller's thread.
629 pub fn set_budget(&self, bytes: u64) {
630 self.budget.store(bytes, Ordering::Release);
631 self.segment_bytes.store(segment_for(bytes), Ordering::Release);
632 }
633
634 /// The process-wide line this cache holds itself to its fair share of.
635 /// `0` turns the global check off. Tests set it small; production takes
636 /// [`default_global_line`] at open.
637 pub fn set_global_line(&self, bytes: u64) {
638 self.global_line.store(bytes, Ordering::Release);
639 }
640
641 /// The disk line, changed live. `0` turns it off. Takes effect at the next
642 /// rotation, like the budget.
643 pub fn set_disk_line_percent(&self, percent: u64) {
644 self.disk_line_percent.store(percent, Ordering::Release);
645 }
646
647 /// A test's disk reading, in percent, in place of `statvfs`.
648 #[cfg(test)]
649 pub(crate) fn set_disk_reading_for_test(&self, percent: u64) {
650 self.disk_used_override.store(percent, Ordering::Release);
651 }
652
653 /// A test's segment size, so a few write-backs rotate into several segments.
654 #[cfg(test)]
655 pub(crate) fn set_segment_bytes_for_test(&self, bytes: u64) {
656 self.segment_bytes.store(bytes, Ordering::Release);
657 }
658
659 /// True while the filesystem holding this store is at or past the disk
660 /// line. Read afresh per eviction, because each unlink moves it.
661 fn over_disk_line(&self) -> bool {
662 let line = self.disk_line_percent.load(Ordering::Acquire);
663 if line == 0 {
664 self.disk_pressure.store(false, Ordering::Release);
665 return false;
666 }
667 let used = self.disk_used_reading();
668 let Some(used) = used else {
669 // No reading: keep whatever was latched — a statvfs that fails
670 // once must neither start nor end a pressure episode.
671 return self.disk_pressure.load(Ordering::Acquire);
672 };
673 let latched = pressure_after(self.disk_pressure.load(Ordering::Acquire), used, line);
674 self.disk_pressure.store(latched, Ordering::Release);
675 latched
676 }
677
678 /// The disk's used percent: the test override, else `statvfs` on the pool.
679 fn disk_used_reading(&self) -> Option<u64> {
680 #[cfg(test)]
681 {
682 let o = self.disk_used_override.load(Ordering::Acquire);
683 if o != Self::NO_DISK_READING {
684 return Some(o);
685 }
686 }
687 disk_used_percent(self.pool_dir())
688 }
689
690 /// Write-backs refused while the disk line was latched.
691 pub fn pressure_refused_write_backs(&self) -> u64 {
692 self.pressure_refused_write_backs.load(Ordering::Relaxed)
693 }
694
695 /// The directory both pools live in: the eager file is `base` and every lazy
696 /// segment is `base.cN` beside it, so this is the filesystem an unlink here
697 /// gives back to. Not `base` itself: the eager file does not exist until the
698 /// first commit is flushed, `statvfs` on a missing path is `ENOENT`, and a
699 /// disk line read through it was silently off for a store that had not yet
700 /// indexed a commit.
701 fn pool_dir(&self) -> &Path {
702 match self.base.parent() {
703 Some(p) if !p.as_os_str().is_empty() => p,
704 _ => Path::new("."),
705 }
706 }
707
708 /// A lazy segment's read, where an error from a segment an eviction has
709 /// already retired is a MISS. The segments are cloned out of the list and
710 /// read with the lock released, and a segment opens its file by path per
711 /// read (`File::open` in `ExplodedArchive::read_batch`/`content`), so an
712 /// eviction between the clone and the open made the read `ENOENT` — which
713 /// `?` turned into an error for a reader, or into a failed absorb through
714 /// `refold`'s `of_kind`. A retired segment is by definition not in the
715 /// cache any more, so "absent" is the true answer: fall back and re-derive.
716 fn lazy_read<T>(seg: &ExplodedArchive, r: Result<T>) -> Result<Option<T>> {
717 match r {
718 Ok(v) => Ok(Some(v)),
719 Err(_) if seg.is_retired() => Ok(None),
720 Err(e) => Err(e),
721 }
722 }
723
724 /// True when the registered caches together exceed the global line AND this
725 /// cache holds more than its fair share of it — the condition under which
726 /// this cache evicts for the box rather than for itself. Computed without
727 /// holding this cache's own lock, so the registry sweep can lock the others.
728 fn over_global_share(&self) -> bool {
729 let line = self.global_line.load(Ordering::Acquire);
730 if line == 0 {
731 return false;
732 }
733 let all = use_all();
734 // The share is among caches that HOLD something: an idle repository's
735 // empty pool must not dilute the line for the ones actually using it.
736 let live = all
737 .iter()
738 .filter(|(_, u)| u.eager_bytes + u.lazy_bytes > 0)
739 .count()
740 .max(1) as u64;
741 let sum: u64 = all.iter().map(|(_, u)| u.eager_bytes + u.lazy_bytes).sum();
742 if sum <= line {
743 return false;
744 }
745 let mine = self.use_now();
746 mine.eager_bytes + mine.lazy_bytes > line / live
747 }
748
749 pub fn budget(&self) -> u64 {
750 self.budget.load(Ordering::Acquire)
751 }
752
753 pub fn policy(&self) -> ExplodePolicy {
754 self.policy
755 }
756
757 pub fn path(&self) -> &Path {
758 &self.base
759 }
760
761 /// **The eager write** — the indexer's sink, at push time. Unchanged
762 /// behaviour: the policy decides, and what it declines is counted as
763 /// skipped rather than silently dropped.
764 pub fn explode(&self, oid: &[u8], kind: GitObjectKind, payload: &[u8]) -> Result<()> {
765 self.eager.explode(oid, kind, payload)
766 }
767
768 pub fn explode_at(
769 &self,
770 oid: &[u8],
771 kind: GitObjectKind,
772 mode: Option<u32>,
773 path: Option<&str>,
774 payload: &[u8],
775 ) -> Result<()> {
776 self.eager.explode_at(oid, kind, mode, path, payload)
777 }
778
779 /// **The lazy write-back** — one object a reader has just inflated from the
780 /// pack, kept for the next reader.
781 ///
782 /// Buffers only: it appends to the newest segment's in-memory rows and
783 /// returns. Nothing is written, nothing is fsynced, no eviction runs. The
784 /// reader that called it has its bytes already and is not made to wait for
785 /// anybody — [`Self::maintain`] does the I/O on the store's own thread.
786 /// Two readers racing on one oid is benign: the same bytes are buffered
787 /// twice and the newest row wins, which is what
788 /// [`ExplodedArchive::explode_at`] already promises.
789 pub fn write_back(
790 &self,
791 oid: &[u8],
792 kind: GitObjectKind,
793 payload: &[u8],
794 cost: crate::resolve::ReadCost,
795 ) -> Result<bool> {
796 // **A validation read is not a reader.** A `receive-pack` reads every
797 // commit and tree it was sent, once, to prove the push closes — and
798 // then never reads them again. Caching those is writing a read-cache
799 // for a reader that does not exist. MEASURED on torvalds/linux
800 // (2026-09-03, oden): the lazy pool is EMPTY until the instant indexing
801 // completes and then grows to **6 778 047 872 B** before the push
802 // returns, which is 31 % of the whole store and the largest non-pack
803 // thing in it. See `gunnar/.nornir/storage-levers-2026-09-03.md`.
804 //
805 // The caller declares the window ([`suppress_write_back`]) because the
806 // caller is the only thing that knows what kind of read this is: the
807 // store sees the same `read` either way.
808 if write_back_suppressed() {
809 self.suppressed_write_backs.fetch_add(1, Ordering::Relaxed);
810 return Ok(false);
811 }
812 // **No refill under disk pressure.** Eviction is dropping the oldest
813 // segments to give the disk back; caching now would write what the next
814 // tick throws away. The read itself is served; only the copy is skipped.
815 if self.over_disk_line() {
816 self.pressure_refused_write_backs.fetch_add(1, Ordering::Relaxed);
817 return Ok(false);
818 }
819 // **The admission gate.** A shallow, small object is cheaper to inflate
820 // again than to keep: caching it would spend somebody's quota to save
821 // microseconds, and evict something that costs milliseconds.
822 if !worth_caching(cost.depth, cost.inflated, cost.external_base) {
823 self.direct_reads.fetch_add(1, Ordering::Relaxed);
824 return Ok(false);
825 }
826 let seg = {
827 let mut l = self
828 .lazy
829 .lock()
830 .map_err(|_| anyhow::anyhow!("exploded cache poisoned"))?;
831 match l.last() {
832 Some(s) => s.clone(),
833 None => {
834 let s = self.new_segment()?;
835 l.push(s.clone());
836 s
837 }
838 }
839 };
840 seg.explode(oid, kind, payload)?;
841 {
842 let tick = self.written_back.load(Ordering::Relaxed);
843 let mut st = self
844 .seg_cost
845 .lock()
846 .map_err(|_| anyhow::anyhow!("exploded cache poisoned"))?;
847 if st.is_empty() {
848 st.push(SegStat::default());
849 }
850 if let Some(last) = st.last_mut() {
851 last.rows += 1;
852 last.cost += rebuild_cost(cost.depth, cost.inflated, cost.external_base);
853 last.touched = tick;
854 }
855 }
856 self.written_back.fetch_add(1, Ordering::Relaxed);
857 let n = self
858 .dirty
859 .fetch_add(payload.len() as u64, Ordering::AcqRel)
860 + payload.len() as u64;
861 Ok(n >= CACHE_FLUSH_BYTES)
862 }
863
864 /// **Write down what is buffered, rotate if the newest segment is full, and
865 /// evict until the pools fit the budget.** The whole of the tier's I/O, on
866 /// one thread that is nobody's reader.
867 ///
868 /// Returns the bytes reclaimed, so a caller can log applied output rather
869 /// than an intention.
870 pub fn maintain(&self) -> Result<u64> {
871 let newest = {
872 let l = self
873 .lazy
874 .lock()
875 .map_err(|_| anyhow::anyhow!("exploded cache poisoned"))?;
876 l.last().cloned()
877 };
878 if let Some(seg) = newest {
879 seg.flush()?;
880 self.dirty.store(0, Ordering::Release);
881 if seg.disk_bytes() >= self.segment_bytes.load(Ordering::Acquire) {
882 let fresh = self.new_segment()?;
883 let mut l = self
884 .lazy
885 .lock()
886 .map_err(|_| anyhow::anyhow!("exploded cache poisoned"))?;
887 l.push(fresh);
888 if let Ok(mut st) = self.seg_cost.lock() {
889 st.push(SegStat {
890 rows: 0,
891 cost: 0,
892 touched: self.written_back.load(Ordering::Relaxed),
893 });
894 }
895 }
896 }
897 self.evict_to_budget()
898 }
899
900 /// Drop segments until the two pools fit, **cheapest-to-rebuild first**.
901 ///
902 /// Not plain FIFO. A segment is ranked by
903 /// `mean rebuild cost × recency`, and the lowest score goes: a segment of
904 /// shallow 20 KiB blobs is worth less than one of depth-40 chains even if
905 /// it was written more recently, because re-reading the first costs
906 /// microseconds an object and the second costs milliseconds. Recency is
907 /// still half the rank, so a cold segment of expensive objects does
908 /// eventually go rather than pinning the pool.
909 ///
910 /// **The eager pool is never touched**: commits are what `graph`, `reach`
911 /// and a tag peel read, and a store that answers those slowly is worse than
912 /// one that reads a blob slowly. If the eager pool alone is over budget the
913 /// lazy pool goes to nothing and the store is reported `over`.
914 fn evict_to_budget(&self) -> Result<u64> {
915 let budget = self.budget.load(Ordering::Acquire);
916 if budget == 0
917 && self.global_line.load(Ordering::Acquire) == 0
918 && self.disk_line_percent.load(Ordering::Acquire) == 0
919 {
920 return Ok(0);
921 }
922 let eager = self.eager.disk_bytes();
923 let mut freed = 0u64;
924 loop {
925 // Re-read per iteration: each eviction moves this cache's share.
926 let over_share = self.over_global_share();
927 let over_disk = self.over_disk_line();
928 let (used, victim) = {
929 let l = self
930 .lazy
931 .lock()
932 .map_err(|_| anyhow::anyhow!("exploded cache poisoned"))?;
933 let lazy: u64 = l.iter().map(|s| s.disk_bytes()).sum();
934 // The newest segment is the one being written into; evicting it
935 // would drop rows a reader just paid for and start the same
936 // segment again. There is always at least one live segment.
937 // Within its own budget (`0` = unlimited) AND not over its
938 // share of the box AND the disk under its line: nothing to do.
939 // Over the disk line the OLDEST segment goes; otherwise the
940 // cheapest, for whichever reason applies.
941 let within_own = budget == 0 || eager + lazy <= budget;
942 if (within_own && !over_share && !over_disk) || l.len() < 2 {
943 (eager + lazy, None)
944 } else {
945 let at = if over_disk { 0 } else { self.cheapest_segment(l.len() - 1) };
946 (eager + lazy, Some(l[at].clone()))
947 }
948 };
949 let Some(victim) = victim else {
950 let _ = used;
951 break;
952 };
953 let bytes = victim.disk_bytes();
954 let rows = victim.rows().unwrap_or(0);
955 let path = victim.path().to_path_buf();
956 {
957 let mut l = self
958 .lazy
959 .lock()
960 .map_err(|_| anyhow::anyhow!("exploded cache poisoned"))?;
961 // By identity, not by the index read under the first lock: a
962 // second evictor (a `maintain` called directly by `gc` or a test
963 // while a kicked thread runs) may have removed a segment in
964 // between, and `remove(at)` would then drop the WRONG one from
965 // the list — never unlinked, never counted, a leak until restart.
966 let Some(now) = l.iter().position(|s| std::sync::Arc::ptr_eq(s, &victim)) else {
967 // The other evictor took this one; look again.
968 continue;
969 };
970 l.remove(now);
971 if let Ok(mut st) = self.seg_cost.lock()
972 && now < st.len()
973 {
974 st.remove(now);
975 }
976 }
977 // Out of the list, and now closed to writers BEFORE the unlink, so
978 // no holder of an older `Arc` can re-create the path.
979 victim.retire();
980 drop(victim);
981 // The file goes even if some reader still holds an `Arc` on the
982 // segment: an unlinked file keeps answering the handles that are
983 // open, and the blocks come back when the last one closes. So the
984 // budget is honoured immediately and no read in flight fails.
985 std::fs::remove_file(&path).ok();
986 if over_disk {
987 tracing::info!(target: "znippy.git", root = %self.root, bytes, "explode cache: oldest segment dropped, disk at or past its line");
988 }
989 self.evicted_segments.fetch_add(1, Ordering::Relaxed);
990 self.evicted_bytes.fetch_add(bytes, Ordering::Relaxed);
991 self.evicted_rows.fetch_add(rows, Ordering::Relaxed);
992 freed += bytes;
993 }
994 Ok(freed)
995 }
996
997 /// The index of the segment with the lowest `mean rebuild cost × recency`,
998 /// searched over `[0, exclusive)` so the segment being written into is
999 /// never a candidate. Ties and unknown costs (a segment adopted from a
1000 /// previous process, which carries no history) fall back to the oldest,
1001 /// which is what plain FIFO would have chosen.
1002 fn cheapest_segment(&self, exclusive: usize) -> usize {
1003 let Ok(st) = self.seg_cost.lock() else {
1004 return 0;
1005 };
1006 let newest = st.last().map(|s| s.touched).unwrap_or(0).max(1);
1007 let mut best = 0usize;
1008 let mut best_score = u128::MAX;
1009 for i in 0..exclusive.min(st.len()) {
1010 let s = st[i];
1011 let mean = if s.rows == 0 { 0 } else { s.cost / s.rows };
1012 // Recency in [1, 1024]: a segment written at the very start scores
1013 // 1, the newest scores 1024. Multiplying rather than adding keeps
1014 // the two halves on one scale without a weight nobody can justify.
1015 let recency = 1 + (s.touched as u128 * 1023) / newest as u128;
1016 let score = (mean as u128 + 1) * recency;
1017 if score < best_score {
1018 best_score = score;
1019 best = i;
1020 }
1021 }
1022 best
1023 }
1024
1025 fn new_segment(&self) -> Result<std::sync::Arc<ExplodedArchive>> {
1026 let n = self.next_seg.fetch_add(1, Ordering::AcqRel);
1027 let name = format!(
1028 "{}.c{n}",
1029 self.base.file_name().unwrap_or_default().to_string_lossy()
1030 );
1031 let path = self.base.with_file_name(name);
1032 Ok(std::sync::Arc::new(ExplodedArchive::open_with_policy(
1033 &path,
1034 ExplodePolicy::Full,
1035 )?))
1036 }
1037
1038 /// The object's kind and bytes from either pool, or `None` — *fall back and
1039 /// re-derive*, never *wrong*.
1040 ///
1041 /// Eager first: it is the smaller table, its index is the one already warm,
1042 /// and under the shipped policy it holds exactly the objects a graph walk
1043 /// asks for. Then the lazy segments **newest first**, so a re-entered
1044 /// object is found before the stale copy an older segment may still hold.
1045 pub fn content(&self, oid: &[u8]) -> Result<Option<(GitObjectKind, Vec<u8>)>> {
1046 if let Some(hit) = self.eager.content(oid)? {
1047 return Ok(Some(hit));
1048 }
1049 let segs = self.segments()?;
1050 for s in segs.iter().rev() {
1051 if let Some(Some(hit)) = Self::lazy_read(s, s.content(oid))? {
1052 return Ok(Some(hit));
1053 }
1054 }
1055 Ok(None)
1056 }
1057
1058 pub fn has(&self, oid: &[u8]) -> Result<bool> {
1059 if self.eager.has(oid)? {
1060 return Ok(true);
1061 }
1062 for s in self.segments()?.iter().rev() {
1063 if Self::lazy_read(s, s.has(oid))? == Some(true) {
1064 return Ok(true);
1065 }
1066 }
1067 Ok(false)
1068 }
1069
1070 pub fn name(&self, oid: &[u8]) -> Result<Option<(Option<u32>, Option<String>)>> {
1071 if let Some(n) = self.eager.name(oid)? {
1072 return Ok(Some(n));
1073 }
1074 for s in self.segments()?.iter().rev() {
1075 if let Some(Some(n)) = Self::lazy_read(s, s.name(oid))? {
1076 return Ok(Some(n));
1077 }
1078 }
1079 Ok(None)
1080 }
1081
1082 /// Every oid of one kind, across both pools, newest spelling winning.
1083 ///
1084 /// The graph fold's input. Under the shipped policy every commit is in the
1085 /// eager pool, so this is one table's scan and the lazy segments add
1086 /// nothing — but it unions rather than assuming that, because a store
1087 /// opened under a different policy must still fold a whole graph.
1088 pub fn of_kind(&self, kind: GitObjectKind) -> Result<Vec<(Vec<u8>, Vec<u8>)>> {
1089 let mut out = self.eager.of_kind(kind)?;
1090 let segs = self.segments()?;
1091 if segs.is_empty() {
1092 return Ok(out);
1093 }
1094 let mut seen: std::collections::HashSet<Vec<u8>> =
1095 out.iter().map(|(o, _)| o.clone()).collect();
1096 for s in segs.iter().rev() {
1097 for (oid, payload) in Self::lazy_read(s, s.of_kind(kind))?.unwrap_or_default() {
1098 if seen.insert(oid.clone()) {
1099 out.push((oid, payload));
1100 }
1101 }
1102 }
1103 Ok(out)
1104 }
1105
1106 /// Retire every oid `live` rejects — in **both** pools.
1107 ///
1108 /// Both, and that is not defensive: a tombstone written only into the eager
1109 /// pool would leave the lazy pool's row live, `content` would fall through
1110 /// to it, and a `gc`'d object would come back by the second door. That is
1111 /// the same defect
1112 /// `a_wholly_dead_pack_does_not_come_back_when_the_store_is_reopened` found
1113 /// the first time, one pool along.
1114 pub fn retain(&self, live: &dyn Fn(&[u8]) -> bool) -> Result<u64> {
1115 let mut n = self.eager.retain(live)?;
1116 for s in self.segments()? {
1117 n += Self::lazy_read(&s, s.retain(live))?.unwrap_or(0);
1118 }
1119 Ok(n)
1120 }
1121
1122 /// **`repo gc` reclaims the cache.** Every lazy segment is unlinked; the
1123 /// eager pool keeps its rows, because they are the graph. Returns the bytes
1124 /// that left the filesystem.
1125 pub fn drop_cache(&self) -> Result<u64> {
1126 let victims = {
1127 let mut l = self
1128 .lazy
1129 .lock()
1130 .map_err(|_| anyhow::anyhow!("exploded cache poisoned"))?;
1131 if let Ok(mut st) = self.seg_cost.lock() {
1132 st.clear();
1133 }
1134 std::mem::take(&mut *l)
1135 };
1136 let mut freed = 0;
1137 for s in victims {
1138 let bytes = s.disk_bytes();
1139 let rows = s.rows().unwrap_or(0);
1140 let path = s.path().to_path_buf();
1141 s.retire();
1142 drop(s);
1143 std::fs::remove_file(&path).ok();
1144 self.evicted_segments.fetch_add(1, Ordering::Relaxed);
1145 self.evicted_bytes.fetch_add(bytes, Ordering::Relaxed);
1146 self.evicted_rows.fetch_add(rows, Ordering::Relaxed);
1147 freed += bytes;
1148 }
1149 self.dirty.store(0, Ordering::Release);
1150 Ok(freed)
1151 }
1152
1153 pub fn flush(&self) -> Result<()> {
1154 self.eager.flush()?;
1155 for s in self.segments()? {
1156 Self::lazy_read(&s, s.flush())?;
1157 }
1158 self.dirty.store(0, Ordering::Release);
1159 Ok(())
1160 }
1161
1162 /// What the memory watchdog reaches for: every derived, rebuildable
1163 /// structure in every pool.
1164 pub fn drop_caches(&self) {
1165 self.eager.drop_caches();
1166 if let Ok(l) = self.lazy.lock() {
1167 for s in l.iter() {
1168 s.drop_caches();
1169 }
1170 }
1171 }
1172
1173 pub fn rows(&self) -> Result<u64> {
1174 let mut n = self.eager.rows()?;
1175 for s in self.segments()? {
1176 n += s.rows()?;
1177 }
1178 Ok(n)
1179 }
1180
1181 /// Rows in the EAGER pool only — what `adopt_journal` compares against the
1182 /// index, because the lazy pool's size is a function of what has been read
1183 /// and says nothing about whether a pack was absorbed.
1184 pub fn eager_rows(&self) -> Result<u64> {
1185 self.eager.rows()
1186 }
1187
1188 pub fn disk_bytes(&self) -> u64 {
1189 self.use_now().used()
1190 }
1191
1192 pub fn use_now(&self) -> CacheUse {
1193 let (lazy_bytes, segments) = match self.lazy.lock() {
1194 Ok(l) => (l.iter().map(|s| s.disk_bytes()).sum(), l.len() as u64),
1195 Err(_) => (0, 0),
1196 };
1197 CacheUse {
1198 eager_bytes: self.eager.disk_bytes(),
1199 lazy_bytes,
1200 budget: self.budget.load(Ordering::Acquire),
1201 segments,
1202 evicted_segments: self.evicted_segments.load(Ordering::Relaxed),
1203 evicted_bytes: self.evicted_bytes.load(Ordering::Relaxed),
1204 evicted_rows: self.evicted_rows.load(Ordering::Relaxed),
1205 written_back: self.written_back.load(Ordering::Relaxed),
1206 direct_reads: self.direct_reads.load(Ordering::Relaxed),
1207 suppressed_write_backs: self.suppressed_write_backs.load(Ordering::Relaxed),
1208 }
1209 }
1210
1211 pub fn note_rederived(&self) {
1212 self.eager.note_rederived();
1213 }
1214
1215 pub fn pread_served(&self) -> u64 {
1216 self.eager.pread_served()
1217 + self
1218 .segments()
1219 .map(|s| s.iter().map(|x| x.pread_served()).sum::<u64>())
1220 .unwrap_or(0)
1221 }
1222
1223 pub fn skipped(&self) -> u64 {
1224 self.eager.skipped()
1225 }
1226
1227 /// The engine-wide counters, summed over both pools, so no caller can tell
1228 /// the two apart by its instruments.
1229 pub fn engine_stats(&self) -> ExplodedStats {
1230 let mut s = self.eager.engine_stats();
1231 if let Ok(segs) = self.segments() {
1232 for x in segs {
1233 let e = x.engine_stats();
1234 s.rows += e.rows;
1235 s.written += e.written;
1236 s.served += e.served;
1237 s.absent += e.absent;
1238 }
1239 }
1240 s
1241 }
1242
1243 /// **Start the write-back thread if one is not already running — and
1244 /// return to the reader immediately either way.**
1245 ///
1246 /// The owner's law, from the design note: *nobody waits*. The reader already
1247 /// has its bytes; the append, the fsync, the rotation and the eviction all
1248 /// happen here, on a thread that is nobody's request. A thread per 4 MiB of
1249 /// write-back rather than a parked thread per store, because a box holding
1250 /// 1 200 open repositories (lane V) cannot afford 1 200 idle threads and
1251 /// can easily afford one spawn per few thousand objects read.
1252 ///
1253 /// The flag is the whole of the mutual exclusion: a second reader finding
1254 /// it set does nothing, and the running thread loops until the buffer is
1255 /// empty before clearing it, so no work is dropped between the two.
1256 pub fn kick(self: &std::sync::Arc<Self>) {
1257 if self
1258 .maintaining
1259 .compare_exchange(false, true, Ordering::AcqRel, Ordering::Acquire)
1260 .is_err()
1261 {
1262 return;
1263 }
1264 let me = self.clone();
1265 let spawned = std::thread::Builder::new()
1266 .name("explode-writeback".into())
1267 .stack_size(256 << 10)
1268 .spawn(move || {
1269 loop {
1270 if let Err(e) = me.maintain() {
1271 tracing::warn!(target: "znippy.git", error = %e, "explode cache maintenance");
1272 break;
1273 }
1274 if me.dirty.load(Ordering::Acquire) < CACHE_FLUSH_BYTES {
1275 break;
1276 }
1277 }
1278 me.maintaining.store(false, Ordering::Release);
1279 });
1280 if spawned.is_err() {
1281 // No thread to be had. Clearing the flag is what makes the next
1282 // reader try again rather than the cache silently stopping.
1283 self.maintaining.store(false, Ordering::Release);
1284 }
1285 }
1286
1287 fn segments(&self) -> Result<Vec<std::sync::Arc<ExplodedArchive>>> {
1288 Ok(self
1289 .lazy
1290 .lock()
1291 .map_err(|_| anyhow::anyhow!("exploded cache poisoned"))?
1292 .clone())
1293 }
1294}
1295
1296impl crate::exploded::PayloadSink for ExplodedCache {
1297 fn wants(&self, kind: GitObjectKind) -> bool {
1298 crate::exploded::PayloadSink::wants(&self.eager, kind)
1299 }
1300 fn explode(&self, oid: &[u8], kind: GitObjectKind, payload: &[u8]) -> Result<()> {
1301 ExplodedCache::explode(self, oid, kind, payload)
1302 }
1303 fn flush(&self) -> Result<()> {
1304 self.eager.flush().map(|_| ())
1305 }
1306}
1307
1308fn registry() -> &'static Mutex<Vec<std::sync::Weak<ExplodedCache>>> {
1309 static R: std::sync::OnceLock<Mutex<Vec<std::sync::Weak<ExplodedCache>>>> =
1310 std::sync::OnceLock::new();
1311 R.get_or_init(|| Mutex::new(Vec::new()))
1312}
1313
1314/// **Every open store's explode band**, as `(store root, use)`. What the
1315/// dashboard tick and `Admin.StorageInfo` read; a `metadata` call per pool and
1316/// no store lock. Dead entries are swept on the way past.
1317pub fn use_all() -> Vec<(String, CacheUse)> {
1318 let mut r = registry().lock().unwrap_or_else(|e| e.into_inner());
1319 r.retain(|w| w.strong_count() > 0);
1320 r.iter()
1321 .filter_map(|w| w.upgrade())
1322 .map(|c| (c.root.clone(), c.use_now()))
1323 .collect()
1324}
1325
1326/// Set the budget of every open store under `root` (a prefix match, so an
1327/// account's whole tree is one call). Returns how many were changed — applied
1328/// output, so a control plane that set nothing can say so.
1329pub fn set_budget_under(root: &str, bytes: u64) -> usize {
1330 let mut n = 0;
1331 let mut r = registry().lock().unwrap_or_else(|e| e.into_inner());
1332 r.retain(|w| w.strong_count() > 0);
1333 for c in r.iter().filter_map(|w| w.upgrade()) {
1334 if c.root.starts_with(root) {
1335 c.set_budget(bytes);
1336 n += 1;
1337 }
1338 }
1339 n
1340}
1341
1342/// One segment is 1/16 of the budget, clamped. An unlimited budget still
1343/// rotates, so a pool that is later given a budget can be evicted without a
1344/// rewrite.
1345fn segment_for(budget: u64) -> u64 {
1346 if budget == 0 {
1347 return MAX_SEGMENT.min(1 << 28);
1348 }
1349 // The floor yields to the budget rather than overriding it: a 16 MiB
1350 // minimum against a 1 MiB quota would mean the pool never rotates and
1351 // therefore never evicts — a cap that silently is not one, which is the
1352 // exact shape of bug LAW 2 is about. So the floor is `min(MIN_SEGMENT,
1353 // budget/4)`: a real budget gets 16 MiB segments, a tiny one gets tiny ones.
1354 let floor = MIN_SEGMENT.min(budget / 4).max(64 << 10);
1355 (budget / SEGMENTS_PER_BUDGET).clamp(floor, MAX_SEGMENT)
1356}
1357
1358#[cfg(test)]
1359mod tests {
1360 use super::*;
1361
1362 fn tmp(name: &str) -> PathBuf {
1363 let d = std::env::temp_dir().join(format!("explode-cache-{}-{name}", std::process::id()));
1364 let _ = std::fs::remove_dir_all(&d);
1365 std::fs::create_dir_all(&d).unwrap();
1366 d.join("objects.exploded")
1367 }
1368
1369 /// A cost the admission gate admits: deep enough and big enough that the
1370 /// cache is the cheaper place for it. Tests that are about eviction must
1371 /// not accidentally be about admission.
1372 /// An INCOMPRESSIBLE payload of `len` bytes (a tiny LCG seeded by `seed`).
1373 /// The eviction tests reason about bytes on disk against a budget, and since
1374 /// lever 3 (2026-09-04) a row is stored zlib-compressed when that is smaller
1375 /// — a `vec![3u8; 64 << 10]` lands as ~80 bytes and the pool never reaches
1376 /// the budget those tests were sized for ("nothing was evicted" ×3). Noise
1377 /// does not compress, so the budget arithmetic means what it did.
1378 fn noise(len: usize, seed: u8) -> Vec<u8> {
1379 let mut x: u32 = 0x9E37_79B9 ^ (seed as u32).wrapping_mul(0x0101_0101);
1380 (0..len)
1381 .map(|_| {
1382 x = x.wrapping_mul(1_664_525).wrapping_add(1_013_904_223);
1383 (x >> 24) as u8
1384 })
1385 .collect()
1386 }
1387
1388 fn dear(payload: &[u8]) -> crate::resolve::ReadCost {
1389 crate::resolve::ReadCost {
1390 depth: CHEAP_DEPTH + 8,
1391 inflated: payload.len() as u64,
1392 external_base: false,
1393 }
1394 }
1395
1396 /// Assert at compile time that the eviction fixtures below stay under the
1397 /// size cap — a fixture that grew past it would be silently DECLINED and
1398 /// the test would then be asserting nothing.
1399 const _: () = assert!((64 << 10) < MAX_CACHED_BYTES as usize);
1400
1401 fn oid(n: u32) -> Vec<u8> {
1402 let mut o = vec![0u8; 20];
1403 o[..4].copy_from_slice(&n.to_be_bytes());
1404 o
1405 }
1406
1407 /// ★ **THE ANSWER TO "does it take the disk anyway?"**
1408 ///
1409 /// Every object of a repository is read once, each one written back — the
1410 /// full-history browse that would otherwise reproduce the eager tier. The
1411 /// tier must **plateau at its budget**, not climb.
1412 ///
1413 /// Seen RED by returning `Ok(0)` from `evict_to_budget`: the pool climbs to
1414 /// 9.4 MiB against a 2 MiB budget and the assertion names the number.
1415 #[test]
1416 fn the_tier_plateaus_at_its_budget_under_a_full_scan() {
1417 let base = tmp("plateau");
1418 let budget = 2 << 20;
1419 let c = ExplodedCache::open(&base, ExplodePolicy::Commits, budget).unwrap();
1420 let payload = vec![7u8; 8 << 10];
1421 let mut peak = 0u64;
1422 for i in 0..2000u32 {
1423 c.write_back(&oid(i), GitObjectKind::Blob, &payload, dear(&payload)).unwrap();
1424 // The store's own thread, doing what it does between reads.
1425 c.maintain().unwrap();
1426 peak = peak.max(c.use_now().used());
1427 }
1428 let end = c.use_now();
1429 let written = 2000u64 * payload.len() as u64;
1430 assert!(
1431 peak <= budget as u64 + segment_for(budget) + (1 << 20),
1432 "the tier climbed to {peak} B against a {budget} B budget after {written} B of \
1433 write-backs — it is a tier, not a cache"
1434 );
1435 assert!(
1436 end.evicted_bytes > 0 && end.evicted_segments > 0,
1437 "nothing was evicted, so the plateau is an accident of size: {end:?}"
1438 );
1439 }
1440
1441 /// An account at its budget still serves **every** read correctly — only
1442 /// slower. Eviction is never an error a client sees.
1443 #[test]
1444 fn an_evicted_object_is_absent_and_never_wrong() {
1445 let base = tmp("absent");
1446 let c = ExplodedCache::open(&base, ExplodePolicy::Commits, 1 << 20).unwrap();
1447 let payload = noise(64 << 10, 3);
1448 for i in 0..200u32 {
1449 c.write_back(&oid(i), GitObjectKind::Blob, &payload, dear(&payload)).unwrap();
1450 c.maintain().unwrap();
1451 }
1452 // Whatever survived comes back byte-identical; whatever did not comes
1453 // back `None`, which is the caller's cue to re-inflate the chain.
1454 let mut hits = 0;
1455 for i in 0..200u32 {
1456 match c.content(&oid(i)).unwrap() {
1457 Some((k, b)) => {
1458 assert_eq!(k, GitObjectKind::Blob);
1459 assert_eq!(b, payload, "an evicting cache returned the wrong bytes");
1460 hits += 1;
1461 }
1462 None => {}
1463 }
1464 }
1465 assert!(hits > 0, "the cache evicted everything, including the newest segment");
1466 assert!(hits < 200, "nothing was evicted, so this proves nothing");
1467 }
1468
1469 /// Commits are the last thing standing: the eager pool is never unlinked,
1470 /// whatever the lazy pool does.
1471 /// **The box has one line, and the biggest pool pays for it.** Two caches,
1472 /// each far under its own per-store budget, share a small global line: the
1473 /// one holding most of the box's cache evicts on its tick; the one under
1474 /// its fair share does not, however far over the line the box is.
1475 ///
1476 /// Robust to whatever else is registered in this process: the fairness
1477 /// claim is made with B holding ONE 8 KiB row (over its share only if more
1478 /// than ~128 caches held bytes at once), and the convergence claim is made
1479 /// on A's own bytes, not on a sum that other tests' pools can move.
1480 ///
1481 /// Seen RED by making `over_global_share` return `false` always: A evicted
1482 /// nothing ("the big pool must evict for the box") and stayed at 3 MiB
1483 /// over a 1 MiB line. Restored.
1484 #[test]
1485 fn the_global_line_is_paid_by_the_cache_over_its_share() {
1486 let base_a = tmp("global-a");
1487 let base_b = tmp("global-b");
1488 // 4 MiB per store → 1 MiB segments (the budget/4 floor), so 3 MiB rotates into four and
1489 // the per-store budget itself never evicts anything here.
1490 let a = std::sync::Arc::new(ExplodedCache::open(&base_a, ExplodePolicy::Commits, 4 << 20).unwrap())
1491 .registered();
1492 let b = std::sync::Arc::new(ExplodedCache::open(&base_b, ExplodePolicy::Commits, 4 << 20).unwrap())
1493 .registered();
1494 // Fill with the global check OFF, so the shape under test is set up
1495 // before it is asked about.
1496 a.set_global_line(0);
1497 b.set_global_line(0);
1498 let big = noise(64 << 10, 5);
1499 for i in 0..48u32 {
1500 a.write_back(&oid(i), GitObjectKind::Blob, &big, dear(&big)).unwrap();
1501 a.maintain().unwrap();
1502 }
1503 let small = noise(8 << 10, 6);
1504 b.write_back(&oid(100), GitObjectKind::Blob, &small, dear(&small)).unwrap();
1505 b.maintain().unwrap();
1506 let a0 = a.use_now();
1507 let b0 = b.use_now();
1508 let line = 1u64 << 20;
1509 assert!(a0.lazy_bytes > line, "precondition: A alone is over the line, has {}", a0.lazy_bytes);
1510 assert!(a0.segments >= 3, "precondition: A rotated into segments, has {}", a0.segments);
1511 assert_eq!(a0.evicted_segments + b0.evicted_segments, 0, "precondition: nothing evicted yet");
1512
1513 a.set_global_line(line);
1514 b.set_global_line(line);
1515
1516 // B ticks first: under its share, it must not pay for A.
1517 b.maintain().unwrap();
1518 assert_eq!(b.use_now().evicted_segments, 0, "a cache under its fair share must not evict for the box");
1519
1520 // A ticks: over its share, it evicts for the box until it is under the
1521 // line on its own (it cannot stop above it: then the box is over and
1522 // it is over any share of it).
1523 a.maintain().unwrap();
1524 let a1 = a.use_now();
1525 assert!(a1.evicted_segments > 0, "the big pool must evict for the box");
1526 assert!(
1527 a1.lazy_bytes + a1.eager_bytes <= line,
1528 "A must have shrunk under the line on its own: {} > {line}",
1529 a1.lazy_bytes + a1.eager_bytes
1530 );
1531 assert!(a1.lazy_bytes < 4 << 20, "its own budget was never the reason");
1532 assert_eq!(b.use_now().evicted_segments, 0, "B still untouched");
1533 }
1534 /// Fill one store into several lazy segments with its budget and the
1535 /// global line both far away, so only the disk line can evict.
1536 fn filled_for_disk(name: &str) -> ExplodedCache {
1537 let c = ExplodedCache::open(&tmp(name), ExplodePolicy::Commits, 64 << 20).unwrap();
1538 c.set_global_line(0);
1539 c.segment_bytes.store(256 << 10, Ordering::Release);
1540 c.disk_used_override.store(10, Ordering::Release);
1541 let big = noise(64 << 10, 9);
1542 for i in 0..24u32 {
1543 c.write_back(&oid(i), GitObjectKind::Blob, &big, dear(&big)).unwrap();
1544 c.maintain().unwrap();
1545 }
1546 let u = c.use_now();
1547 assert!(u.segments >= 4, "precondition: rotated into segments, has {}", u.segments);
1548 assert_eq!(u.evicted_segments, 0, "precondition: nothing evicted under a 10 % disk");
1549 c
1550 }
1551
1552 /// ★ **At 80 % disk the oldest cached objects go.** Owner 2026-09-28.
1553 ///
1554 /// Seen RED by leaving `over_disk` out of the eviction condition: at 85 %
1555 /// nothing was evicted.
1556 #[test]
1557 fn a_disk_at_its_line_drops_the_oldest_segments() {
1558 let c = filled_for_disk("disk-line");
1559 let newest_before = c.segments().unwrap().last().unwrap().path().to_path_buf();
1560 assert!(c.content(&oid(0)).unwrap().is_some(), "precondition: the oldest write-back is cached");
1561 c.disk_used_override.store(85, Ordering::Release);
1562 c.maintain().unwrap();
1563 let u = c.use_now();
1564 assert!(u.evicted_segments > 0, "a disk at 85 % must drop cache segments");
1565 assert_eq!(u.segments, 1, "under pressure the pool gives back all but the live segment");
1566 assert_eq!(
1567 c.segments().unwrap()[0].path(),
1568 newest_before.as_path(),
1569 "the segment kept is the newest one, so what went was the oldest"
1570 );
1571 assert!(c.content(&oid(0)).unwrap().is_none(), "the oldest write-back is gone");
1572 }
1573
1574 /// Under the line, or with the line off, the disk reading changes nothing.
1575 #[test]
1576 fn a_disk_under_its_line_or_with_it_off_drops_nothing() {
1577 let c = filled_for_disk("disk-under");
1578 c.disk_used_override.store(79, Ordering::Release);
1579 c.maintain().unwrap();
1580 assert_eq!(c.use_now().evicted_segments, 0, "79 % is under an 80 % line");
1581 c.set_disk_line_percent(0);
1582 c.disk_used_override.store(99, Ordering::Release);
1583 c.maintain().unwrap();
1584 assert_eq!(c.use_now().evicted_segments, 0, "a line of 0 is off");
1585 }
1586
1587 /// The real reading is a percentage, the way `df` prints it.
1588 #[test]
1589 fn the_real_disk_reading_is_a_percentage() {
1590 let p = disk_used_percent(&std::env::temp_dir()).expect("statvfs answers for the temp dir");
1591 assert!(p <= 100, "{p}");
1592 }
1593
1594 /// **The disk line reads the filesystem the pools live on, from the first
1595 /// open.** Review 2026-09-28: it was read through `base`, the eager FILE,
1596 /// which does not exist until a commit is flushed — `statvfs` said ENOENT
1597 /// and the line was off. Seen RED with `disk_used_percent(&self.base)`
1598 /// restored in `over_disk_line`: a real reading at its own value did not
1599 /// count as reached.
1600 #[test]
1601 fn the_disk_line_reads_the_pool_directory_before_the_eager_file_exists() {
1602 let base = tmp("disk-fresh");
1603 let c = ExplodedCache::open(&base, ExplodePolicy::Commits, 64 << 20).unwrap();
1604 assert!(!base.exists(), "precondition: nothing flushed, no eager file");
1605 assert!(disk_used_percent(&base).is_none(), "precondition: statvfs of a missing file");
1606 let p = disk_used_percent(base.parent().unwrap()).expect("statvfs of the pool dir");
1607 c.disk_used_override.store(ExplodedCache::NO_DISK_READING, Ordering::Release);
1608 c.set_disk_line_percent(p.max(1));
1609 assert_eq!(c.over_disk_line(), p >= p.max(1), "the real reading at {p} % against a {p} % line");
1610 }
1611
1612 /// **An eviction between a reader's snapshot and its read is a miss, never
1613 /// an error, and never an orphan file.** Review 2026-09-28: `segments()`
1614 /// hands out `Arc`s and the lock is released, a segment opens its file BY
1615 /// PATH per read, so a segment unlinked in between answered `ENOENT` — and
1616 /// `content`/`of_kind` propagated it with `?` (on the push path that is
1617 /// `refold` failing the absorb). And a flush through such an `Arc` opened
1618 /// the path with `O_CREAT`, putting the file back where no pool lists it.
1619 ///
1620 /// Seen RED twice, one guard at a time: without `lazy_read`'s retired arm,
1621 /// "opening …/objects.exploded.c1" came back as an `Err`; without the
1622 /// retired check in `ExplodedArchive::flush`, "a flush through an evicted
1623 /// segment re-created …/objects.exploded.c1".
1624 #[test]
1625 fn a_read_or_flush_through_an_evicted_segment_is_a_miss_and_leaves_no_file() {
1626 let base = tmp("evict-race");
1627 let c = ExplodedCache::open(&base, ExplodePolicy::Commits, 64 << 20).unwrap();
1628 c.set_global_line(0);
1629 c.segment_bytes.store(256 << 10, Ordering::Release);
1630 let big = noise(64 << 10, 9);
1631 for i in 0..24u32 {
1632 c.write_back(&oid(i), GitObjectKind::Blob, &big, dear(&big)).unwrap();
1633 c.maintain().unwrap();
1634 }
1635 drop(c);
1636 // A reopen, so no segment has built its index: the first read has to
1637 // open the file, which is exactly the window.
1638 let c = ExplodedCache::open(&base, ExplodePolicy::Commits, 64 << 20).unwrap();
1639 c.set_global_line(0);
1640 let snapshot = c.segments().unwrap();
1641 assert!(snapshot.len() >= 4, "precondition: several segments, has {}", snapshot.len());
1642 let oldest = snapshot[0].clone();
1643 let path = oldest.path().to_path_buf();
1644
1645 c.disk_used_override.store(85, Ordering::Release);
1646 c.maintain().unwrap();
1647 assert!(!path.exists(), "precondition: the oldest segment was unlinked");
1648 assert!(oldest.is_retired(), "an evicted segment is retired");
1649
1650 // The raw segment read is the hazard …
1651 assert!(oldest.content(&oid(0)).is_err(), "precondition: the file is gone under the Arc");
1652 // … and through the cache's door it is a miss.
1653 assert_eq!(ExplodedCache::lazy_read(&oldest, oldest.content(&oid(0))).unwrap(), None);
1654 assert_eq!(
1655 ExplodedCache::lazy_read(&oldest, oldest.of_kind(GitObjectKind::Blob)).unwrap(),
1656 None
1657 );
1658
1659 // A late writer holding the old Arc must not bring the file back.
1660 oldest.explode(&oid(999), GitObjectKind::Blob, b"late").unwrap();
1661 oldest.flush().unwrap();
1662 assert!(!path.exists(), "a flush through an evicted segment re-created {}", path.display());
1663 }
1664
1665 #[test]
1666 fn eviction_never_takes_a_commit() {
1667 let base = tmp("commits");
1668 let c = ExplodedCache::open(&base, ExplodePolicy::Commits, 1 << 20).unwrap();
1669 for i in 0..50u32 {
1670 c.explode(&oid(1_000_000 + i), GitObjectKind::Commit, b"tree x\n")
1671 .unwrap();
1672 }
1673 c.flush().unwrap();
1674 let blob = noise(64 << 10, 1);
1675 for i in 0..200u32 {
1676 c.write_back(&oid(i), GitObjectKind::Blob, &blob, dear(&blob)).unwrap();
1677 c.maintain().unwrap();
1678 }
1679 assert!(c.use_now().evicted_segments > 0, "nothing was evicted");
1680 for i in 0..50u32 {
1681 assert!(
1682 c.content(&oid(1_000_000 + i)).unwrap().is_some(),
1683 "a commit was evicted — graph and reach are now blind"
1684 );
1685 }
1686 }
1687
1688 /// `repo gc` reclaims the cache and keeps the graph.
1689 #[test]
1690 fn dropping_the_cache_reclaims_the_lazy_pool_only() {
1691 let base = tmp("gc");
1692 let c = ExplodedCache::open(&base, ExplodePolicy::Commits, 0).unwrap();
1693 c.explode(&oid(9), GitObjectKind::Commit, b"tree y\n").unwrap();
1694 let blob = vec![2u8; 32 << 10];
1695 for i in 0..100u32 {
1696 c.write_back(&oid(i), GitObjectKind::Blob, &blob, dear(&blob)).unwrap();
1697 }
1698 c.maintain().unwrap();
1699 let before = c.use_now();
1700 assert!(before.lazy_bytes > 0);
1701 let freed = c.drop_cache().unwrap();
1702 assert!(freed > 0, "gc reclaimed nothing");
1703 let after = c.use_now();
1704 assert_eq!(after.lazy_bytes, 0, "the lazy pool survived gc");
1705 assert_eq!(after.eager_bytes, before.eager_bytes, "gc took the graph too");
1706 assert!(c.content(&oid(9)).unwrap().is_some());
1707 }
1708
1709 /// The segments a previous process wrote are adopted, so a restart does not
1710 /// lose the cache and does not lose track of its size either.
1711 #[test]
1712 fn a_reopen_adopts_the_segments_it_left() {
1713 let base = tmp("reopen");
1714 let blob = vec![5u8; 32 << 10];
1715 let bytes = {
1716 let c = ExplodedCache::open(&base, ExplodePolicy::Commits, 0).unwrap();
1717 for i in 0..40u32 {
1718 c.write_back(&oid(i), GitObjectKind::Blob, &blob, dear(&blob)).unwrap();
1719 }
1720 c.maintain().unwrap();
1721 c.use_now().lazy_bytes
1722 };
1723 assert!(bytes > 0);
1724 let again = ExplodedCache::open(&base, ExplodePolicy::Commits, 0).unwrap();
1725 assert_eq!(again.use_now().lazy_bytes, bytes, "the segments were not adopted");
1726 assert_eq!(
1727 again.content(&oid(7)).unwrap().map(|(_, b)| b),
1728 Some(blob),
1729 "an adopted segment does not answer"
1730 );
1731 }
1732
1733 /// ★ **DO NOT EXPLODE UNIFORMLY: a cheap object is never cached.**
1734 ///
1735 /// The owner, 2026-08-30: the index points at the object's own zlib stream at
1736 /// its own offset, so a whole object is one direct inflate. Caching that
1737 /// spends quota to save microseconds — and evicts something that costs
1738 /// milliseconds. So the gate is rebuild cost, not "was read".
1739 ///
1740 /// Seen RED by making `write_back` admit everything: 400 shallow 1 KiB
1741 /// objects land in the pool, `direct_reads` stays 0, and the assertion
1742 /// names both.
1743 #[test]
1744 fn a_shallow_small_object_is_read_directly_and_never_cached() {
1745 let base = tmp("cheap");
1746 let c = ExplodedCache::open(&base, ExplodePolicy::Commits, 0).unwrap();
1747 let small = vec![1u8; 1 << 10];
1748 let big = vec![2u8; 64 << 10];
1749
1750 // Cheap: shallow and small. Not worth a byte of anybody's quota.
1751 for i in 0..400u32 {
1752 let took = c
1753 .write_back(
1754 &oid(i),
1755 GitObjectKind::Blob,
1756 &small,
1757 crate::resolve::ReadCost {
1758 depth: CHEAP_DEPTH,
1759 inflated: small.len() as u64,
1760 external_base: false,
1761 },
1762 )
1763 .unwrap();
1764 assert!(!took, "a cheap object asked for a flush");
1765 }
1766 c.maintain().unwrap();
1767 let after_cheap = c.use_now();
1768 assert_eq!(
1769 after_cheap.lazy_bytes, 0,
1770 "400 shallow 1 KiB objects put {} B in the cache; a direct read is \
1771 microseconds and this is quota spent to save them",
1772 after_cheap.lazy_bytes
1773 );
1774 assert_eq!(
1775 after_cheap.direct_reads, 400,
1776 "the policy declined {} of 400 cheap objects — it is not running",
1777 after_cheap.direct_reads
1778 );
1779 assert_eq!(after_cheap.written_back, 0);
1780
1781 // Expensive by DEPTH, and by an external base. Each on its own admits.
1782 c.write_back(
1783 &oid(1000),
1784 GitObjectKind::Blob,
1785 &small,
1786 crate::resolve::ReadCost { depth: CHEAP_DEPTH + 1, inflated: small.len() as u64, external_base: false },
1787 )
1788 .unwrap();
1789 c.write_back(
1790 &oid(1002),
1791 GitObjectKind::Blob,
1792 &small,
1793 crate::resolve::ReadCost { depth: 0, inflated: small.len() as u64, external_base: true },
1794 )
1795 .unwrap();
1796 // **Big is NOT a reason to cache.** A 64 KiB — or a 3 MiB — whole blob
1797 // is one linear inflate and a bad tenant of a shared budget, and it is
1798 // declined however deep it is. This is lane AA's second copy.
1799 c.write_back(
1800 &oid(1001),
1801 GitObjectKind::Blob,
1802 &big,
1803 crate::resolve::ReadCost { depth: 40, inflated: (MAX_CACHED_BYTES + 1) as u64, external_base: true },
1804 )
1805 .unwrap();
1806 c.maintain().unwrap();
1807 let end = c.use_now();
1808 assert_eq!(end.written_back, 2, "an expensive object was declined");
1809 assert_eq!(
1810 end.direct_reads, 401,
1811 "an object over the size cap was cached instead of read directly"
1812 );
1813 for n in [1000u32, 1002] {
1814 assert!(
1815 c.content(&oid(n)).unwrap().is_some(),
1816 "the expensive object {n} is not in the cache"
1817 );
1818 }
1819 assert!(
1820 c.content(&oid(1001)).unwrap().is_none(),
1821 "an object over the size cap is in the cache"
1822 );
1823 assert!(
1824 c.content(&oid(7)).unwrap().is_none(),
1825 "a declined object is in the cache after all"
1826 );
1827 }
1828
1829 /// ★ **Eviction ranks by rebuild cost, not by age alone.**
1830 ///
1831 /// Two segments, the older one full of expensive objects and the newer one
1832 /// full of barely-admitted ones. Plain FIFO takes the expensive segment
1833 /// because it is older; ranking by `mean rebuild cost × recency` takes the
1834 /// cheap one, which is the whole point — evict what is cheap to recompute.
1835 ///
1836 /// Seen RED by restoring `Some(l[0].clone())`: the depth-40 objects go and
1837 /// the barely-admitted ones stay.
1838 #[test]
1839 fn eviction_takes_the_cheap_segment_before_the_old_expensive_one() {
1840 let base = tmp("costrank");
1841 // Small budget, small segments: three segments fit and the fourth
1842 // forces one out.
1843 let c = ExplodedCache::open(&base, ExplodePolicy::Commits, 1 << 20).unwrap();
1844 let payload = noise(32 << 10, 9);
1845 let dear = crate::resolve::ReadCost { depth: 40, inflated: payload.len() as u64, external_base: false };
1846 // Barely admitted: one link past the cheap depth.
1847 let cheapish = crate::resolve::ReadCost {
1848 depth: CHEAP_DEPTH + 1,
1849 inflated: payload.len() as u64,
1850 external_base: false,
1851 };
1852
1853 // Segment 0: expensive. Rotate it out by filling past the segment size.
1854 for i in 0..16u32 {
1855 c.write_back(&oid(i), GitObjectKind::Blob, &payload, dear).unwrap();
1856 }
1857 c.maintain().unwrap();
1858 // Segment 1+: cheapish.
1859 for i in 100..160u32 {
1860 c.write_back(&oid(i), GitObjectKind::Blob, &payload, cheapish).unwrap();
1861 c.maintain().unwrap();
1862 }
1863 let end = c.use_now();
1864 assert!(end.evicted_segments > 0, "nothing was evicted; this proves nothing");
1865 // The expensive objects are the ones still there.
1866 let expensive_left = (0..16u32).filter(|i| c.content(&oid(*i)).unwrap().is_some()).count();
1867 let cheap_left = (100..160u32).filter(|i| c.content(&oid(*i)).unwrap().is_some()).count();
1868 assert!(
1869 expensive_left > 0,
1870 "every depth-40 object was evicted while {cheap_left} barely-admitted ones \
1871 survived — eviction is ranking by age alone: {end:?}"
1872 );
1873 }
1874
1875 /// A budget is parsed, and a typo is refused rather than silently taking a
1876 /// default nobody is billed for.
1877 #[test]
1878 fn a_budget_is_read_and_a_typo_is_refused() {
1879 assert_eq!(parse_budget("8G").unwrap(), 8 << 30);
1880 assert_eq!(parse_budget(" 512m ").unwrap(), 512 << 20);
1881 assert_eq!(parse_budget("1048576").unwrap(), 1 << 20);
1882 assert_eq!(parse_budget("unlimited").unwrap(), 0);
1883 assert!(parse_budget("8 gigs").is_err());
1884 assert!(parse_budget("").is_err());
1885 }
1886
1887 /// The default is derived from the filesystem, and the appliance arithmetic
1888 /// in the module docs is the one the code does.
1889 #[test]
1890 fn the_default_budget_is_the_smaller_of_a_tenth_and_the_ceiling() {
1891 let d = default_budget(&std::env::temp_dir());
1892 assert!(d >= MIN_BUDGET && d <= BUDGET_CEILING, "{d}");
1893 }
1894
1895 /// **A validation read caches nothing, and the same reads without the guard
1896 /// fill the pool** — both halves, because either alone proves nothing.
1897 ///
1898 /// The RED half is the first loop: these are objects the admission policy
1899 /// WANTS (deep chains), so a pool that stays empty under the guard cannot be
1900 /// explained by `worth_caching` declining them. The counter is asserted too,
1901 /// because "the pool is empty" is also what a cache that was handed nothing
1902 /// looks like.
1903 #[test]
1904 fn a_suppressed_thread_caches_nothing_and_an_unsuppressed_one_fills_the_pool() {
1905 let payload = vec![7u8; 64 << 10];
1906 let costly = crate::resolve::ReadCost {
1907 depth: CHEAP_DEPTH + 8,
1908 inflated: payload.len() as u64,
1909 external_base: false,
1910 };
1911
1912 // ── RED: no guard. The same objects, admitted. ──────────────────────
1913 let hot = ExplodedCache::open(&tmp("nosuppress"), ExplodePolicy::Commits, 0).unwrap();
1914 for i in 0..200u32 {
1915 hot.write_back(&oid(i), GitObjectKind::Tree, &payload, costly).unwrap();
1916 }
1917 hot.maintain().unwrap();
1918 let hot_use = hot.use_now();
1919 assert!(
1920 hot_use.lazy_bytes > 0,
1921 "without the guard these objects must be cached, or the GREEN half \
1922 below is proving that `worth_caching` declined them and nothing else"
1923 );
1924 assert_eq!(hot_use.written_back, 200);
1925 assert_eq!(hot_use.suppressed_write_backs, 0);
1926
1927 // ── GREEN: the identical loop inside the guard. ─────────────────────
1928 let cold = ExplodedCache::open(&tmp("suppress"), ExplodePolicy::Commits, 0).unwrap();
1929 {
1930 let _guard = suppress_write_back();
1931 assert!(write_back_suppressed());
1932 for i in 0..200u32 {
1933 let took = cold.write_back(&oid(i), GitObjectKind::Tree, &payload, costly).unwrap();
1934 assert!(!took, "a validation read asked for a flush");
1935 }
1936 }
1937 cold.maintain().unwrap();
1938 let cold_use = cold.use_now();
1939 assert_eq!(
1940 cold_use.lazy_bytes, 0,
1941 "a receive-pack's connectivity walk put {} B in the cache",
1942 cold_use.lazy_bytes
1943 );
1944 assert_eq!(cold_use.written_back, 0);
1945 assert_eq!(
1946 cold_use.suppressed_write_backs, 200,
1947 "the gate must be what declined them — {} suppressed, not 200",
1948 cold_use.suppressed_write_backs
1949 );
1950 // The policy was never consulted: a suppressed read is declined BEFORE
1951 // the admission gate, so it is not a `direct_read` either.
1952 assert_eq!(cold_use.direct_reads, 0);
1953 }
1954
1955 /// The guard is released on drop, including out of a panic — a thread that
1956 /// panicked mid-walk must not stop caching for the rest of its life.
1957 #[test]
1958 fn the_guard_is_released_on_drop_and_on_unwind_and_it_nests() {
1959 assert!(!write_back_suppressed(), "a fresh thread starts unsuppressed");
1960 {
1961 let _a = suppress_write_back();
1962 {
1963 let _b = suppress_write_back();
1964 assert!(write_back_suppressed());
1965 }
1966 assert!(
1967 write_back_suppressed(),
1968 "the inner guard ending reopened the cache while the outer scope \
1969 is still validating"
1970 );
1971 }
1972 assert!(!write_back_suppressed());
1973
1974 let unwound = std::panic::catch_unwind(|| {
1975 let _g = suppress_write_back();
1976 assert!(write_back_suppressed());
1977 panic!("a connectivity walk died");
1978 });
1979 assert!(unwound.is_err());
1980 assert!(
1981 !write_back_suppressed(),
1982 "a panic inside the guard left this thread never caching again"
1983 );
1984 }
1985}
1986
1987#[cfg(test)]
1988mod disk_hysteresis {
1989 use super::{pressure_after, DISK_LINE_HYSTERESIS_PERCENT};
1990
1991 #[test]
1992 fn the_line_trips_at_80_and_holds_until_under_75() {
1993 assert_eq!(DISK_LINE_HYSTERESIS_PERCENT, 5);
1994 assert!(!pressure_after(false, 79, 80), "under the line, not latched: no pressure");
1995 assert!(pressure_after(false, 80, 80), "at the line: trips");
1996 assert!(pressure_after(true, 78, 80), "between 75 and 80 after tripping: holds");
1997 assert!(pressure_after(true, 75, 80), "75 is not under 75: holds");
1998 assert!(!pressure_after(true, 74, 80), "under 75: releases");
1999 assert!(!pressure_after(false, 77, 80), "between the marks without having tripped: stays off");
2000 }
2001}