Skip to main content

znippy_plugin_git/
gc.rs

1//! Garbage collection, as a trait with two implementations.
2//!
3//! Both reclaim the same thing: the payload of entries nothing points at any
4//! more. `supersede_as_delta` replaces a generation's bytes with a delta and
5//! drops its index rows, but the old *blob* stays in the file, unreferenced —
6//! measured in znippy-common, an 8 056 624-byte archive went to 8 057 731 after
7//! 4 000 000 bytes of live payload became 11. Reclaiming that is what a GC is
8//! here for.
9//!
10//! ## Neither implementation copies a git object through a codec
11//!
12//! Both route the copy through
13//! [`znippy_common::compact_archive`](znippy_common::compact_archive), which
14//! copies every live chunk's on-disk bytes **verbatim**: no decode, no
15//! re-encode, no delta recomputation, `compressed` / `uncompressed_size` /
16//! blake3 carried unchanged, and a delta chunk staying a delta chunk at the same
17//! depth. Nothing in this module reimplements that loop (LAW 5); the two
18//! implementations differ only in *where the result ends up*.
19//!
20//! ## The two, and the one real difference between them
21//!
22//! | | [`CompactInPlace`] (A) | [`NewGeneration`] (B, the default) |
23//! |---|---|---|
24//! | result | the same path, rewritten | a **new** file, `x.znippy` → `x.g1.znippy` |
25//! | commit point | `rename(staged, archive)` | `rename(work, x.g1.znippy)`, then `unlink(x.znippy)` |
26//! | verified before committing? | **no** — the rename has already happened | **yes** — the new generation is read back in full first |
27//! | interrupted at any byte | the original is still there, dead payload and all | the original is still there, and so is the debris |
28//!
29//! ### Crash safety, by construction, in both
30//!
31//! Nothing is ever written over live data. A: `compact_archive` stages a
32//! `*.compact-<pid>-<nanos>` file *beside* the destination and the only mutation
33//! of the archive's own name is one `rename(2)`, which is atomic — so an
34//! interruption at any byte leaves the original serving and a stray staged file
35//! behind. B: the compaction runs against a **hard link** to the original inode,
36//! so the original *name* is untouched for the whole run; the new generation is
37//! then verified, renamed into place, and only after that is the old name
38//! unlinked. There is no window in either where a reader finds no archive.
39//!
40//! B's extra guarantee is the one A structurally cannot give: A commits by
41//! rename before anything has read the result back, so if the compacted file
42//! were bad, the good one is already gone. B verifies first and deletes last.
43//! That is why B is the default.
44//!
45//! ## The third ending, not yet written
46//!
47//! B's last step deletes the superseded generation. [`retire_to_holger`] is the
48//! alternative: ship it to nordisk's artifact server instead, because a sealed
49//! archive is already the shape an artifact store wants. **That function is
50//! deliberately empty** — signature and contract only. Read its doc comment for
51//! why the shape works; do not add a client to this file.
52//!
53//! ## What "verified" means here
54//!
55//! Every entry is **read back through the ordinary reader** with
56//! `ZnippyArchive::extract_file_verified` — delta chains reconstructed, blake3
57//! checked against the index — plus a set comparison of the manifest
58//! (`relative_path` and `uncompressed_size`) against the original, because an
59//! archive that verifies against its own index but has lost an entry is
60//! internally consistent and still wrong.
61//!
62//! It is deliberately **not** [`znippy_common::verify_archive_integrity`], and
63//! that is a finding rather than a preference: on the fixture in
64//! [`tests::both_implementations_reclaim_the_dead_payload_and_change_no_entry`]
65//! — four generations, three of them superseded into deltas — that function
66//! reports "3 corrupt entries of 4" on a perfectly good archive, because its
67//! `decompress_archive` path does not reconstruct delta chunks. It cannot be
68//! used to gate a GC of any archive `supersede_as_delta` has touched, which is
69//! every archive a GC is worth running on.
70
71use std::path::{Path, PathBuf};
72
73use anyhow::{Context, Result, anyhow, bail};
74use znippy_common::{
75    ArtifactMeta, CompactReport, ZnippyArchive, compact_archive, get_all_files_meta,
76};
77
78/// What one GC run did. Owned by the `git-storage-trait` contract; re-exported
79/// here so `crate::gc::GcReport` stays a valid path. Every field is measured
80/// off the filesystem or off the archive after the fact — see the trait crate
81/// for per-field docs, and note `retired_packs` is always `0` from a bare
82/// [`Gc::run`]: `GitOps::gc` fills it in.
83pub use git_storage_trait::GcReport;
84
85/// Reclaim the dead payload in a znippy archive.
86pub trait Gc {
87    fn run(&self, archive: &Path) -> Result<GcReport>;
88    fn name(&self) -> &'static str;
89}
90
91// ── the third ending: retire the old generation to holger ─────────────────────
92
93/// Where a pensioned generation goes. Deliberately just an address and a name:
94/// what the endpoint means is the transport's business, and no transport is
95/// decided.
96#[derive(Debug, Clone, PartialEq, Eq)]
97pub struct HolgerTarget {
98    /// Where to reach the holger instance. Opaque to this crate.
99    pub endpoint: String,
100    /// The repository the generation belongs to, so the artifact is filed under
101    /// something meaningful at the destination.
102    pub repository: String,
103}
104
105/// What a completed retirement reports. Every field is about the archive that
106/// moved; nothing here describes an index, because a sealed archive carries its
107/// own and no index is shipped separately.
108#[derive(Debug, Clone, PartialEq, Eq)]
109pub struct RetirementReceipt {
110    /// The name the generation was filed under at the destination.
111    pub artifact: String,
112    pub bytes_shipped: u64,
113    /// The archive's digest, as computed locally and as confirmed by holger.
114    /// They are the same value or the retirement failed.
115    pub digest_hex: String,
116    /// Holger read the archive back and it verified. Never `false` on success —
117    /// the local copy is not removed until this is true.
118    pub verified_at_destination: bool,
119    /// Whether the local file was unlinked afterwards.
120    pub local_copy_removed: bool,
121}
122
123/// **Retire a pensioned generation to holger instead of deleting it. NOT
124/// IMPLEMENTED — deliberately.**
125///
126/// This is the alternative ending to [`NewGeneration`]. That implementation
127/// seals a new generation, proves it, and then unlinks the old one (step 5). The
128/// old one is not worthless: it is a complete, sealed, self-describing archive
129/// of a real state of a real repository. Shipping it to holger — nordisk's
130/// artifact/package server — keeps it as an artifact rather than destroying it,
131/// and the local disk is freed either way.
132///
133/// # What makes this viable, which is the part worth recording now
134///
135/// * **A sealed archive never changes.** That is the whole property. Retiring
136///   one is therefore a **copy and a delete**, not a migration: there is no
137///   schema to translate, no writer to quiesce, no reader to redirect
138///   mid-flight, and no state at the destination that can go stale relative to
139///   the source. An artifact store wants exactly this shape — immutable,
140///   self-contained, addressable by digest — and a sealed `.znippy` already is
141///   it.
142/// * **It can be verified at the destination before the local copy goes.** The
143///   archive carries its own index and a blake3 per chunk, so holger can read
144///   back what it received and prove it, with no help from here. The ordering is
145///   the same one [`NewGeneration`] already uses and for the same reason: ship,
146///   verify, *then* delete. There is never a moment when the only copy is the
147///   one in flight.
148/// * **It composes with GC rather than replacing it.** The generation being
149///   retired has already been superseded by a proven newer one, so nothing is
150///   serving from it. Retirement is not on any read path.
151///
152/// # What is deliberately NOT here
153///
154/// No client, no connection, no holger dependency, no digest algorithm choice,
155/// no naming scheme at the destination. Those are the decisions this stub is
156/// holding a place for. [`tests::retiring_to_holger_is_deliberately_empty`]
157/// asserts the emptiness so a half-implementation cannot pass for a finished
158/// one.
159///
160/// # Panics
161///
162/// Always.
163pub fn retire_to_holger(_generation: &Path, _target: &HolgerTarget) -> Result<RetirementReceipt> {
164    todo!(
165        "retire a pensioned generation to holger: deliberately empty. The contract is settled \
166         (ship the sealed archive, have holger verify it, only then unlink the local copy); \
167         the transport, the digest and the destination naming are not. Do not add a holger \
168         client from this file without deciding them first."
169    )
170}
171
172// ── Impl A — CompactInPlace ───────────────────────────────────────────────────
173
174/// **Delegate to znippy's own compaction, in place.**
175///
176/// A thin wrapper over [`znippy_common::compact_archive`] and deliberately
177/// nothing more: that function is the one that knows how to copy a live chunk
178/// without touching its codec, and a second copy of that loop in this crate
179/// would be a second thing to keep correct.
180///
181/// The archive keeps its name. An interruption leaves the original serving,
182/// which is `compact_archive`'s own contract (stage beside, one atomic rename),
183/// asserted in `tests/gc_crash.rs` by killing a real process mid-copy.
184///
185/// It cannot verify before committing: by the time there is a compacted file to
186/// read, the rename has already replaced the original. The post-hoc check this
187/// runs is therefore a report, not a gate — use [`NewGeneration`] when the
188/// difference matters, which is normally.
189#[derive(Debug, Clone, Copy)]
190pub struct CompactInPlace {
191    /// Read the result back and blake3-check it after the rename. On by default;
192    /// it is a full pass over the archive, so a caller compacting a very large
193    /// archive on a timer may turn it off.
194    pub verify: bool,
195}
196
197impl Default for CompactInPlace {
198    /// `verify: true`. Written out rather than derived, because a derived
199    /// `Default` would be `false` — the safe-looking spelling giving the less
200    /// safe object.
201    fn default() -> Self {
202        Self { verify: true }
203    }
204}
205
206impl CompactInPlace {
207    /// With the post-hoc verification on.
208    pub fn new() -> Self {
209        Self::default()
210    }
211}
212
213impl Gc for CompactInPlace {
214    fn run(&self, archive: &Path) -> Result<GcReport> {
215        let CompactReport {
216            bytes_before,
217            bytes_after,
218            rows,
219            delta_rows,
220        } = compact_archive(archive)
221            .with_context(|| format!("compacting {} in place", archive.display()))?;
222
223        let verified = if self.verify {
224            read_back_every_entry(archive).with_context(|| {
225                format!(
226                    "verifying {} after an in-place compaction — the original is already gone, \
227                     which is exactly the window NewGeneration closes",
228                    archive.display()
229                )
230            })?;
231            true
232        } else {
233            false
234        };
235
236        Ok(GcReport {
237            strategy: self.name(),
238            archive: archive.to_path_buf(),
239            retired: None,
240            bytes_before,
241            bytes_after,
242            rows,
243            delta_rows,
244            verified,
245            retired_packs: 0,
246        })
247    }
248
249    fn name(&self) -> &'static str {
250        "CompactInPlace"
251    }
252}
253
254// ── Impl B — NewGeneration ────────────────────────────────────────────────────
255
256/// Where a [`NewGeneration`] run may be stopped, for the crash-safety guards.
257///
258/// Stopping returns an `Err` and leaves the filesystem in **exactly** the state a
259/// process death at that point would leave it — nothing is unwound. That is the
260/// point: the test then asserts what a reader finds.
261#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
262pub enum StopAfter {
263    #[default]
264    Never,
265    /// The work copy has been compacted; nothing has been verified or renamed.
266    Compact,
267    /// The new generation has been verified but not yet renamed into place.
268    Verify,
269    /// The new generation is in place under its own name; the old archive has
270    /// **not** been unlinked. Both are on disk and both are readable.
271    Rename,
272}
273
274/// **GC into a new generation, keeping the old one until the new one is proven.**
275///
276/// `repo.znippy` becomes `repo.g1.znippy`, then `repo.g2.znippy`, and so on. The
277/// old file is unlinked only after the new one has been read back in full.
278///
279/// The sequence, and why each step is where it is:
280///
281/// 1. **`link(archive, work)`** — a hard link, so the work name and the archive
282///    name are the same inode and not one byte is copied. This is what lets step
283///    2 be `compact_archive` verbatim (LAW 5) while the archive's own name stays
284///    untouched: `compact_archive` renames over the name it was given, and the
285///    name it is given here is the work link, not the archive.
286/// 2. **`compact_archive(work)`** — stages beside `work` and renames over it.
287///    The archive's inode is still fully referenced by the archive's own name
288///    throughout, so every reader of `repo.znippy` is unaffected.
289/// 3. **verify** — [`verify_archive_integrity`] over the new file, plus a
290///    manifest set comparison against the original. Both are read back off disk.
291/// 4. **`rename(work, repo.gN.znippy)`** — atomic; the new generation now has
292///    its permanent name.
293/// 5. **`unlink(repo.znippy)`** — the old index and old data go, and only now.
294///
295/// A death at any point leaves at least one complete, readable archive:
296/// before 4 the original under its own name (plus debris), after 4 and before 5
297/// both of them, after 5 the new generation. Asserted, per step, in
298/// [`tests::an_interruption_at_every_step_leaves_a_readable_archive`].
299#[derive(Debug, Clone, Copy, Default)]
300pub struct NewGeneration {
301    /// Test-only interruption point. [`StopAfter::Never`] in every real run.
302    pub stop_after: StopAfter,
303}
304
305impl NewGeneration {
306    pub fn new() -> Self {
307        Self::default()
308    }
309
310    /// Stop the run at `stop`, leaving the filesystem as a crash there would.
311    pub fn stopping_after(stop: StopAfter) -> Self {
312        Self { stop_after: stop }
313    }
314}
315
316/// The default GC. B, because it is the one that verifies before it deletes.
317pub fn default_gc() -> NewGeneration {
318    NewGeneration::new()
319}
320
321/// `repo.znippy` → `repo.g1.znippy` → `repo.g2.znippy` → …
322///
323/// The generation lives in the stem rather than in the extension so the file is
324/// still a `.znippy` to everything that dispatches on extension. A stem that
325/// already ends in `.gN` is advanced; anything else starts at `g1`.
326pub fn next_generation(archive: &Path) -> Result<PathBuf> {
327    let name = archive
328        .file_name()
329        .and_then(|n| n.to_str())
330        .ok_or_else(|| anyhow!("{} has no usable file name", archive.display()))?;
331    let (stem, ext) = match name.rsplit_once('.') {
332        Some((s, e)) if !s.is_empty() => (s, Some(e)),
333        _ => (name, None),
334    };
335    let next = match stem.rsplit_once('.') {
336        Some((head, marker)) if is_generation(marker) => {
337            format!("{head}.g{}", marker[1..].parse::<u64>().unwrap_or(0) + 1)
338        }
339        _ => format!("{stem}.g1"),
340    };
341    let file = match ext {
342        Some(e) => format!("{next}.{e}"),
343        None => next,
344    };
345    Ok(archive.with_file_name(file))
346}
347
348fn is_generation(s: &str) -> bool {
349    s.len() > 1 && s.starts_with('g') && s[1..].bytes().all(|b| b.is_ascii_digit())
350}
351
352/// Read every entry back through the ordinary reader, reconstructing delta
353/// chains and checking each entry's blake3 against the index. Returns the bytes
354/// that came out.
355///
356/// This is the expensive half of a GC and it is the half that makes the cheap
357/// half safe: it is one full decode pass over the archive.
358pub(crate) fn read_back_every_entry(archive: &Path) -> Result<u64> {
359    let manifest = manifest(archive)?;
360    let ar = ZnippyArchive::open(archive)
361        .with_context(|| format!("opening {} to verify it", archive.display()))?;
362    let mut bytes = 0u64;
363    for (path, expected) in &manifest {
364        let got = ar.extract_file_verified(path).with_context(|| {
365            format!(
366                "{} does not read back {path} — the new generation is not usable",
367                archive.display()
368            )
369        })?;
370        if got.len() as u64 != *expected {
371            bail!(
372                "{}: {path} reads back {} bytes, the index says {expected}",
373                archive.display(),
374                got.len()
375            );
376        }
377        bytes += got.len() as u64;
378    }
379    Ok(bytes)
380}
381
382/// The manifest as a comparable set: path and uncompressed size, sorted.
383fn manifest(archive: &Path) -> Result<Vec<(String, u64)>> {
384    let mut m: Vec<(String, u64)> = get_all_files_meta(archive)
385        .with_context(|| format!("listing {}", archive.display()))?
386        .into_iter()
387        .map(
388            |ArtifactMeta {
389                 relative_path,
390                 uncompressed_size,
391                 ..
392             }| { (relative_path, uncompressed_size) },
393        )
394        .collect();
395    m.sort();
396    Ok(m)
397}
398
399fn unique_sibling(archive: &Path, tag: &str) -> PathBuf {
400    let unique = std::time::SystemTime::now()
401        .duration_since(std::time::UNIX_EPOCH)
402        .map(|d| d.as_nanos())
403        .unwrap_or(0);
404    let mut p = archive.as_os_str().to_owned();
405    p.push(format!(".{tag}-{}-{unique}", std::process::id()));
406    PathBuf::from(p)
407}
408
409/// fsync the directory, so the rename and the unlink are durable and not just
410/// visible. Best-effort: a filesystem that refuses to open a directory is not a
411/// reason to fail a GC that has otherwise succeeded.
412fn sync_dir(path: &Path) {
413    if let Some(parent) = path.parent()
414        && let Ok(f) = std::fs::File::open(parent)
415    {
416        let _ = f.sync_all();
417    }
418}
419
420impl Gc for NewGeneration {
421    fn run(&self, archive: &Path) -> Result<GcReport> {
422        let bytes_before = std::fs::metadata(archive)
423            .with_context(|| format!("stat {}", archive.display()))?
424            .len();
425        let before = manifest(archive)?;
426        let target = next_generation(archive)?;
427        if target.exists() {
428            bail!(
429                "{} already exists — a previous GC left a generation behind, or two are running \
430                 at once",
431                target.display()
432            );
433        }
434
435        // 1. A second name for the same inode. No bytes move; the archive's own
436        //    name keeps pointing at the original for the whole compaction.
437        let work = unique_sibling(archive, "gc");
438        std::fs::hard_link(archive, &work).with_context(|| {
439            format!(
440                "hard-linking {} to {} — a new generation is produced by compacting a second \
441                 name for the same inode, which needs both on one filesystem",
442                archive.display(),
443                work.display()
444            )
445        })?;
446
447        // 2. znippy's own compaction, verbatim, against the work name.
448        let compacted = compact_archive(&work);
449        let CompactReport {
450            bytes_after,
451            rows,
452            delta_rows,
453            ..
454        } = match compacted {
455            Ok(r) => r,
456            Err(e) => {
457                let _ = std::fs::remove_file(&work);
458                return Err(e.context(format!(
459                    "compacting a new generation of {}",
460                    archive.display()
461                )));
462            }
463        };
464        if self.stop_after == StopAfter::Compact {
465            bail!("interrupted after compact (test)");
466        }
467
468        // 3. Read the result back before anything is committed. On any failure
469        //    the work copy goes and the original is untouched — that is the
470        //    whole difference from A.
471        if let Err(e) = read_back_every_entry(&work) {
472            let _ = std::fs::remove_file(&work);
473            return Err(e.context(format!(
474                "the new generation of {} did not verify — nothing was replaced",
475                archive.display()
476            )));
477        }
478        let after = manifest(&work)?;
479        if after != before {
480            let _ = std::fs::remove_file(&work);
481            bail!(
482                "the new generation of {} carries {} entries, the original {} — nothing was \
483                 replaced",
484                archive.display(),
485                after.len(),
486                before.len()
487            );
488        }
489        if self.stop_after == StopAfter::Verify {
490            bail!("interrupted after verify (test)");
491        }
492
493        // 4. The new generation takes its permanent name.
494        std::fs::rename(&work, &target)
495            .with_context(|| format!("naming the new generation {}", target.display()))?;
496        sync_dir(&target);
497        if self.stop_after == StopAfter::Rename {
498            bail!("interrupted after rename (test)");
499        }
500
501        // 5. Only now: the old index and the old data.
502        std::fs::remove_file(archive).with_context(|| format!("retiring {}", archive.display()))?;
503        sync_dir(archive);
504
505        Ok(GcReport {
506            strategy: self.name(),
507            archive: target,
508            retired: Some(archive.to_path_buf()),
509            bytes_before,
510            bytes_after,
511            rows,
512            delta_rows,
513            verified: true,
514            retired_packs: 0,
515        })
516    }
517
518    fn name(&self) -> &'static str {
519        "NewGeneration"
520    }
521}
522
523#[cfg(test)]
524mod tests {
525    use super::*;
526    use znippy_common::{
527        SupersedeOutcome, ZnippyArchive, ZnippyReader, create_archive, read_delta_map,
528        supersede_as_delta,
529    };
530
531    /// Four generations of a pack, three of them superseded into deltas, so the
532    /// archive holds real dead payload for a GC to find. Modelled on
533    /// znippy-common's own `compaction_reclaims_the_superseded_blob` fixture.
534    fn fixture(dir: &Path, name: &str) -> (PathBuf, Vec<Vec<u8>>) {
535        let archive = dir.join(name);
536        let mut st = 0x5151_2323_abcd_ef01u64;
537        let base: Vec<u8> = (0..600_000u32)
538            .map(|_| {
539                st = st.wrapping_add(0x9e37_79b9_7f4a_7c15);
540                let mut z = st;
541                z = (z ^ (z >> 30)).wrapping_mul(0xbf58_476d_1ce4_e5b9);
542                (z ^ (z >> 27)) as u8
543            })
544            .collect();
545        let mut gens: Vec<Vec<u8>> = vec![base];
546        for g in 1..4 {
547            let mut next = gens[g - 1].clone();
548            next.extend_from_slice(format!("generation {g} tail ").repeat(300).as_bytes());
549            gens.push(next);
550        }
551        let files: Vec<(String, Vec<u8>)> = gens
552            .iter()
553            .enumerate()
554            .map(|(i, b)| (format!("pack-{i}.pack"), b.clone()))
555            .collect();
556        create_archive(&archive, &files, 3).unwrap();
557        for i in (0..3).rev() {
558            let out = supersede_as_delta(
559                &archive,
560                &format!("pack-{i}.pack"),
561                &format!("pack-{}.pack", i + 1),
562                3 - 1 - i,
563                3,
564            )
565            .unwrap();
566            assert!(
567                matches!(out, SupersedeOutcome::Delta { .. }),
568                "gen {i}: {out:?}"
569            );
570        }
571        (archive, gens)
572    }
573
574    fn reads_back(archive: &Path, gens: &[Vec<u8>]) {
575        let ar = ZnippyArchive::open(archive)
576            .unwrap_or_else(|e| panic!("{} does not open: {e}", archive.display()));
577        for (i, want) in gens.iter().enumerate() {
578            assert_eq!(
579                &ar.extract_file(&format!("pack-{i}.pack")).unwrap(),
580                want,
581                "{} lost generation {i}",
582                archive.display()
583            );
584        }
585    }
586
587    /// Both implementations reclaim the dead payload and neither changes a byte
588    /// any entry reads back as — delta chunks at their original depth included.
589    ///
590    /// Asserted on applied output: the file size on disk, and every generation
591    /// extracted and compared byte for byte through the ordinary reader.
592    ///
593    /// Seen RED by having `NewGeneration::run` `rename(archive, target)` instead
594    /// of renaming the compacted work file — i.e. publishing the *old* file
595    /// under the new generation's name: "called `Result::unwrap()` on an `Err`
596    /// value: retiring /tmp/…/NewGeneration.znippy … No such file or directory
597    /// (os error 2)" — the rename had already consumed the name step 5 goes on
598    /// to unlink. Restored.
599    #[test]
600    fn both_implementations_reclaim_the_dead_payload_and_change_no_entry() {
601        let dir = tempfile::tempdir().unwrap();
602        let raw = |gens: &[Vec<u8>]| gens.iter().map(|g| g.len() as u64).sum::<u64>();
603
604        for gc in [
605            &CompactInPlace::new() as &dyn Gc,
606            &NewGeneration::new() as &dyn Gc,
607        ] {
608            let (archive, gens) = fixture(dir.path(), &format!("{}.znippy", gc.name()));
609            let before = std::fs::metadata(&archive).unwrap().len();
610            let report = gc.run(&archive).unwrap();
611
612            assert_eq!(report.strategy, gc.name());
613            assert_eq!(
614                report.rows,
615                4,
616                "{}: a GC must not change the row count",
617                gc.name()
618            );
619            assert_eq!(
620                report.delta_rows,
621                3,
622                "{}: the delta map must travel",
623                gc.name()
624            );
625            assert_eq!(report.bytes_before, before);
626            let on_disk = std::fs::metadata(&report.archive).unwrap().len();
627            assert_eq!(
628                report.bytes_after,
629                on_disk,
630                "{}: reported size is not the size",
631                gc.name()
632            );
633            assert!(
634                on_disk * 2 < raw(&gens),
635                "{}: the new archive is {on_disk} bytes for {} bytes of generations — the dead \
636                 payload was not reclaimed",
637                gc.name(),
638                raw(&gens)
639            );
640
641            reads_back(&report.archive, &gens);
642            let map = read_delta_map(&report.archive).unwrap();
643            assert!(
644                map.iter().all(|(p, _, _)| p != "pack-3.pack"),
645                "{}: the live generation went behind a link: {map:?}",
646                gc.name()
647            );
648        }
649    }
650
651    /// The difference between A and B, as an assertion rather than a claim: A
652    /// keeps the name and has no retired file; B produces `x.g1.znippy`, retires
653    /// `x.znippy`, and the retired name is gone from the filesystem.
654    ///
655    /// Seen RED by removing step 5 (the `remove_file(archive)`), so the old
656    /// generation survived its replacement: "B kept the old generation after
657    /// proving the new one". Restored.
658    #[test]
659    fn a_renames_over_the_original_and_b_produces_a_new_generation() {
660        let dir = tempfile::tempdir().unwrap();
661
662        let (a_path, gens) = fixture(dir.path(), "a.znippy");
663        let a = CompactInPlace::new().run(&a_path).unwrap();
664        assert_eq!(a.archive, a_path, "A moved the archive");
665        assert_eq!(a.retired, None);
666        assert!(a_path.exists(), "A removed the archive it compacted");
667        reads_back(&a_path, &gens);
668
669        let (b_path, gens) = fixture(dir.path(), "b.znippy");
670        let b = NewGeneration::new().run(&b_path).unwrap();
671        assert_eq!(b.archive, dir.path().join("b.g1.znippy"));
672        assert_eq!(b.retired, Some(b_path.clone()));
673        assert!(
674            !b_path.exists(),
675            "B kept the old generation after proving the new one"
676        );
677        assert!(b.archive.exists());
678        reads_back(&b.archive, &gens);
679        assert!(b.verified, "B must not report an unverified success");
680
681        // And again, from the new generation: g1 → g2.
682        let b2 = NewGeneration::new().run(&b.archive).unwrap();
683        assert_eq!(b2.archive, dir.path().join("b.g2.znippy"));
684        assert!(!b.archive.exists());
685        reads_back(&b2.archive, &gens);
686    }
687
688    /// **A crash at any step of B leaves a complete, readable archive.**
689    ///
690    /// Not simulated debris: the run really stops at that step and nothing is
691    /// unwound, so the filesystem is in the state a `kill -9` there produces.
692    /// Asserted on applied output — every generation extracted and compared byte
693    /// for byte from whichever archive is supposed to be serving.
694    ///
695    /// Seen RED by moving the `remove_file(archive)` to just after the hard link
696    /// (i.e. retiring the old name before the new one exists): at
697    /// `StopAfter::Compact` the assertion fired with "compact: the original was
698    /// unlinked early" — the very window this ordering exists to close.
699    /// Restored.
700    #[test]
701    fn an_interruption_at_every_step_leaves_a_readable_archive() {
702        let dir = tempfile::tempdir().unwrap();
703
704        for (step, label) in [
705            (StopAfter::Compact, "compact"),
706            (StopAfter::Verify, "verify"),
707            (StopAfter::Rename, "rename"),
708        ] {
709            let (archive, gens) = fixture(dir.path(), &format!("{label}.znippy"));
710            let target = next_generation(&archive).unwrap();
711            let err = NewGeneration::stopping_after(step)
712                .run(&archive)
713                .expect_err("the interruption did not stop the run");
714            assert!(err.to_string().contains(label), "wrong stop: {err}");
715
716            // The original is still there and still serves, at every step.
717            assert!(archive.exists(), "{label}: the original was unlinked early");
718            reads_back(&archive, &gens);
719
720            match step {
721                StopAfter::Rename => {
722                    // After the rename both names exist and both serve. That is
723                    // the only step where two complete archives are on disk.
724                    assert!(target.exists(), "the renamed generation is missing");
725                    reads_back(&target, &gens);
726                }
727                _ => assert!(
728                    !target.exists(),
729                    "{label}: a generation took its permanent name before it was proven"
730                ),
731            }
732        }
733    }
734
735    /// A stale generation left by an earlier interruption is refused rather than
736    /// silently overwritten — the one case where B could destroy a good archive.
737    ///
738    /// Seen RED by deleting the `target.exists()` check: the run succeeded and
739    /// overwrote the debris — "overwrote a generation: GcReport { strategy:
740    /// \"NewGeneration\", archive: \"…/x.g1.znippy\", retired: Some(\"…/x.znippy\"),
741    /// bytes_before: 2440284, bytes_after: 624084, rows: 4, delta_rows: 3,
742    /// verified: true }". Had the debris been a *real* earlier generation, it
743    /// would have gone the same way. Restored.
744    #[test]
745    fn an_existing_generation_is_never_overwritten() {
746        let dir = tempfile::tempdir().unwrap();
747        let (archive, gens) = fixture(dir.path(), "x.znippy");
748        let target = next_generation(&archive).unwrap();
749
750        // Debris from an interrupted earlier run, in the place the next
751        // generation wants.
752        std::fs::write(&target, b"not an archive").unwrap();
753
754        let err = NewGeneration::new()
755            .run(&archive)
756            .expect_err("overwrote a generation");
757        assert!(
758            err.to_string().contains("already exists"),
759            "wrong error: {err}"
760        );
761        assert_eq!(std::fs::read(&target).unwrap(), b"not an archive");
762        reads_back(&archive, &gens);
763    }
764
765    /// **The holger retirement is asserted to be empty**, so it cannot be
766    /// mistaken for an oversight. If this starts failing, `retire_to_holger`
767    /// grew a body — which is the intended future. Delete this test in the same
768    /// commit that does it, on purpose.
769    ///
770    /// Seen RED by replacing the `todo!()` with `Ok(RetirementReceipt { … })`:
771    /// "the holger retirement no longer panics — if it was implemented, delete
772    /// this test in the same commit: ()". Restored.
773    #[test]
774    fn retiring_to_holger_is_deliberately_empty() {
775        let target = HolgerTarget {
776            endpoint: "unspecified".into(),
777            repository: "nordisk/znippy".into(),
778        };
779        let outcome = std::panic::catch_unwind(|| {
780            let _ = retire_to_holger(Path::new("/nonexistent.g1.znippy"), &target);
781        });
782        let payload = outcome.expect_err(
783            "the holger retirement no longer panics — if it was implemented, delete this test in \
784             the same commit",
785        );
786        let msg = payload
787            .downcast_ref::<String>()
788            .cloned()
789            .or_else(|| payload.downcast_ref::<&str>().map(|s| s.to_string()))
790            .unwrap_or_default();
791        assert!(
792            msg.contains("deliberately empty"),
793            "it panics, but not as the placeholder: {msg}"
794        );
795    }
796
797    /// The generation names advance, and a `.znippy` stays a `.znippy`.
798    ///
799    /// Seen RED by making `is_generation` always false, so a marker is never
800    /// recognised and never advances: "assertion `left == right` failed / left:
801    /// \"/srv/repo.g1.g1.znippy\" / right: \"/srv/repo.g2.znippy\"". Restored.
802    #[test]
803    fn generation_names_advance_in_the_stem() {
804        let n = |s: &str| {
805            next_generation(Path::new(s))
806                .unwrap()
807                .to_string_lossy()
808                .into_owned()
809        };
810        assert_eq!(n("/srv/repo.znippy"), "/srv/repo.g1.znippy");
811        assert_eq!(n("/srv/repo.g1.znippy"), "/srv/repo.g2.znippy");
812        assert_eq!(n("/srv/repo.g9.znippy"), "/srv/repo.g10.znippy");
813        // Not a generation marker: `git` is not `g<digits>`.
814        assert_eq!(n("/srv/repo.git.znippy"), "/srv/repo.git.g1.znippy");
815        assert_eq!(n("/srv/repo"), "/srv/repo.g1");
816    }
817}