Expand description
Garbage collection: one trait, two implementations
(gc::CompactInPlace in place, gc::NewGeneration into a new
generation — the default).
Garbage collection, as a trait with two implementations.
Both reclaim the same thing: the payload of entries nothing points at any
more. supersede_as_delta replaces a generation’s bytes with a delta and
drops its index rows, but the old blob stays in the file, unreferenced —
measured in znippy-common, an 8 056 624-byte archive went to 8 057 731 after
4 000 000 bytes of live payload became 11. Reclaiming that is what a GC is
here for.
§Neither implementation copies a git object through a codec
Both route the copy through
znippy_common::compact_archive, which
copies every live chunk’s on-disk bytes verbatim: no decode, no
re-encode, no delta recomputation, compressed / uncompressed_size /
blake3 carried unchanged, and a delta chunk staying a delta chunk at the same
depth. Nothing in this module reimplements that loop (LAW 5); the two
implementations differ only in where the result ends up.
§The two, and the one real difference between them
[CompactInPlace] (A) | [NewGeneration] (B, the default) | |
|---|---|---|
| result | the same path, rewritten | a new file, x.znippy → x.g1.znippy |
| commit point | rename(staged, archive) | rename(work, x.g1.znippy), then unlink(x.znippy) |
| verified before committing? | no — the rename has already happened | yes — the new generation is read back in full first |
| interrupted at any byte | the original is still there, dead payload and all | the original is still there, and so is the debris |
§Crash safety, by construction, in both
Nothing is ever written over live data. A: compact_archive stages a
*.compact-<pid>-<nanos> file beside the destination and the only mutation
of the archive’s own name is one rename(2), which is atomic — so an
interruption at any byte leaves the original serving and a stray staged file
behind. B: the compaction runs against a hard link to the original inode,
so the original name is untouched for the whole run; the new generation is
then verified, renamed into place, and only after that is the old name
unlinked. There is no window in either where a reader finds no archive.
B’s extra guarantee is the one A structurally cannot give: A commits by rename before anything has read the result back, so if the compacted file were bad, the good one is already gone. B verifies first and deletes last. That is why B is the default.
§The third ending, not yet written
B’s last step deletes the superseded generation. [retire_to_holger] is the
alternative: ship it to nordisk’s artifact server instead, because a sealed
archive is already the shape an artifact store wants. That function is
deliberately empty — signature and contract only. Read its doc comment for
why the shape works; do not add a client to this file.
§What “verified” means here
Every entry is read back through the ordinary reader with
ZnippyArchive::extract_file_verified — delta chains reconstructed, blake3
checked against the index — plus a set comparison of the manifest
(relative_path and uncompressed_size) against the original, because an
archive that verifies against its own index but has lost an entry is
internally consistent and still wrong.
It is deliberately not znippy_common::verify_archive_integrity, and
that is a finding rather than a preference: on the fixture in
[tests::both_implementations_reclaim_the_dead_payload_and_change_no_entry]
— four generations, three of them superseded into deltas — that function
reports “3 corrupt entries of 4” on a perfectly good archive, because its
decompress_archive path does not reconstruct delta chunks. It cannot be
used to gate a GC of any archive supersede_as_delta has touched, which is
every archive a GC is worth running on.
Structs§
- Compact
InPlace - Delegate to znippy’s own compaction, in place.
- GcReport
- What one GC run did. Owned by the
git-storage-traitcontract; re-exported here socrate::gc::GcReportstays a valid path. Every field is measured off the filesystem or off the archive after the fact — see the trait crate for per-field docs, and noteretired_packsis always0from a bareGc::run:GitOps::gcfills it in. What one GC run did. Every field is measured off the store after the fact — nothing here is an input echoed back. - Holger
Target - Where a pensioned generation goes. Deliberately just an address and a name: what the endpoint means is the transport’s business, and no transport is decided.
- NewGeneration
- GC into a new generation, keeping the old one until the new one is proven.
- Retirement
Receipt - What a completed retirement reports. Every field is about the archive that moved; nothing here describes an index, because a sealed archive carries its own and no index is shipped separately.
Enums§
- Stop
After - Where a
NewGenerationrun may be stopped, for the crash-safety guards.
Traits§
- Gc
- Reclaim the dead payload in a znippy archive.
Functions§
- default_
gc - The default GC. B, because it is the one that verifies before it deletes.
- next_
generation repo.znippy→repo.g1.znippy→repo.g2.znippy→ …- retire_
to_ holger - Retire a pensioned generation to holger instead of deleting it. NOT IMPLEMENTED — deliberately.