Expand description
One gc at a time, for the whole box — the queue behind Repos.Gc.
One gc at a time, for the whole box.
Repos.Gc exists so a console button can maintain a repository. The moment
it exists, ten repository owners can press ten buttons inside one second,
and a gc is not cheap: it seals the live logs, compacts the archive, and
unlinks the entire lazily-exploded read cache — hundreds of thousands of
files on a large repository, all of it I/O. Ten of those at once is not ten
times the work, it is a box that has stopped answering.
§Why the server and not the caller
A console can serialise itself. It cannot serialise the CLI, a second console, a script, or the next client anyone writes — and every one of those holds a credential that reaches the same disk. A rule that only one participant obeys is not an invariant; it is a convention that holds until the first person who did not read it. The server is the only place that sees every request, so it is the only place the rule can actually live.
§The two properties, and why they are tested rather than argued
- Never two at once. A single jera lane takes jobs off one queue, one at a time.
The queue counts the runs that were in flight together and remembers the
peak, so
tests/gc_queue.rsasserts on an observed maximum rather than on a sleep that happened not to overlap. Seen red against a thread-per-request queue, which reported a peak of 10 against ten requests. - A double press is one job. A request naming a repository that is already queued or running returns the job that is already there. Impatient double-clicking is the normal case for a button, not the exception, and a queue that took it literally would run the expensive thing twice for no reason. Asserted on the number of times the runner was actually CALLED, which a queue that merely deduplicated its bookkeeping could not fake.
Both properties now live in crate::job_queue and are asserted here, on
GcQueue, unchanged: a property proved about the runner is a property of
every kind on it, but tests/gc_queue.rs is what says it holds for this
door, and that file did not move.
§What a finished job is for
The worker does not drop a job when it finishes it. A console polls, and a
poller that was between requests at the instant the gc completed would
otherwise get “no such job” — indistinguishable from “you never asked”. So a
completed job is kept for FINISHED_RETENTION, and beyond that a bounded
number of the most recent are kept anyway, so a busy server forgets by age
and a quiet one does not forget at all.
§What this module is, and is not
It is not the maintenance. That is crate::admin::gc_one, which the CLI
verb also calls, and this module reaches it through a runner handed in at
construction (one implementation: one writer, and a queue that could be tested without a
disk under it). The runner is the only thing here that knows what a gc is.
It is not the queue either, any more. The jera lane, the FIFO, the
dedup, the retention and the instruments are crate::job_queue, shared
with the purge queue and with anything added later; what is left here is
the gc’s answers to that runner’s questions — what a gc’s subject is (a
repository path), what a gc produces (bytes), and what a gc is (the runner).
One thing a gc gained by moving: the worker now catches a panicking gc and
reports it as a FAILED job instead of dying with it, which the purge queue
had paid for separately and the gc queue never had.
Structs§
- Gc
- The gc kind. Everything the shared runner needs to know about a gc, and nothing else.
- GcQueue
- The queue: one worker, one job at a time, FIFO, deduplicated by repository.
- GcReport
- What a gc produced.
Constants§
- FINISHED_
KEPT - Finished jobs kept regardless of age, most recent first.
- FINISHED_
RETENTION - How long a finished job stays readable. Ten minutes is far longer than any console poll interval and far shorter than a session, so a button that completed is still reporting its bytes when whoever pressed it looks back, and a server that has run thousands is not holding all of them.