Expand description
One account purge at a time, and the lockout always comes off. The
Accounts.Purge queue; see its module docs for why it is its own line and
not the gc one.
One account purge at a time, and the lockout always comes off.
Accounts.Purge empties a user’s whole account: every repository, every
soft-delete tombstone still holding bytes, every orphan store under the
account’s directory subtree, and the namespace rows themselves. That is
minutes of pure unlink I/O on a large account, so it is queued rather than
run inline, for exactly the reason crate::gc_queue gives for a gc.
§Why this is a SECOND LINE and not more jobs on the gc one
The mechanism is shared — both kinds run on crate::job_queue, which is
one worker, one FIFO, dedup by subject and a finished job kept for a poller.
What is not shared is the line, and that is the ruling this module has
always argued for. It survives the extraction unchanged:
A purge holds a total lockout on a customer’s account for the whole time it is queued. A gc holds nothing — it is maintenance, it is re-derivable, and it can wait behind anything without costing anybody anything. Put them in one queue and a user is locked out of their account while the box works through an hour of somebody else’s repository maintenance, because a gc that arrived first is ahead of them. The queue a job waits in is part of what the job costs, and these two jobs cost different things while waiting.
So CoreControlPlane holds two crate::job_queue::JobQueues and each has
its own jera lane. What used to be two copies of the queue is now two
instances of one, which is the same two lines and half the code.
§The property this module exists for
The gate goes on before the job is enqueued, in the RPC handler, so
there is no window in which a purge is accepted and the user can still push.
That is deliberately not the queue’s business — the queue cannot promise
it, because the promise is about what is true when Accounts.Purge
returns, which is before the queue has done anything at all. The gate is
taken by the caller, under an id the caller takes from
PurgeQueue::next_job_id first.
It comes off here, in JobKind::settle, and it comes off on every
path out of the runner:
- the runner returns
Ok→ released; - the runner returns
Err→ released, and the job is FAILED with the reason and the list of stores still on the disk; - the runner panics → caught, released, the job is FAILED saying so, and the worker survives. An unwinding worker takes the queue with it, so the account after the panicking one would sit QUEUED for ever behind a thread that no longer exists — one account’s bug bricking the next one’s.
The catch and the always-runs settle are now crate::job_queue’s, which
is the whole gain from the extraction: a gc, and a mirror, and anything
after them get the same two guarantees without a second person having to
rediscover why they matter.
What this module cannot cover is the process dying between two unlinks.
That is what the lease in gunnar_core::gate is for: a purge’s gate carries
a deadline, the runner re-stamps it as it goes, and a gate whose deadline
has passed is not a gate. Belt here, braces there, because the outcome they
both prevent — an account shut by a row nobody remembers, held by a process
that is gone — is worse than an unfinished purge. The boot sweep that
releases every lease a killed process left behind is
CoreControlPlane::release_stale_purge_leases, called once at start-up: it
is about rows in the identity table and not about jobs, so it is not the
queue’s and did not move here.
§What this module is not
It is not the purge. That is [crate::admin::purge_account], reached
through a runner handed in at construction, so the queue can be tested with
no disk under it and there is one implementation of what emptying an account
means (one implementation). The release is handed in the same way and is
AuthTable::lift_gate_held_by in the real server — held_by, so a purge
finishing cannot lift a suspension an operator put on while it ran.
Structs§
- Purge
- The purge kind. What a purge is, and — the part a gc has no equivalent of — what a purge has to give back when it ends, however it ends.
- Purge
Order - What a purge was asked for, carried on the job so the worker can hand it back to the runner minutes later.
- Purge
Queue - The queue: one worker, one purge at a time, FIFO, deduplicated by account.
- Purge
Report - What a purge found, reclaimed, and gave back.
Constants§
- FINISHED_
KEPT - Finished jobs kept regardless of age, most recent first.
- FINISHED_
RETENTION - How long a finished purge stays readable. The same ten minutes
crate::gc_queue::FINISHED_RETENTIONuses, and for the same reason: far longer than any console poll, far shorter than a session.
Type Aliases§
- Gate
Release - How the lockout this job took is released.
- Progress
- The handle a running purge reports through.
- Purge
Job - One account purge, running or completed.
- Purge
Runner - What a purge actually is, handed in so this module does not have to know.
- Purge
State - Where a purge is.
JobState’s four words, shared with every other kind of job soPurgeStatusandGcStatuscannot drift apart about whatDonemeans.