Skip to main content

Module purge_queue

Module purge_queue 

Source
Expand description

One account purge at a time, and the lockout always comes off. The Accounts.Purge queue; see its module docs for why it is its own line and not the gc one. One account purge at a time, and the lockout always comes off.

Accounts.Purge empties a user’s whole account: every repository, every soft-delete tombstone still holding bytes, every orphan store under the account’s directory subtree, and the namespace rows themselves. That is minutes of pure unlink I/O on a large account, so it is queued rather than run inline, for exactly the reason crate::gc_queue gives for a gc.

§Why this is a SECOND LINE and not more jobs on the gc one

The mechanism is shared — both kinds run on crate::job_queue, which is one worker, one FIFO, dedup by subject and a finished job kept for a poller. What is not shared is the line, and that is the ruling this module has always argued for. It survives the extraction unchanged:

A purge holds a total lockout on a customer’s account for the whole time it is queued. A gc holds nothing — it is maintenance, it is re-derivable, and it can wait behind anything without costing anybody anything. Put them in one queue and a user is locked out of their account while the box works through an hour of somebody else’s repository maintenance, because a gc that arrived first is ahead of them. The queue a job waits in is part of what the job costs, and these two jobs cost different things while waiting.

So CoreControlPlane holds two crate::job_queue::JobQueues and each has its own jera lane. What used to be two copies of the queue is now two instances of one, which is the same two lines and half the code.

§The property this module exists for

The gate goes on before the job is enqueued, in the RPC handler, so there is no window in which a purge is accepted and the user can still push. That is deliberately not the queue’s business — the queue cannot promise it, because the promise is about what is true when Accounts.Purge returns, which is before the queue has done anything at all. The gate is taken by the caller, under an id the caller takes from PurgeQueue::next_job_id first.

It comes off here, in JobKind::settle, and it comes off on every path out of the runner:

  • the runner returns Ok → released;
  • the runner returns Err → released, and the job is FAILED with the reason and the list of stores still on the disk;
  • the runner panics → caught, released, the job is FAILED saying so, and the worker survives. An unwinding worker takes the queue with it, so the account after the panicking one would sit QUEUED for ever behind a thread that no longer exists — one account’s bug bricking the next one’s.

The catch and the always-runs settle are now crate::job_queue’s, which is the whole gain from the extraction: a gc, and a mirror, and anything after them get the same two guarantees without a second person having to rediscover why they matter.

What this module cannot cover is the process dying between two unlinks. That is what the lease in gunnar_core::gate is for: a purge’s gate carries a deadline, the runner re-stamps it as it goes, and a gate whose deadline has passed is not a gate. Belt here, braces there, because the outcome they both prevent — an account shut by a row nobody remembers, held by a process that is gone — is worse than an unfinished purge. The boot sweep that releases every lease a killed process left behind is CoreControlPlane::release_stale_purge_leases, called once at start-up: it is about rows in the identity table and not about jobs, so it is not the queue’s and did not move here.

§What this module is not

It is not the purge. That is [crate::admin::purge_account], reached through a runner handed in at construction, so the queue can be tested with no disk under it and there is one implementation of what emptying an account means (one implementation). The release is handed in the same way and is AuthTable::lift_gate_held_by in the real server — held_by, so a purge finishing cannot lift a suspension an operator put on while it ran.

Structs§

Purge
The purge kind. What a purge is, and — the part a gc has no equivalent of — what a purge has to give back when it ends, however it ends.
PurgeOrder
What a purge was asked for, carried on the job so the worker can hand it back to the runner minutes later.
PurgeQueue
The queue: one worker, one purge at a time, FIFO, deduplicated by account.
PurgeReport
What a purge found, reclaimed, and gave back.

Constants§

FINISHED_KEPT
Finished jobs kept regardless of age, most recent first.
FINISHED_RETENTION
How long a finished purge stays readable. The same ten minutes crate::gc_queue::FINISHED_RETENTION uses, and for the same reason: far longer than any console poll, far shorter than a session.

Type Aliases§

GateRelease
How the lockout this job took is released.
Progress
The handle a running purge reports through.
PurgeJob
One account purge, running or completed.
PurgeRunner
What a purge actually is, handed in so this module does not have to know.
PurgeState
Where a purge is. JobState’s four words, shared with every other kind of job so PurgeStatus and GcStatus cannot drift apart about what Done means.