Module wake 

The wake sweeper: the background task that re-drives runs whose durable timer has come due.

Why a sweeper at all

A run parked on a timer is passive data. Nothing in this process holds it, nothing is scheduled for its instant, and a restart forgets nothing because there was nothing to forget: the deadline lives in the log. So waking is not a callback firing, it is somebody re-reading the store and re-driving what is overdue. This task is that somebody, for an operator who runs a server; salvor wake is the same thing for one who runs cron.

It is the resume path, not a second driver

Every due run goes through [crate::runs::redrive], the exact function the resume endpoint’s recover arm calls. A woken run therefore rebuilds its agent the same way, drives over the same loop (or the same graph engine), records the same events, and reports the same errors as one a person woke over HTTP. There is no wake-specific drive to keep in step with the real one, and no wake-specific verb: the deadline is enforced inside RunCtx::await_wake against the injected clock, so a run driven a minute early simply records nothing and stays asleep.

Not fighting the drivers already running

The server tracks which runs a task in this process is still driving (AppState::is_run_active). The sweeper skips those, and it drives sequentially, so within one pass it can never queue a run twice; across passes, a run stays in that set from the moment [crate::runs::redrive] spawns its task until the task ends, which is exactly the span during which re-driving it would be wrong. A sleeping status and an active driver are contradictory states in any case (the fold reports sleeping only for a log that stopped), so the check is a guard against a stale read, not the normal case.

Nor fighting a client

A run opened through /v1/client-runs is driven by its caller under a single-writer drive token, not by a task in this process, so AppState::is_run_active never sees it; a separate check (AppState::is_client_run) is what keeps the sweeper off it. A client-driven run’s timer is the client’s to wake, since re-driving one here would be a second writer racing its drive token, so a due one is left asleep here regardless of how overdue it is.

That check reads a registry that dies with the process, so the run’s own log is consulted too (see client_runs::log_is_client_driven): a client-driven run’s RunStarted records driven_by: client, which survives a restart the leases do not. A restarted server therefore still leaves a napping client-driven run to its client, rather than adopting every one it no longer remembers.

One bad run does not stop the sweep

Every failure is per-run: an agent this server has never had registered, a graph it does not hold, a build that will not build. Each is logged and the loop moves to the next run, and the run is left asleep with its log untouched, still due, so registering the missing definition is enough to make the next pass wake it. Only the store listing itself failing ends a pass, and even that only ends the pass: the next one tries again.

An unwakeable run logs the same fields every pass, but only the first sighting is loud: AppState::mark_unwakeable_warned names the first pass WARN and every later one DEBUG, so an operator learns about the gap once instead of every sweep interval for as long as it stays unregistered, while the fields to find and fix it stay available to anyone watching at debug level. The record clears the moment the run wakes or drops out of the due set, so it never mutes a genuinely new nap.

Structs

Sweeper

A running sweeper, which stops when this value is dropped.

Functions

spawn_sweeper

Spawns the sweeper over state.

sweep

One pass: select the runs whose deadline has passed, re-drive each, and report the ids a drive was started for.