Skip to main content

Module daemon

Module daemon 

Source
Expand description

The unattended scheduler and independently leased task workers.

The scheduler chooses tasks and execution lanes; a detached worker owns each queue claim, graph driver and outcome settlement. Replacing the scheduler reconstructs occupied slots from live worker jobs, without touching their attempts, passes or paid agent calls. Finite maintenance jobs follow the same lifetime rule. Explicit parking is durable control, never an upgrade step.

§A crashed worker is legible

A task is written Running, with its run id, before graph execution. The worker keeps its claim until the outcome is recorded. Recovery requires both no live worker reservation and an obtainable task claim, and settles the saved run through the same policy as a live finish. A scheduler’s missing heartbeat alone is never evidence that a run died.

§Retries are bounded

Every attempt at a task consumes one of Opts::max_attempts, after which the task is crate::queue::TaskStatus::Held for a human. The one exception is a run that ended Stalled: the panel collapsed because the agent CLIs hit their quota, which is a fact about the machine and not about the task, so it must not spend an attempt. Without that exception a quota outage would quietly hold the entire backlog, and the operator would come back to a reset quota and nothing left that the loop is willing to run.

Structs§

Current
What the loop is working on, for the status file.
Opts
How the loop should behave.
Reading
The daemon’s published state, read permissively.
Status
The daemon’s liveness, published to <home>/daemon.json.
Stop
A cooperative stop, shared with whoever asked the loop to run.
Verdict
What a finished run tells the queue about the task it came from.

Constants§

HEARTBEAT
How often the status file is refreshed. A reader treats a status file older than STALE_SECS as “no daemon”, so the heartbeat has to be brisk enough that a busy daemon is never mistaken for a dead one.
POLL
Default queue poll interval.
SCHEMA
On-disk format for Status. Bumped when a field’s meaning changes.
STALE_CLAIM
How old a claim has to be before startup sweeps it. Longer than any run this graph plausibly takes, so a sweep cannot pull a task out from under a daemon that is merely slow.
STALE_SECS
How old a heartbeat may be before a reader calls the daemon dead. Six missed beats: long enough to survive a slow filesystem, short enough that a crashed daemon is not still reported as running a task.
STALLED_RUNNING
How long a task may sit TaskStatus::Running with no live daemon’s heartbeat naming it before crate::conduct is shown it as stalled.

Functions§

apply_choice_actions
Apply answered typed choices. Only the daemon calls this in production; deputies settle the question and never execute its actions.
clear_status
Delete the status file. Called on the way out so a clean exit reads as “no daemon” rather than as a daemon whose heartbeat merely stopped.
current_work
Read-only evidence of runs owned by a live daemon or independent worker.
finish_attempt
Settle task after its run executed: the verdict, the diagnosis, the supersession of earlier attempts and the save. Shared by the loop and by magi run --resume, so an attempt is counted one way wherever it ran. Returns the quota losses this execution added to the run.
foreign_loop
A loop in another process that is alive right now: a fresh heartbeat in <home>/daemon.json published by a pid that is not own_pid. Some(pid) carries the pid it published (None inside when it published none - a fresh heartbeat is still evidence of a live loop).
hold_if_runnable
A task a hand-started run leaves runnable (failed with attempts to spare, or requeued) is held: nobody asked for a retry, and a loop starting later must not spend agent calls on it. magi task release retries.
is_working_on
Whether a live daemon is working on this run at this moment.
is_working_on_short
Whether a live daemon is working on a run whose short id is this one.
is_working_on_task
Whether a live daemon is working on this task at this moment.
loop_would_resume
Whether an attempt at a task whose newest run is run would resume that run rather than start a fresh competition. magi run --resume hands a run to a live loop only when this holds.
managed
Whether an independent scheduler can be controlled without its web parent.
read_status
Read <home>/daemon.json permissively, or None when there is nothing usable there.
request_control
Stop dispatch immediately; only an explicit park reaches existing workers.
restart_detached
Replace the scheduler, leaving all active task and conversation jobs alone.
run_claimed
Low-level claimed-task execution shared by the detached standalone worker and focused lifecycle fixtures. The caller must retain the queue claim for this entire call. CLI callers transfer ownership through run_detached_claimed instead of executing inside their own runtime.
run_detached_claimed
Transfer a standalone caller’s claim only after the detached worker is live, then follow that worker to its final (non-automatically-retrying) outcome.
run_scheduler_worker
Hidden worker dispatch entry. The worker infrastructure owns its job lease.
run_service_worker
Finite maintenance jobs survive scheduler replacement without duplicating agent calls or cancelling an operation between its effect and receipt.
run_task_worker
Execute and settle one task with the queue claim owned by this process.
serve
Run the loop until Ctrl-C, or until the queue drains with Opts::once.
serve_until
serve, but stopping when stop is set as well as on Ctrl-C.
settle
Record a finished run against the task it came from.
spawn_claimed
Launch the standalone task worker with a claim-to-reservation handoff. Returns only after the worker owns a durable execution reservation.
spawn_detached
Start a detached, singleton scheduler using this executable generation.
status_path
Where the status file lives.
supersede_prior_runs
Rewrite this task’s now-moot earlier attempts — Task::superseded_attempts — from Blocked/Stalled to RunStatus::Superseded, once the task itself has actually finished and its own last attempt is provably why: this loads that run and requires Merged/Ready before touching anything earlier in runs, since Task::succeed can also be called directly on a task whose last attempt never landed at all.
sweep_stale_claims
Remove claim files whose owner is provably dead, or that have simply outlived older_than, and return the task ids swept.
write_status
Publish the status file for this process.
write_status_to
Publish a status to an explicit path.