Skip to main content

Module formation_launch

Module formation_launch 

Source
Expand description

Launching a formation: every member of an admitted roster as a mur run subprocess of its own, peers first and the entry member last, and every member stopped once the entry member’s process has ended.

The launcher supervises; it runs no session. Each member, the entry member included, is a mur run --capsule <capsule> --capsule-version <version> process this module starts and owns. The launcher waits on the entry member’s process, so the peers are stopped however the entry member ends — its task finishing, its own exit(143), or a SIGKILL of its process.

The launch decides what the entry member does ([entry_args]). Launched with a task, it runs that one task and exits, which ends the formation. Launched without one, it takes tasks at its door one after another, with no idle timeout, until the formation is stopped: mur stop, a signal to the launcher, or the deadline.

Every member holds a lifeline. Each member is handed the read end of a pipe of its own (crate::lifeline), whose write end only the launcher holds and never writes to. A member that reads EOF winds down as a first SIGTERM would wind it down. The launcher ends a formation by closing lifelines; when the launcher itself dies — SIGKILL and the OOM killer included — the kernel closes them, so no member outlives its formation.

Ready means the door answers as the session that reported it. A peer is ready when the door at the URL its readiness line reported serves an agent card naming the session id that line reported, read through crate::running’s door probe. A live pid proves nothing, and a door some other process holds on the reported port names another session. Every member has one deadline, counted from its spawn.

All or nothing. A peer that fails to come up refuses the whole launch: every member already started is stopped and reaped before the refusal is returned, and the entry member is never started.

The launcher is the formation’s one principal. It generates the formation’s signing key (FormationAuthority) after the id and holds it in memory alone. Each member gets a pipe of its own, whose read end only that member’s mur run inherits, named by FORMATION_CHANNEL_ENV. Its first line — written before the member is spawned — carries the formation id, the member’s name, the verification key and one token per member it may call. Once every peer’s door answers, each peer that calls other peers is written one address line naming their doors; the entry member’s first line and address line are both written before it is spawned. Nothing here writes a token to stdout, stderr, a file, argv or an environment variable. Callees never include the entry member: it takes the formation’s tasks at its door from the operator or a client and serves no peer, and admission never admits an edge into it. The key and the write ends are dropped once every member is reaped.

Teardown.

  • One owner holds every started member, and every return path stops and reaps them. Its Drop does the same, so an early return or an unwinding panic leaves nothing running.
  • Stopping the members is closing every lifeline, up to MEMBER_STOP_GRACE for each to end in order, SIGKILL to the process group of any still running, and a reap. The launcher sends no member SIGTERM. Members are waited on concurrently.
  • Every lifeline is created before the first member is spawned, and the launcher spawns nothing but members, so no process but the launcher ever holds a write end.
  • Peers lead process groups of their own, so a terminal’s ^C reaches only the launcher and the entry member, and the launcher stops the peers in order. The entry member stays in the launcher’s group and behaves at a terminal as a hand-run mur run does.
  • Under panic = "abort" no destructor runs, so a panic hook — installed once per process — SIGKILLs every registered member’s process group before the abort. It reads a lock-free registry of member pids, never a mutex a panicking thread might hold.
  • catch_launcher_signals turns SIGINT, SIGTERM and SIGHUP into a flag every wait here polls, so a signalled launcher stops its members — the entry member through its lifeline, like every other — before it exits.

Structs§

FormationExit
A formation that has ended, every member stopped and reaped.
FormationLaunchOptions
What a formation launch is given besides its roster.
FormationMarker
Who launched a formation, from which project: the contents of FORMATION_MARKER_FILE.
MemberFailure
One member that did not come up, and why.
MemberLaunchFailure
A launch that did not complete. By the time this is returned, nothing it started is running.
ReadyPeer
A peer whose door answered as the session its readiness line named.
RunningFormation
A formation whose peers are ready: the one owner of every member it started, and of the formation’s signing key.
StoppedMember
A member whose process was still running when the launcher stopped it.

Enums§

FormationEnding
How the entry member’s process, and with it the formation, ended.
MemberFailureReason
Why one member did not come up.

Constants§

CAPSULE_SHA256_FLAG
The hidden mur run flag carrying the sha256 admission bound a member to.
FORMATIONS_DIR
The directory under the murmur home that holds every formation’s peer directories.
FORMATION_MARKER_FILE
The ownership marker in <murmur home>/formations/<frm_id>/, a FormationMarker. Member names start with a letter and hold no ., so it never collides with a member’s directory.
MEMBER_READY_TIMEOUT
How long a member has, from its spawn, to report itself and have its door answer.
MEMBER_STOP_GRACE
How long a member has, after its lifeline is closed, to end in order before its process group is killed.
PEER_BIND_ADDR
The address every peer binds its door on. Members of one formation share a host.
STORE_ROOT_FLAG
The hidden mur run flag naming the directory a peer resolves its artifacts, its lock and its workspace from, apart from its --workdir.

Functions§

catch_launcher_signals
Catch SIGINT, SIGTERM and SIGHUP for the rest of this process’s life, so a formation launcher stops its members before it exits rather than dying with them running.
caught_signal
The signal catch_launcher_signals caught, if any.
formation_member_roots
Every records group a formation’s peers record under: the records group of each directory under <murmur home>/formations/<frm_id>/, sorted, for each one that exists.
formation_member_roots_in
formation_member_roots under formations_dir in place of <murmur home>/formations. Each member’s records group is still resolved under this host’s ~/.murmur/records.
launch_formation
Start every peer of roster at once and wait for each door to answer.
prune_ended_formations
Remove the ended formation directories of project_dir that the entry member’s policy does not keep, and return what went.
prune_ended_formations_in
prune_ended_formations over formations_dir and running_dir in place of the murmur home’s.
render_stopped
name (pid N), …, or none.
signal_name
The conventional name of the three signals a launcher catches.