Module death_note 

Durable record of what killed the server process on paths that never reach the graceful-shutdown log. The server’s death note: a durable record of what is killing the process on paths that never reach the graceful-shutdown log.

Two production servers died on 2026-08-16 (pids 52844 and 2681) with identical faces: the main log ends mid-flight on routine lines — no drain, no panic output, no OS crash report. This module exists so the third such death is a diagnosis instead of a mystery. It arms three recorders around the run loop, writing to <AION_HOME>/logs/aion-server.death.log:

• an armed/disarmed bracket: one line when the server boots, one line when the run scope exits by ordinary control flow (clean shutdown or an error return, written by [DeathNote]’s Drop). A note that is armed but never disarmed, with no other entry, means the process was destroyed without the run loop seeing it — the SIGKILL / raw-_exit class no in-process handler can observe.
• a panic hook, chained in front of the previously installed hook, that records every panic’s thread, location, payload, and backtrace. A panic in a spawned task can be survivable; the entry says so. Death by panic reads as a PANIC entry followed by the drop-written DISARMED line as the unwind leaves the run scope.
• a termination-signal watcher for the catchable signals whose default action kills the process silently (SIGHUP, SIGQUIT, SIGUSR1, SIGUSR2): the note records the signal, then re-applies the signal’s default action so process behaviour is unchanged — the death is noted, not prevented. SIGTERM and SIGINT are recorded as observations only: the graceful drain in [crate::run] owns those two, and both watchers coexist because tokio and this module register through the same signal-hook-registry chain.
• breadcrumbs: work sites call [breadcrumb] as they begin a unit of in-flight work (today: every declared action body the server executes), so a corpse’s LAST breadcrumb names what was running when the process died. The third 2026-08-16 death (pid 25680, ~16:27Z) happened mid-declared-action with only an ordinary INFO log line to say so; this makes that fact a note entry the corpse reader sees first.

NOT covered, stated plainly: SIGKILL/SIGSTOP (uncatchable by kernel contract); the fatal-fault signals (SIGSEGV/SIGBUS/SIGILL/SIGFPE) and raw abort(), whose handlers must run in async-signal context and therefore require unsafe this workspace denies (the OS crash reporter remains the observer for that class — both 2026-08-16 corpses left no crash report, which is itself evidence against that class); and a direct exit() that bypasses the run scope. With the note armed, each of those reads as ARMED-without-DISARMED plus the absence of every entry above — a narrow, named remainder instead of an anonymous death.

Structs

DeathNote

The armed death note. Constructed by DeathNote::arm early in the server run loop; its Drop writes the DISARMED line, so any ordinary exit from the run scope — clean shutdown, error return, or a panic unwinding through it — closes the bracket on the record.

Functions

breadcrumb

Append a BREADCRUMB entry to the armed death note, naming work now in flight so a corpse’s last breadcrumb identifies the site that never finished. A silent no-op when no note is armed (tests, tools, the CLI).