Crate shep_daemon

Expand description

The daemon: a process-supervision engine plus the control plane that exposes it over a unix socket

Every command that reaches the flock goes through one SupervisorHandle. Pure decision logic (brain, backoff, entry assembly) is IO-free, so it tests deterministically under a paused tokio clock. RpcServer exposes the engine to a CLI client over $SHEP_HOME/run/shep.sock, and boot assembles both into one running daemon. The CLI re-executes itself with a hidden daemon subcommand to daemonize.

§Module taxonomy

A linked name below is public; a name in plain backticks is crate-private and has no rendered page to link to. The split is the crate’s API boundary: the two commented blocks of module declarations say which consumer holds each public one open.

§Engine

Process-lifecycle decision logic and the actor that runs it.

brain: restart decision tree given exit outcome, uptime, and budget
backoff: restart delay per the spec’s exponential backoff rule
assemble: process env, log paths, and spawn spec assembly
entry: process lifecycle state, restart budget, reload state machine
runner: ProcessRunner spawn seam, two impls
fake: deterministic scripted runner, absent from a default-features build
kill: the kill ladder, portable and generic over the process handle
supervisor: the actor: owns entries, spawns per-sheep tasks, routes commands
channel: shepherd channel codec (child↔daemon messages, newline-JSON)
cron: the Clock seam and the cron_restart worker
limits: the MemorySampler seam over a sheep’s process tree, and the enforcer that reports a max_memory breach
probes: the Prober seam and the liveness loop that reports a sheep unhealthy after failure_threshold consecutive failures; os::OsProber is the hand-rolled HTTP/TCP/exec implementation
watch: the WatchSource seam over notify’s debounced events
extras: arms the four subsystems above while a sheep is online, and turns a memory breach or a liveness failure into a guarded restart

§Plane

The control plane a CLI client talks to: event bus, request dispatch, the socket, the persisted muster roll, and the boot sequence wiring it all.

bus: the event bus: topic-glob filtering, per-subscriber forwarder tasks
rpc: verb routing onto SupervisorHandle, typed errors, deadlines
dogs: what a dog is spawned as (dog_app) and the [<name>] section, read from dogs.toml, served back to it over the socket (dog_section)
server: the connection layer: peer-cred auth, handshake, subscriptions
snapshot: the muster roll: debounced atomic flock.json writes, restore
boot: 0700 layout dirs, pidfile, socket bind with stale-socket recovery, the readiness pipe, signal handlers, ordered teardown (unix only)

§Platform

Platform glue underneath both tiers above.

sys: adopting an inherited descriptor, this crate’s only unsafe surface on unix (unix only)
privilege: user/group config to numeric uid/gid, one portable resolve() over a unix impl and a non-unix stub that refuses outright
notify: one READY=1 datagram to $NOTIFY_SOCKET, sent by boot once the muster restore has finished (unix only)
tokio_runner: real ProcessRunner over tokio::process (unix only)

§Quick start

Builds a supervisor engine with a scripted fake runner, registers one app, and lists the live processes. Needs --all-features (test-fakes).

use shep_daemon::fake::{ProcScript, ScriptedRunner};
use shep_daemon::supervisor::spawn_supervisor;
use shep_core::config::AppConfig;
use shep_core::config::normalize;
use shep_core::paths::ShepPaths;
use std::path::Path;

// Create a fake runner that spawns one process never exiting
let runner = ScriptedRunner::new(vec![ProcScript::never_exits()]);

// Set up temporary paths for this example
let paths = ShepPaths::resolve(&|_| None, Path::new("/tmp/shep-example"));

// Create the event bus every subscriber reads
let events = shep_daemon::new_bus();

// Spawn the supervisor actor
let handle = spawn_supervisor(runner, paths, events);

// Build one app config and normalize it
let app = AppConfig::minimal("web", "./server");
let resolved = normalize(app)?;

// Start the app (creates one instance)
let infos = handle.start(vec![resolved]).await?;
println!("Started: {} instance(s)", infos.len());

// List all registered processes
let list = handle.list().await;
for info in &list {
    println!("  ID {} ({}): {:?}", info.id, info.name, info.status);
}

// Gracefully shut down all processes
handle.shutdown().await;

Ok(())

§Reference

ProcessRunner spawns a child process and returns a RunningProcess handle plus a ProcIo bundle with channels for logs and shepherd messages. spawn_supervisor wires these together into the core actor loop.

§Quick start

Boots a full daemon through boot on a temporary $SHEP_HOME with the same scripted fake runner, then round-trips one Ping over the wire codec server speaks. Needs --all-features on a unix target.

use shep_daemon::boot::{BootOptions, boot};
use shep_daemon::fake::ScriptedRunner;
use shep_core::paths::ShepPaths;
use shep_core::protocol::{
    Envelope, Hello, HelloReply, PROTOCOL_VERSION, Request, ServerFrame, codec, decode_frame,
    encode_frame,
};
use tokio::net::UnixStream;
use tokio_util::codec::Framed;
use futures_util::{SinkExt, StreamExt};

// A throwaway $SHEP_HOME: `boot` creates its 0700 layout inside it.
let paths = ShepPaths::resolve(&|_| None, std::path::Path::new("/tmp/shep-daemon-example"));

// Boot with the scripted fake runner — no real children, just the plane.
let daemon = boot(ScriptedRunner::new(vec![]), paths, BootOptions::default()).await?;
let socket = daemon.socket().to_path_buf();
tokio::spawn(daemon.run());

// Connect and speak the wire protocol directly: Hello, then Ping.
let stream = UnixStream::connect(&socket).await?;
let mut frames = Framed::new(stream, codec());
frames
    .send(encode_frame(&Hello {
        client_version: "0.1.0".to_string(),
        protocol: PROTOCOL_VERSION,
        // Only a dog names one; see `Hello::dog_name`.
        dog_name: None,
    })?)
    .await?;
let ack: HelloReply = decode_frame(&frames.next().await.unwrap()?)?;
let ack = ack.expect("the daemon must ack our protocol");
println!("daemon pid: {}", ack.pid);

frames
    .send(encode_frame(&Envelope {
        id: 1,
        deadline_ms: Some(1_000),
        body: Request::Ping,
    })?)
    .await?;
let frame: ServerFrame = decode_frame(&frames.next().await.unwrap()?)?;
println!("reply: {frame:?}");

Ok(())

Modules§

assemble: Spawn assembly: pure functions that build SpawnSpec from app config.
boot: Daemon boot: layout, pidfile, control-socket bind, and the run/teardown sequence
channel: The shepherd channel’s message shapes, re-exported from shep-core.
dogs: The dog contract: what a dog is spawned as, and how it is served its own configuration
limits: Process-tree memory limits (spec §4).
notify: Readiness reporting to an init system that supervises this process directly.
privilege: Privilege drop: resolving an app’s requested user/group to numeric ids.
probes: The Prober seam and the liveness probe loop (spec §7).
rpc: Portable RPC dispatch: verb routing, typed errors, per-call deadlines
runner: Spawn seam between the daemon engine and the OS
snapshot: The muster roll: persisted flock state for restart-survival (shep muster)
supervisor: The supervisor actor: owns the flock’s lifecycle state machine.
sys: Adopting an inherited descriptor: this crate’s only unsafe on unix
tokio_runner: Real ProcessRunner over actual OS processes.

Structs§

Bus: The daemon’s event channel, plus the one question the channel cannot answer: whether anything is listening for log lines.
SharedEvent: One published BusEvent, plus the wire frame its subscribers share.

Functions§

new_bus: Creates the daemon’s process-wide event bus.