Expand description
Domain logic and API types shared by the runnerd daemon and its clients.
api and config build everywhere. The host feature (on by default) adds what only the daemon runs: Docker, the
GitHub client, the caches, the scheduler, sizing advice, insights, the job history store and what observes running jobs.
Clients depend on this crate without it, and with the client feature for client, their connection to the daemon.
Modules§
- actions
- The owner’s job actions (ADR 0134): cancelling a job’s workflow run and re-running a job, or its run’s failed jobs,
on GitHub, for
POST /v1/jobs/{id}/cancelandPOST /v1/jobs/{id}/rerun. A request without a confirmation gets a preview: the job and its run’s other jobs from the history, the run’s current attempt read from GitHub, and a confirmation issued for exactly that (flights). A confirmed request is authorized against one snapshot of the config (policy) and sent once through the engine’s own GitHub client, off the engine’s loop. Once the token is in hand, and before each send (a 401’s second included), the snapshot is checked, the run read again for its attempt, and the snapshot checked once more, with nothing in between that and the send. - admission
- Admission by measured peaks: a job’s memory peak and average CPU predicted from the job history (
Predictor), and the history replayed through admission by caps and by predictions, with or without a CPU budget (simulate), to show what each would have done with the same jobs, with the barrierscheduler::planputs up for a starved job or without, and with jobs arriving when they did or after what they followed, or as they really ran (trace); what the host’s memory leaves runners (host); and what all that runs counts for in the budget (pooled, ADR 0099), with the growth count’s brake (growth). A job’s memory and usual run time go by its last runs (recent, ADR 0070, 0097). Pure: no I/O, no clock. - api
- The daemon’s JSON API, version 1: the request and response types every client deserializes, and the
principal a transport authenticates. Plain data only: no I/O, no Docker or GitHub types, no secrets but a
Pairing’s one-time link, which itsDebugleaves out. - boots
- What the host says about its boots: whether a past boot ended in an orderly shutdown whose final sync finished,
read from the systemd journal, and whether this host’s journal keeps past boots for runnerd to read (ADR 0141).
It runs
journalctl, within a deadline (command), and reads nothing else; it never writes, and never decides what to empty. - buildcache
- The build cache every runner shares (ADR 0052, 0053, 0083): the entries of Gradle’s HTTP build cache, and of the
other tools the runner image points at it, Turborepo’s and Nx’s among them, kept per repository in a directory of
the daemon’s and served to each runner on a Unix socket of its own, which the runner image forwards to
127.0.0.1. Owns the entries, the size limit, the sockets, and what each runner was served and stored. Which socket a request came in on is who sent it; no job can claim another’s. It never decides whether a job may write: the engine tells it (Writes), from what GitHub says of the job. It never logs a request’s headers, which may carry a tool’s token. Where the config turns it on, the same socket serves GitHub Actions’ cache (actions, ADR 0082) from the same entries’ directory and limits. - cache
- The warm caches runner containers share and keep between jobs: each profile’s read-only snapshot, each slot’s volume (its layers’ changes and its Docker Engine’s data), the shared pnpm store and Maven repository, and the Docker Hub mirror on the runners’ network. Owns their names, mounts and lifecycle; it never decides when a runner starts, and it never touches a volume a running container uses.
- command
- The commands runnerd runs (the doctor’s
systemctlandgh,gh auth tokenfor the GitHub client, andjournalctlfor how a past boot ended,boots), each in a process group of its own, so none outlives its deadline or the daemon: past its deadline, its whole group is killed, whatever its processes did with the pipes, and once the daemon stops, every group still running is killed and no command starts (ADRs 0132 and 0134). - config
runnerd.toml: the repositories to serve, the runner profiles they use, the shared caches and the host budget they must fit in. Builds without thehostfeature, so clients can read a config; its only I/O is reading the file and askingsystemctlfor a slice’sMemoryMax. It never talks to Docker or GitHub.- docker
- Runner containers through the Docker Engine API: the connection, container, volume and image requests, and the
DockerApitrait the engine and the caches call. It decides nothing about scheduling, and a JIT config only ever reaches a container as a root-only file, never the create request or a log line. - github
- The GitHub REST API as the scheduler and the history need it: queued jobs, a job’s steps, just-in-time runner
configs, repository runners, cancelling a run, and the owner’s job actions (ADR 0134). Owns the token (
token), which workflow each run is of and what it is named (workflows, ADR 0140), the pages of a listing, conditional requests and the answers kept for them, rate-limit backoff, and the count of what it asks and the limit it last read. It must never put the token in a log line, aDebugoutput or an error, nor send it anywhere but its base URL, and it decides nothing about scheduling. A job action’s request is read for its status and headers only: its body is never read, and what it returns is a class and a status, never text. - insights
- Insights from the job history (ADR 0051): jobs that got slower and stayed so, jobs that fail and pass on a re-run,
jobs whose memory nears their profile’s cap, jobs that find almost nothing in the build cache, repositories whose
downloads memory pressure throttled, jobs their CPU cap held back by the rule auto sizing reads (ADR 0087), jobs
held for the host’s memory by programs outside CI (ADR 0100), and how long each repository’s jobs wait, this week against the last when the history holds all of last week; and the
daily digest. What the host’s memory came to while CI ran, and the warning when it ran low on a host credit
(ADR 0108), are
host’s. Pure: the daemon reads the history, the config and the clock and passes them in. It sends nothing and changes nothing. - isolation
- How a runner container is kept from the host while it runs its own Docker Engine. A backend decides only how
the container is created; its name, limits, labels and slice are the same for every backend (
docker). It also words what the host must have for a backend to start one, so a refused start andrunnerd doctorsay it alike; it reads nothing from the host. - observe
- What runnerd learns about running jobs without GitHub’s help: CPU, memory and network of each runner container
from its cgroup v2 files and
/proc(metrics), what it and the drive under Docker’s root write (disk), the job a runner took and its log from inside the container (capture), and which runner pulled what through the Docker Hub mirror (mirror). Reads files and parses output; it starts no container, writes nothing to the host, and decides nothing about scheduling. - record
- Records of failed jobs: which runner containers may be kept, the files a record must never hold, which records go
to keep within
[records], what a shell in a record shows first, and whether a failed job’s log says its runner failed it (runner_fault). Pure functions: the engine (scheduler::engine) commits, lists and removes the images, and the daemon serves the shell. - scheduler
- Which runners to start and stop, as a pure function of queued jobs, slots, warm runners and the memory and CPU
budgets (
plan), how many warm runners each repository wants (warm), and the loop that applies those decisions with GitHub and Docker (engine).planandwarmdo no I/O and read no clock. - sizing
- Right-sizing advice from the job history: the spread of each profile’s and each job’s sampled peaks, waits and
slot use, and the
memoryandslotsto set. Pure: the store reads the history, the daemon serves the result (api::Sizing). It never changes the config: a client applies advice throughPUT /v1/config, which validates it like any other config, and under[sizing] autothe daemon applies whatauto::changespicks the same way. - store
- The job history: a SQLite database of jobs, their runners, steps, samples and logs, the events clients resume
from, cache sizes, the paused repositories and where the notifier got to. Owns the schema, the one writer and
the event broadcast. Writes are messages to a writer thread, so the engine never waits on the disk; reads and
pruning run on Tokio’s blocking pool, pruning on its own connection, a batch at a time. It decides nothing about
scheduling and stores no secret: no token, no JIT config, no runner
_diaglog.