Skip to main content

Module cache

Module cache 

Source
Expand description

The warm caches runner containers share and keep between jobs: each profile’s read-only snapshot, each slot’s volume (its layers’ changes and its Docker Engine’s data), the shared pnpm store and Maven repository, and the Docker Hub mirror on the runners’ network. Owns their names, mounts and lifecycle; it never decides when a runner starts, and it never touches a volume a running container uses.

Modules§

mirror
The Docker Hub mirror’s storage: finding the links in it whose blob is gone or empty, removing them before the mirror starts, and the doctor’s check of a running mirror (docs/adr/0144). It reaches the storage only through a container of the mirror’s image, never the host’s view of the volume; it writes only while no mirror runs, and only reads a running one’s.

Structs§

Cleared
What empty_volumes came to.
Emptied
A shared cache trim_shared emptied.
Measure
What a measure container (image/seed.sh measure) runs on: the profile’s image, its slot 1 and the snapshot, both read-only, and the isolation backend.
Names
The names of everything runnerd creates in Docker, under one prefix: runnerd, or another for tests.
Refusal
A volume empty_volumes left.
Refused
A volume trim_shared was to remove that Docker refused to.
Reseed
What reseed runs on: the profile’s image and limits, its slots and the isolation backend.
StartOver
What start_layers_over runs on: a slot’s volume a host crash may have torn and the guard could not empty, and the image, slice and isolation of a runner.
Trimmed
What trim_shared measured and emptied.

Enums§

Kept
What kept a volume empty_volumes did not remove.
Measured
What a measure found: what a reseed would weigh against caches.reseed_at.
Mirror
Whether the mirror’s container is down, so its storage can be removed.

Constants§

MIRROR_IMAGE
The Docker Hub pull-through cache (registry 3.1.2, multi-platform index).
PREFIX
The prefix of the names of what the daemon makes in Docker (Names).

Functions§

docker_budget_gib
One slot’s share of disk among slots, in whole GiB as image/layers.sh takes it: at least 1, at most DOCKER_SLOT_MAX_GIB.
docker_disk
What the slots’ Docker Engines may keep together, in bytes: a quarter of Docker’s disk not otherwise used, which is what is free plus what the slot volumes already hold, measured together (docs/adr/0054).
empty_volumes
Removes each of volumes that exists, after the containers of runnerd’s that mount it and no longer run, such as a runner or a seed a crash stopped: Docker refuses to remove a volume a container mounts, even a stopped one. It leaves a volume a running container mounts, or a container that is not runnerd’s, and removes no such container.
finish_reseed
Waits for a seed or measure container to exit, removes it, and returns its output, in a background task. Also takes over a seed container an earlier daemon left running: stopping it between its rm and mv would lose that snapshot layer. It returns only once Docker said the container exited or is gone: one whose wait and removal both failed, as when the Engine’s API went away, may still be rewriting the snapshot or a slot’s layers, so it asks again every ASK_AGAIN, and the profile reseeds until then (docs/adr/0152). Never awaited in the tick.
measure
Measures what the profile’s slot 1 changed (image/seed.sh measure), in a container that mounts slot 1’s volume and the snapshot read-only, so it may run while the profile’s runners do. A measure container an earlier daemon left is removed first.
measure_spec
The container measure runs: seed.sh measure on slot 1’s volume and the snapshot, both read-only.
mirror_body
The mirror container: Docker Hub through a proxy registry, which refuses pushes, so a job can only make it fetch what Docker Hub serves. Content not pulled again within the TTL expires. No published port. It keeps no blob descriptor cache: the proxy’s expiry of one repository’s copy of a blob deletes the blob other repositories still link to, and the cache would go on answering their pulls with no data (docs/adr/0144).
mirror_down
Stops the mirror, and removes a repair container a killed daemon left on its storage, which would keep the volume from being removed; its storage stays.
mirror_up
Starts the mirror unless it runs, after removing the links in its storage whose blob is gone (mirror::Repair), which a failed repair does not hold back.
prepare
Creates what a profile’s runner in slot (1-based) mounts and joins, if missing.
reseed
Makes slot 1’s layers the profile’s new snapshot and drops every slot’s own layer changes (image/seed.sh), if slot 1 changed at least min_changes. A snapshot that would outgrow max_size starts its tools layer, then its Android layer, over from the image until the rest fits, and every layer when that is not enough, each with every slot’s changes to it (docs/adr/0065, docs/adr/0152); after a reseed that did not finish, it starts over whatever the limits (docs/adr/0066). Call only while none of the profile’s slots runs. Returns the seed container’s output.
runner_mounts
What a runner container in slot slot (1-based) of the profile mounts (paths from image/Dockerfile).
seeded
Whether the output of a reseed (image/seed.sh) says it replaced the snapshot, and so dropped every slot’s own layer changes, rather than left them because slot 1 changed too little.
start_layers_over
Deletes everything in a slot’s volume but its Docker Engine’s store (image/layers.sh‘s DOCKER_DATA_ROOT), in a container that mounts it: its layers’ changes and work directories, and the record of the snapshot they were made against. The slot’s next runner starts them over from the snapshot, and a reseed finds nothing of slot 1’s to copy (ADR 0142). For a volume the guard could not remove, as another container holds it: Docker lets a second container mount it. A container an earlier daemon left is removed first.
trim_shared
Empties the shared caches that outgrew their limit, the mirror’s storage among them once the mirror is down (Mirror::Down), and removes the volumes of slots past the count slots gives their profile, as a lowered count leaves them: each holds a slot’s layers and Docker store, which no job prunes again (docs/adr/0054). Call only while no runner runs, and leave out of slots a profile whose reseed runs, which mounts its slots’ volumes.
volumes
Every cache volume the config’s profiles use, what it holds and its limit, with size 0.