Module forge_cache 

Convention for a remote forge step’s host-persistent build cache.

The gap this closes

A remote subprocess gets image + argv + the forge_produced mount and nothing else, so a step that compiles a source tree compiles it from scratch every single run: the container’s writable layer (where CARGO_TARGET_DIR lands by default) is thrown away when kamaji reaps the exited container, and /yah/produced is per-run and reaped on destroy. mesofact-musl’s x86_64 leg measured 9m56s / 9m57s / 11m10s across its successful runs and every one of those was a cold full release build.

This is the third mount under forge_state::HOST_ROOT, and — exactly as R603-B6’s handoff promised — it needs neither new yubaba code nor a yubaba.service edit: ensure_forge_state_dirs already mkdirs any forge bind under that root.

Why the key is derived, not caller-supplied

A shared target dir keyed by nothing is a correctness bug, not merely a race. mesofact-musl carries concurrency_key = "mesofact-musl", but that is camp-side scheduling: it does not constrain a second camp, or a hand-rolled dispatch, aiming at the same worker. The key is therefore derived by the dispatcher from pipeline + step + target triple ([key_from_parts]) rather than written in a TOML, so two different pipelines — or the same pipeline’s two triples — cannot land on one target dir however the run was started.

Two runs of the same pipeline+step+triple DO share, and that is the whole point: cargo is designed for exactly that reuse, and its own .cargo-lock in the target dir serializes two builds that overlap in time.

Eviction is explicit

An unbounded cache on a worker rootfs is R702’s subject. Both holders of a cache root — yubaba on the worker, qed for the local-container placement — sweep it with [evict_plan]: anything idle past [RETENTION] goes, and if the filesystem is below [FREE_FLOOR_BYTES] the least-recently-used dirs go too, until it is not. The cache can therefore never consume the last few GB of a build worker’s /var.

Constants

CACHE_DIR_ENV

Environment variable carrying CONTAINER_DIR into the step, so an argv never has to hardcode the convention.

CONTAINER_DIR

Conventional container-side directory a cached forge step’s build scratch lives in. Bind-mounted onto a host-persistent, key-scoped dir.

FREE_FLOOR_BYTES

Below this much free space on the filesystem holding a cache root, least-recently-used cache dirs are evicted until it is above it again. This is the bound that matters on a build worker: us-west-003’s /var is a 60 GB LV, and a release target dir is multiple GB.

HOST_ROOT

Host root under which each cache key gets a directory: <HOST_ROOT>/<key>/. Under super::forge_state::HOST_ROOT, so yubaba’s ensure_forge_state_dirs creates it and yubaba.service already grants write access to it.

MAX_KEY_LEN

Longest derived key kept verbatim; longer ones are truncated and disambiguated with a digest by key_from_parts.

RETENTION

A cache dir untouched for this long is evicted. Long enough that a weekly release still hits a warm cache; short enough that a renamed step’s orphan does not sit on the disk forever.

Functions

cache_dir_under

The same derivation against an arbitrary root — for the local-container placement, whose cache lives under the camp’s own .yah/cache and has no HOST_ROOT to hang off. Split out for the same reason super::forge_produced::host_path_under is: the key validation is the whole safety content of both, and two copies is one copy that can be fixed alone.

durable_mount

The build-cache bind mount for one key: host <HOST_ROOT>/<key> → container CONTAINER_DIR, writable. None for an invalid key — the caller must refuse rather than mount something else.

evict_plan

Which cache dirs to delete, given every dir in a cache root with its last-modified time, the current free space on that filesystem, and the floor to hold.

host_dir

The host-persistent cache directory for one key, under HOST_ROOT.

is_valid_key

Whether key is safe to use as a single path component under HOST_ROOT. Deliberately narrow: alphanumerics plus ., -, _, non-empty, length-capped, and never a bare ./... Everything a dispatcher derives passes; nothing a hostile spec could write escapes.

key_from_parts

Derive the sharing key from the parts that must not collide: the pipeline, the step within it, and the target triple.