Skip to main content

Module store

Module store 

Source
Expand description

Cache::store per CACHE-017 and CACHE-018.

Two-phase store algorithm:

  1. Pick a random suffix, build the tmp directory path .tmp-<hex-key>-<suffix> on the entry’s shard, and create_dir_all its outputs/ subdirectory.
  2. For each output: read its bytes via the filesystem trait, hash them under the active HashAlgo, write the copy under outputs/<hex-content-hash>, set the recorded mode, fsync.
  3. Write stdout and stderr into the tmp directory, fsync; compute their hashes under the same algorithm.
  4. Assemble the Manifest, serialise it, write it as the last file in the tmp directory, fsync.
  5. fsync the tmp directory. If an entry already exists at the final path (re-store of the same key), remove it: POSIX rename(2) returns ENOTEMPTY for a non-empty directory destination, so the tmp directory cannot be renamed onto an existing populated entry. Then atomically rename the tmp directory to the final entry path, and fsync the shard directory so the rename survives a power loss (POSIX requirement).

The manifest is written LAST inside the tmp directory but only becomes reachable when the rename succeeds; before the rename no manifest exists at <shard>/<key>/manifest.json, so a concurrent lookup observes a miss until the entry is complete.

Per CACHE-018, only successful runs (exit status 0) are eligible to be stored. That is the caller’s contract: the method does not take an exit_status argument and unilaterally writes 0 to the manifest’s exit_status field. A non-zero exit run that nonetheless reaches Cache::store would be a contract violation on the caller’s side, not a contract this method enforces.

Structs§

StoreInputs
Bundle of inputs to Cache::store.
StoredOutput
One declared output of a successful task run, in the shape the cache needs to ingest it (CACHE-013).

Enums§

StoreError
Failure modes for Cache::store.