pub async fn spool_or_buffer<S, E>(
stream: S,
dir: &Path,
memory_cap: usize,
) -> Result<Box<dyn NarSource>, StoreError>Expand description
Turn a one-shot byte stream into a re-openable NarSource, bounded
either way.
Preferred path: spool to a file in dir in NAR_CHUNK_BYTES steps, peak
one chunk. Fallback: if the spool file cannot be created โ no dir, no
permission, a full volume โ buffer in memory instead, hard-capped at
memory_cap, refusing past it with StoreError::TooLarge.
The fallback is chosen before any bytes are read, deliberately. A spool that fails halfway has already consumed part of a one-shot stream and cannot be restarted, so mid-write faults surface as errors rather than silently switching strategy and truncating the upload.
Tier honesty: the fallback path is bounded, not streaming โ a machine
with no usable spool directory has a memory_cap-sized worst case per
concurrent ingest, and NARs above the cap are refused rather than cached.
That is a deliberate trade against the alternative, which is the pod dying
and taking every in-flight build with it.
ยงErrors
Propagates a read error from stream, a write error to the spool file, or
StoreError::TooLarge when the memory fallback is in use and exceeded.