rto_exec/assets.rs
1//! Pinned analyzer assets: what a run needs before it can happen, and what
2//! happens when it is not there.
3//!
4//! ADR-0014's working model is *mostly offline, degrade gracefully, pre-download
5//! expected*. That is a provisioning contract, and this module is it:
6//!
7//! - **`roteiro security prefetch`** installs and verifies every pinned asset an
8//! analyzer needs, recording its digest and the time it was fetched.
9//! - **`roteiro security status`** reports each digest and fetch time.
10//! - **A run never provisions.** Cold cache fails with
11//! [`ExecError::AssetsUnavailableOffline`], which names the missing assets,
12//! their pinned digests, and the exact command to fix it. Never an implicit
13//! fetch; never a silent fall back to whatever the host happens to have
14//! installed.
15//!
16//! # The one rule that makes the rest work
17//!
18//! Provisioning writes; running reads. A run that quietly materialised its own
19//! inputs would make "did this machine have the pinned rules?" unanswerable
20//! after the fact — and the whole point of stamping `rules_digest` onto an
21//! [`rto_graph::AnalysisRun`] is that the question has an answer.
22//!
23//! # Four kinds of asset
24//!
25//! [`AssetSource::Vendored`] is compiled into the binary — the baseline semgrep
26//! rule set. Installing it needs no network at all, which is what makes a fresh
27//! machine on a plane able to run `prefetch` and then scan.
28//!
29//! [`AssetSource::External`] is a directory Roteiro does **not** fetch: the
30//! `RustSec` advisory database, which is a git checkout rather than a file with
31//! a stable URL. `prefetch` verifies it is there, digests it, and records it, so
32//! a run consults a database whose identity was pinned before it started —
33//! rather than whatever `~/.cargo/advisory-db` happened to contain. If it is
34//! absent, `prefetch` says exactly how to obtain it and refuses.
35//!
36//! [`AssetSource::Download`] is fetched by URL, and arrived with `osv-scanner`
37//! in Stage 22b. Earlier revisions of this module said there was deliberately no
38//! such source because "an unused fetch path is a security surface with no
39//! user"; OSV's per-ecosystem databases are that user. They are single files at
40//! stable URLs — exactly what a digest pin wants, and what the `RustSec` git
41//! checkout could never be. The enum being `#[non_exhaustive]` is what made
42//! adding it a non-breaking change.
43//!
44//! [`AssetSource::PinnedArchive`] is the one with a **compile-time digest**, and
45//! it exists for the sandbox runtime (Stage 24). The difference from `Download`
46//! is not the transport but the target: OSV rebuilds its databases daily, so the
47//! only pin that can be honoured there is the snapshot this machine provisioned.
48//! A published release artifact is immutable, so its correct bytes are knowable
49//! in advance — and where they are knowable, they are checked.
50//!
51//! That closes the gap the [`Fetcher`] contract has to leave open elsewhere. A
52//! fetcher that reports success over a truncated body can defeat a `Download`
53//! asset's pin, because there is nothing to contradict it; it cannot defeat a
54//! `PinnedArchive`, because the expected digest is compiled in and the archive
55//! is verified here, in this crate, before it is installed.
56//!
57//! **Fetching is still confined to provisioning.** The transport is not in this
58//! crate at all: [`provision_with`] takes the fetcher as an argument, and the
59//! plain [`provision`] passes one that refuses. A run resolves assets through
60//! [`resolve`], which has no fetcher to call even if it wanted one — so "a run
61//! never provisions" is a property of the signatures rather than a rule someone
62//! has to remember.
63//!
64//! @rto:0014
65
66use std::collections::BTreeMap;
67use std::path::{Path, PathBuf};
68
69use serde::{Deserialize, Serialize};
70
71use crate::adapter::adapter_for;
72use crate::clock::{age_in_days, rfc3339_utc};
73use crate::runner::ExecError;
74use crate::sha256_hex;
75
76/// The baseline semgrep rule set, compiled in.
77///
78/// Vendoring the bytes rather than reading a file at runtime means the asset is
79/// available on a machine that has only the binary — which is the case
80/// `prefetch` exists to serve.
81pub const BASELINE_RULES: &[u8] = include_bytes!(concat!(
82 env!("CARGO_MANIFEST_DIR"),
83 "/rules/roteiro-baseline.yml"
84));
85
86/// Where an asset comes from.
87#[derive(Debug, Clone, Copy, PartialEq, Eq)]
88#[non_exhaustive]
89pub enum AssetSource {
90 /// Bytes compiled into this binary, installed as a single file.
91 Vendored(&'static [u8]),
92 /// A directory the operator provisions, which Roteiro verifies and pins but
93 /// never fetches. `hint` is the exact command that obtains it.
94 External {
95 /// What to run to obtain it, quoted verbatim in every error.
96 hint: &'static str,
97 },
98 /// A set of files downloaded by URL into one directory, digest-pinned at
99 /// provisioning time.
100 ///
101 /// Downloading happens only in [`provision_with`], and only with a fetcher
102 /// the caller supplied. There is no compile-time digest because the upstream
103 /// files are republished continuously — OSV rebuilds its per-ecosystem
104 /// databases daily — so what is pinned is the snapshot this machine
105 /// provisioned, recorded in [`InstalledAsset::digest`] and re-checked on
106 /// every run. That is the same pin the `RustSec` checkout gets, and it is
107 /// the one that can actually be honoured.
108 Download {
109 /// Each file's path relative to the asset directory, and where it comes
110 /// from. Order is preserved so `prefetch` reports progress in a stable
111 /// sequence.
112 files: &'static [DownloadFile],
113 },
114 /// A single published release artifact with a **compile-time SHA-256**,
115 /// installed as one file and selected by host platform.
116 ///
117 /// Verified in this crate, before installation and again by `build.rs`
118 /// before anything is built against it — so neither a lying fetcher nor a
119 /// redirected URL can substitute different bytes. See
120 /// [`crate::runtime_pins`] for what is pinned and why it has to be.
121 PinnedArchive {
122 /// One entry per supported host platform. A host not listed here cannot
123 /// be provisioned, and is told which platforms are.
124 archives: &'static [crate::runtime_pins::PinnedArchive],
125 },
126}
127
128/// One file of an [`AssetSource::Download`] asset.
129#[derive(Debug, Clone, Copy, PartialEq, Eq)]
130pub struct DownloadFile {
131 /// Where it is installed, relative to the asset directory. Forward slashes;
132 /// never absolute and never containing `..`, which [`provision_with`]
133 /// enforces rather than trusts.
134 pub path: &'static str,
135 /// The URL it is fetched from.
136 pub url: &'static str,
137}
138
139/// What kind of input an asset is — the axis along which it goes stale.
140#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
141#[serde(rename_all = "snake_case")]
142pub enum AssetKind {
143 /// A rule set. Changes only when someone changes it.
144 Rules,
145 /// An advisory database. Changes continuously and independently of the
146 /// source tree, which is why results derived from it are labelled *possibly
147 /// stale* rather than *current*.
148 AdvisoryDb,
149 /// The prebuilt sandbox runtime an analyzer is executed inside.
150 ///
151 /// Unlike the other two it is **immutable for a given release**: one
152 /// published artifact with one correct digest, which is why it is the only
153 /// kind carrying a compile-time pin.
154 SandboxRuntime,
155}
156
157impl AssetKind {
158 /// Stable token for display and `--json`.
159 #[must_use]
160 pub fn as_str(self) -> &'static str {
161 match self {
162 Self::Rules => "rules",
163 Self::AdvisoryDb => "advisory-db",
164 Self::SandboxRuntime => "sandbox-runtime",
165 }
166 }
167}
168
169/// One pinned asset.
170#[derive(Debug, Clone, Copy)]
171pub struct AssetSpec {
172 /// Stable id, used as the cache directory name and in every error.
173 pub id: &'static str,
174 /// The analyzer that needs it.
175 pub analyzer: &'static str,
176 /// What it is.
177 pub kind: AssetKind,
178 /// Where it comes from.
179 pub source: AssetSource,
180 /// The file name it is installed under, for a [`AssetSource::Vendored`]
181 /// asset. Empty for a directory asset.
182 pub file: &'static str,
183 /// Licence of the asset's contents, disclosed by `prefetch` before it
184 /// installs anything — the same disclosure `roteiro model pull` makes.
185 pub licence: &'static str,
186}
187
188/// Every asset this build knows how to provision.
189pub static ASSETS: &[AssetSpec] = &[
190 AssetSpec {
191 id: crate::adapter::semgrep::RULES_ASSET,
192 analyzer: crate::adapter::semgrep::ANALYZER,
193 kind: AssetKind::Rules,
194 source: AssetSource::Vendored(BASELINE_RULES),
195 file: "roteiro-baseline.yml",
196 // Written for this repository; see the rule file's own header for why no
197 // Semgrep Registry rule is vendored.
198 licence: "MIT OR Apache-2.0 (written for this repository)",
199 },
200 AssetSpec {
201 id: crate::adapter::cargo_audit::ADVISORY_DB_ASSET,
202 analyzer: crate::adapter::cargo_audit::ANALYZER,
203 kind: AssetKind::AdvisoryDb,
204 source: AssetSource::External {
205 hint: "git clone --depth 1 https://github.com/RustSec/advisory-db \
206 ~/.roteiro/security/rustsec-advisory-db/db",
207 },
208 file: "",
209 licence: "CC0-1.0 (RustSec advisory database)",
210 },
211 AssetSpec {
212 id: crate::adapter::osv_scanner::DB_ASSET,
213 analyzer: crate::adapter::osv_scanner::ANALYZER,
214 kind: AssetKind::AdvisoryDb,
215 source: AssetSource::Download {
216 files: OSV_DATABASES,
217 },
218 file: "",
219 // OSV.dev aggregates upstream databases and does not relicense them; each
220 // record carries its own terms. The two that dominate this set are named
221 // rather than flattened into one claim, because `cargo deny` governs
222 // crates and would never have looked at an advisory file.
223 licence: "per-record, as published by OSV.dev \
224 (CC0-1.0 for RustSec, CC-BY-4.0 for the GitHub Advisory Database)",
225 },
226 AssetSpec {
227 id: crate::runtime_pins::RUNTIME_ASSET,
228 // Not an analyzer's asset: every analyzer run under the sandboxed
229 // backend needs the same one, and no adapter declares it — so
230 // `assets_for` never returns it, and it is in no analyzer's asset set.
231 //
232 // That is exactly why `run_security_prefetch` falls back to selecting by
233 // this field when `assets_for` comes back empty: `prefetch --analyzer
234 // sandbox` **does** select this archive, alone, which is what lets
235 // someone bootstrapping `exec-boxlite` obtain it without also fetching
236 // ~260 MB of advisory databases. A plain `prefetch` provisions it too.
237 //
238 // The two clauses after the first used to say the opposite — that
239 // `--analyzer <name>` could never select it — and that outlived the
240 // fallback by long enough to put a wrong recipe in AGENTS.md and to
241 // nearly cost a correct review comment its adjudication (#362). The
242 // behaviour is asserted in `the_sandbox_analyzer_selects_the_runtime_
243 // archive_alone` so the two cannot drift again in silence.
244 analyzer: SANDBOX,
245 kind: AssetKind::SandboxRuntime,
246 source: AssetSource::PinnedArchive {
247 archives: crate::runtime_pins::RUNTIME_ARCHIVES,
248 },
249 file: crate::runtime_pins::RUNTIME_FILE,
250 // The archive is a bundle of separately-licensed executables, and
251 // flattening them into one claim is exactly what let 25 MB of GPL
252 // binaries through a licence gate unnoticed. Each is named, and the
253 // full record — including the source-offer duty this creates — is in
254 // `crates/rto-exec/NOTICE-boxlite-runtime.md`, disclosed before install.
255 licence: "mixed: Apache-2.0 (boxlite-shim, boxlite-guest), \
256 GPL-2.0 (mke2fs, debugfs, libkrunfw), \
257 LGPL-2.0-or-later (bwrap) — see NOTICE-boxlite-runtime.md",
258 },
259];
260
261/// The `analyzer` field for an asset that belongs to no single analyzer.
262///
263/// A sentinel rather than an empty string, so `status` prints something a reader
264/// can act on and `--analyzer <name>` cannot accidentally match it.
265pub const SANDBOX: &str = "sandbox";
266
267/// The OSV per-ecosystem databases this build provisions.
268///
269/// The layout is not ours to choose: `osv-scanner --local-db-path <dir>` looks
270/// for `<dir>/osv-scalibr/<ECOSYSTEM>/all.zip`, with the ecosystem spelled
271/// exactly as OSV spells it (`crates.io`, not `cargo`; `PyPI`, not `pypi`).
272///
273/// Four ecosystems, because that is what ADR-0018's matrix asks of this
274/// analyzer: Python, Java and Node are the gap it closes, and `crates.io` is
275/// what makes the Rust cross-reference with `cargo-audit` possible at all.
276/// **`npm/all.zip` alone is roughly 210 MB**, and the four together are around
277/// 260 MB — a real provisioning cost, disclosed by `prefetch` before it fetches
278/// anything.
279pub static OSV_DATABASES: &[DownloadFile] = &[
280 DownloadFile {
281 path: "osv-scalibr/crates.io/all.zip",
282 url: "https://osv-vulnerabilities.storage.googleapis.com/crates.io/all.zip",
283 },
284 DownloadFile {
285 path: "osv-scalibr/PyPI/all.zip",
286 url: "https://osv-vulnerabilities.storage.googleapis.com/PyPI/all.zip",
287 },
288 DownloadFile {
289 path: "osv-scalibr/Maven/all.zip",
290 url: "https://osv-vulnerabilities.storage.googleapis.com/Maven/all.zip",
291 },
292 DownloadFile {
293 path: "osv-scalibr/npm/all.zip",
294 url: "https://osv-vulnerabilities.storage.googleapis.com/npm/all.zip",
295 },
296];
297
298/// The spec for `id`, or `None`.
299#[must_use]
300pub fn asset(id: &str) -> Option<&'static AssetSpec> {
301 ASSETS.iter().find(|a| a.id == id)
302}
303
304/// Every asset `analyzer` needs, in the order its adapter declares them.
305#[must_use]
306pub fn assets_for(analyzer: &str) -> Vec<&'static AssetSpec> {
307 adapter_for(analyzer)
308 .map(|adapter| {
309 adapter
310 .asset_ids()
311 .iter()
312 .filter_map(|id| asset(id))
313 .collect()
314 })
315 .unwrap_or_default()
316}
317
318/// What was recorded about an asset when it was provisioned.
319///
320/// Persisted beside the asset as `installed.json`, so `status` reports what was
321/// actually verified rather than re-deriving it and hoping the answer matches.
322#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
323pub struct InstalledAsset {
324 /// The asset id.
325 pub id: String,
326 /// What it is.
327 pub kind: AssetKind,
328 /// SHA-256 of the asset as installed. For a directory it is a digest over
329 /// the sorted `(relative path, content digest)` list, so it changes when any
330 /// file in the tree changes and does not depend on directory iteration
331 /// order.
332 pub digest: String,
333 /// When `prefetch` verified and recorded it, RFC 3339 UTC.
334 pub fetched_at: String,
335 /// How many files the digest covers, for a directory asset.
336 #[serde(default, skip_serializing_if = "Option::is_none")]
337 pub files: Option<usize>,
338 /// When the asset's contents were published, RFC 3339 UTC — for an advisory
339 /// database that is a git checkout, its `HEAD` commit time.
340 ///
341 /// This is **not** `fetched_at`. Fetching an eight-month-old database today
342 /// does not make it current, and the difference between the two is exactly
343 /// what a *possibly stale* label is about.
344 ///
345 /// It is recorded here because the analyzer will not report it: `cargo audit`
346 /// returns `last-commit: null` and `last-updated: null` whenever it is
347 /// pointed at a database with `--db` instead of resolving one itself —
348 /// verified against cargo-audit 0.22.2, at both a shallow clone and its own
349 /// managed checkout. Pinning the database is what makes a run reproducible,
350 /// so the pinned configuration must not be the one that loses the staleness
351 /// evidence.
352 #[serde(default, skip_serializing_if = "Option::is_none")]
353 pub published_at: Option<String>,
354}
355
356/// An asset's state, as `roteiro security status` reports it.
357#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
358pub struct AssetStatus {
359 /// The asset id.
360 pub id: &'static str,
361 /// The analyzer that needs it.
362 pub analyzer: &'static str,
363 /// What it is.
364 pub kind: AssetKind,
365 /// Where it is (or would be) on disk.
366 pub path: String,
367 /// What was recorded at provisioning time, if it has been provisioned.
368 #[serde(default, skip_serializing_if = "Option::is_none")]
369 pub installed: Option<InstalledAsset>,
370 /// Whole days since it was provisioned, when that can be computed.
371 #[serde(default, skip_serializing_if = "Option::is_none")]
372 pub age_days: Option<i64>,
373 /// Whether the bytes on disk still match the recorded digest. `None` when
374 /// nothing is installed.
375 #[serde(default, skip_serializing_if = "Option::is_none")]
376 pub verified: Option<bool>,
377}
378
379// Re-exported rather than defined here, and it used to be defined here. The
380// resolution moved to `asset_paths.rs` so that `build.rs` can `include!` it:
381// looking for the provisioned sandbox runtime is looking in *this* cache, and a
382// build script that resolved the path with its own copy of the precedence would
383// disagree with `prefetch` the first time either side changed. Kept re-exported
384// so `rto_exec::assets::asset_root` still names it.
385pub use crate::asset_paths::asset_root;
386
387/// Directory a given asset lives in.
388#[must_use]
389pub fn asset_dir(root: &Path, spec: &AssetSpec) -> PathBuf {
390 root.join(spec.id)
391}
392
393/// The path an analyzer is pointed at for this asset: the installed file for a
394/// vendored asset, the directory itself for an external one.
395#[must_use]
396pub fn asset_path(root: &Path, spec: &AssetSpec) -> PathBuf {
397 let dir = asset_dir(root, spec);
398 match spec.source {
399 AssetSource::Vendored(_) | AssetSource::PinnedArchive { .. } => dir.join(spec.file),
400 AssetSource::External { .. } | AssetSource::Download { .. } => dir.join("db"),
401 }
402}
403
404/// Where the provisioning record is kept.
405fn record_path(root: &Path, spec: &AssetSpec) -> PathBuf {
406 asset_dir(root, spec).join("installed.json")
407}
408
409/// Errors raised while provisioning.
410#[derive(Debug, thiserror::Error)]
411#[non_exhaustive]
412pub enum AssetError {
413 /// An [`AssetSource::External`] asset is not present, and Roteiro will not
414 /// fetch it. The message names the command that obtains it.
415 #[error(
416 "asset {id:?} is not provisioned: expected a directory at {path}\n \
417 obtain it with: {hint}\n \
418 then run: roteiro security prefetch --analyzer {analyzer}"
419 )]
420 ExternalMissing {
421 /// The asset id.
422 id: &'static str,
423 /// Where it was expected.
424 path: String,
425 /// The command that obtains it.
426 hint: &'static str,
427 /// The analyzer that needs it.
428 analyzer: &'static str,
429 },
430 /// This build has no such asset.
431 #[error("unknown asset {0:?}")]
432 Unknown(String),
433 /// A downloadable asset was asked for without a fetcher, which is what every
434 /// path except `roteiro security prefetch` does.
435 ///
436 /// This is the offline contract stated as an error rather than as a comment:
437 /// a run that finds a cold cache is told what to run, and is never quietly
438 /// given a network connection instead.
439 #[error(
440 "asset {id:?} is not provisioned and this code path does not download \
441 ({files} file(s), starting with {first})\n \
442 fetch it with: roteiro security prefetch --analyzer {analyzer}"
443 )]
444 FetchNotPermitted {
445 /// The asset id.
446 id: &'static str,
447 /// How many files it is made of.
448 files: usize,
449 /// The first URL, so the message names something concrete.
450 first: &'static str,
451 /// The analyzer that needs it.
452 analyzer: &'static str,
453 },
454 /// A download failed. The message is the fetcher's, because it knows what
455 /// went wrong and this module deliberately knows no transport.
456 #[error("downloading {url} for asset {id:?}: {message}")]
457 Fetch {
458 /// The asset id.
459 id: &'static str,
460 /// The URL that failed.
461 url: &'static str,
462 /// What the fetcher reported.
463 message: String,
464 },
465 /// A [`DownloadFile::path`] is not a plain relative path.
466 ///
467 /// Checked rather than trusted: these paths are compiled in today, but they
468 /// name where bytes from the network are written, and a `..` in one would
469 /// write outside the asset cache.
470 #[error("asset {id:?} declares an unsafe install path {path:?}")]
471 UnsafeInstallPath {
472 /// The asset id.
473 id: &'static str,
474 /// The offending path.
475 path: &'static str,
476 },
477 /// No sandbox runtime is pinned for this host platform.
478 ///
479 /// Refused by name rather than left to fail as a link error later: a
480 /// platform Roteiro has not pinned is a platform whose runtime bytes nobody
481 /// has verified, and building against unverified bytes is the thing this
482 /// whole path exists to prevent.
483 #[error(
484 "asset {id:?} has no pinned archive for this host ({os}/{arch}); \
485 pinned platforms are: {supported}"
486 )]
487 UnsupportedPlatform {
488 /// The asset id.
489 id: &'static str,
490 /// `std::env::consts::OS` for the host.
491 os: &'static str,
492 /// `std::env::consts::ARCH` for the host.
493 arch: &'static str,
494 /// The platforms that do have a pin, comma-separated.
495 supported: String,
496 },
497 /// A pinned archive is not provisioned, and this code path does not
498 /// download.
499 #[error(
500 "asset {id:?} ({target}) is not provisioned and this code path does not download\n \
501 expected at: {path}\n \
502 fetch it with: roteiro security prefetch --allow-download"
503 )]
504 ArchiveMissing {
505 /// The asset id.
506 id: &'static str,
507 /// The host platform it would be fetched for.
508 target: &'static str,
509 /// Where it was expected.
510 path: String,
511 },
512 /// A pinned archive's bytes are not the bytes that were pinned.
513 ///
514 /// This is the check that makes the sandbox runtime reproducible, so it is a
515 /// hard failure with no override: a mismatch is either a truncated download,
516 /// a redirected URL, or a substituted artifact, and none of those is
517 /// something to carry on from. The size is reported alongside because a
518 /// short body is the common case and two unequal digests do not say so.
519 #[error(
520 "asset {id:?} does not match its pinned digest — refusing it\n \
521 from: {url}\n \
522 expected: {expected} ({expected_bytes} bytes)\n \
523 actual: {actual} ({actual_bytes} bytes)"
524 )]
525 DigestMismatch {
526 /// The asset id.
527 id: &'static str,
528 /// Where the bytes came from.
529 url: String,
530 /// The digest that was pinned.
531 expected: &'static str,
532 /// The size that was pinned.
533 expected_bytes: u64,
534 /// The digest of what arrived.
535 actual: String,
536 /// The size of what arrived.
537 actual_bytes: u64,
538 },
539 /// Reading or writing the cache failed.
540 #[error("asset cache I/O at {path}: {source}")]
541 Io {
542 /// What was being touched.
543 path: String,
544 /// The underlying failure.
545 source: std::io::Error,
546 },
547 /// The provisioning record could not be read or written.
548 #[error("asset record: {0}")]
549 Record(#[from] serde_json::Error),
550}
551
552/// How bytes at a URL are written to a local path.
553///
554/// The transport is the caller's: this crate has no HTTP dependency and is not
555/// going to acquire one for a single asset kind. `roteiro security prefetch`
556/// supplies an implementation over the `ureq` client already in the tree; tests
557/// supply one that writes fixture bytes and never opens a socket, which is how
558/// the download path is exercised without a network.
559///
560/// # The contract, and why it cannot be checked here
561///
562/// **An implementation must write the whole file or fail.** A truncated download
563/// that returned `Ok` would be renamed into place, digested, and recorded as the
564/// asset's pin — and [`AssetSource::Download`] has no compile-time digest to
565/// contradict it, so `status` would then report the short file as present and
566/// matching. Staging through a `.partial` file guards against a crash, not
567/// against a fetcher that misreports success.
568///
569/// Nothing in this crate can verify that: completeness is a property of the
570/// transport's framing, and this crate deliberately has no transport. The
571/// shipped implementation is `download_asset_file` in the CLI, which establishes
572/// it from the response's declared length and refuses a body whose length cannot
573/// be established at all.
574pub type Fetcher<'a> = dyn Fn(&str, &Path) -> Result<(), String> + 'a;
575
576/// Install and verify one asset, without any ability to download.
577///
578/// This is what every path except `roteiro security prefetch` calls. A
579/// [`AssetSource::Download`] asset that is not already present therefore fails
580/// with [`AssetError::FetchNotPermitted`] naming the prefetch command, which is
581/// the offline contract expressed as a signature.
582///
583/// It is idempotent: re-running it re-digests and re-stamps, which is what makes
584/// `prefetch` a safe thing to run whenever you are unsure.
585///
586/// # Errors
587/// Returns [`AssetError::ExternalMissing`] when an operator-provisioned asset is
588/// absent, [`AssetError::FetchNotPermitted`] when a downloadable one is, or
589/// [`AssetError::Io`] if the cache cannot be written.
590pub fn provision(root: &Path, spec: &AssetSpec) -> Result<InstalledAsset, AssetError> {
591 provision_with(root, spec, None)
592}
593
594/// Install and verify one asset, downloading through `fetch` where the asset
595/// needs it.
596///
597/// This is the **only** function that writes to the asset cache, and the only
598/// one that can cause a network request. `fetch` is `None` for every caller that
599/// must not fetch; see [`provision`].
600///
601/// # Errors
602/// As [`provision`], plus [`AssetError::Fetch`] if a download fails and
603/// [`AssetError::UnsafeInstallPath`] if a declared install path could escape the
604/// asset directory.
605pub fn provision_with(
606 root: &Path,
607 spec: &AssetSpec,
608 fetch: Option<&Fetcher<'_>>,
609) -> Result<InstalledAsset, AssetError> {
610 let dir = asset_dir(root, spec);
611 std::fs::create_dir_all(&dir).map_err(|source| AssetError::Io {
612 path: dir.display().to_string(),
613 source,
614 })?;
615 let target = asset_path(root, spec);
616
617 let (digest, files) = match spec.source {
618 AssetSource::Vendored(bytes) => {
619 write_atomically(&target, bytes)?;
620 (sha256_hex(bytes), None)
621 }
622 AssetSource::External { hint } => {
623 if !target.is_dir() {
624 return Err(AssetError::ExternalMissing {
625 id: spec.id,
626 path: target.display().to_string(),
627 hint,
628 analyzer: spec.analyzer,
629 });
630 }
631 let (digest, count) = digest_tree(&target)?;
632 (digest, Some(count))
633 }
634 AssetSource::Download { files } => {
635 download_all(spec, files, &target, fetch)?;
636 let (digest, count) = digest_tree(&target)?;
637 (digest, Some(count))
638 }
639 AssetSource::PinnedArchive { archives } => {
640 let digest = provision_archive(spec, archives, &target, fetch)?;
641 (digest, None)
642 }
643 };
644 let published_at = published_at(&target);
645
646 let record = InstalledAsset {
647 id: spec.id.to_owned(),
648 kind: spec.kind,
649 digest,
650 fetched_at: rfc3339_utc(std::time::SystemTime::now()),
651 files,
652 published_at,
653 };
654 let json = serde_json::to_vec_pretty(&record)?;
655 write_atomically(&record_path(root, spec), &json)?;
656 Ok(record)
657}
658
659/// Fetch every file of a [`AssetSource::Download`] asset into `target`.
660///
661/// With no fetcher this refuses unless the files are *already* all there, which
662/// is what makes `provision` idempotent for a downloadable asset without giving
663/// it a network: a second `prefetch --offline`-style call over a warm cache
664/// re-digests and re-stamps rather than failing.
665fn download_all(
666 spec: &AssetSpec,
667 files: &'static [DownloadFile],
668 target: &Path,
669 fetch: Option<&Fetcher<'_>>,
670) -> Result<(), AssetError> {
671 for file in files {
672 if !is_safe_relative(file.path) {
673 return Err(AssetError::UnsafeInstallPath {
674 id: spec.id,
675 path: file.path,
676 });
677 }
678 }
679
680 let missing: Vec<&DownloadFile> = files
681 .iter()
682 .filter(|file| !target.join(file.path).is_file())
683 .collect();
684 if missing.is_empty() {
685 return Ok(());
686 }
687 let Some(fetch) = fetch else {
688 return Err(AssetError::FetchNotPermitted {
689 id: spec.id,
690 files: missing.len(),
691 first: missing[0].url,
692 analyzer: spec.analyzer,
693 });
694 };
695
696 for file in missing {
697 let destination = target.join(file.path);
698 if let Some(parent) = destination.parent() {
699 std::fs::create_dir_all(parent).map_err(|source| AssetError::Io {
700 path: parent.display().to_string(),
701 source,
702 })?;
703 }
704 // Fetch beside the destination and rename, so an interrupted download
705 // never leaves a half-file at the path the analyzer reads.
706 //
707 // Staging protects the pin from a *crash*; it cannot protect it from a
708 // fetcher that returns `Ok` over a short body, because then the rename
709 // happens and the truncated file is what gets digested. That half of the
710 // contract is the fetcher's, and is stated on [`Fetcher`].
711 let partial = destination.with_extension("partial");
712 std::fs::remove_file(&partial).ok();
713 fetch(file.url, &partial).map_err(|message| {
714 // Leave nothing behind. The stray file is not at a path any analyzer
715 // reads, but `digest_tree` covers the whole asset directory — so a
716 // later successful provision (of the remaining files, or after the
717 // operator placed this one by hand) would fold these bytes into the
718 // recorded pin, and removing them afterwards would then read as
719 // tampering.
720 std::fs::remove_file(&partial).ok();
721 AssetError::Fetch {
722 id: spec.id,
723 url: file.url,
724 message,
725 }
726 })?;
727 std::fs::rename(&partial, &destination).map_err(|source| {
728 std::fs::remove_file(&partial).ok();
729 AssetError::Io {
730 path: destination.display().to_string(),
731 source,
732 }
733 })?;
734 }
735 Ok(())
736}
737
738/// The pinned archive for the host this is running on.
739///
740/// # Errors
741/// Returns [`AssetError::UnsupportedPlatform`] naming the platforms that are
742/// pinned, for a host that is not one of them.
743pub fn archive_for_host(
744 spec: &AssetSpec,
745 archives: &'static [crate::runtime_pins::PinnedArchive],
746) -> Result<&'static crate::runtime_pins::PinnedArchive, AssetError> {
747 // Searched in the slice the *spec* carries, not in the global table. They
748 // are the same slice in production, and keeping the lookup parameterised is
749 // what lets the pin be exercised without shipping a fake into the real one.
750 crate::runtime_pins::runtime_target(std::env::consts::OS, std::env::consts::ARCH)
751 .and_then(|target| archives.iter().find(|a| a.target == target))
752 .ok_or_else(|| AssetError::UnsupportedPlatform {
753 id: spec.id,
754 os: std::env::consts::OS,
755 arch: std::env::consts::ARCH,
756 supported: archives
757 .iter()
758 .map(|a| a.target)
759 .collect::<Vec<_>>()
760 .join(", "),
761 })
762}
763
764/// Install the host's pinned archive, verifying its digest before it counts.
765///
766/// Idempotent and offline over a warm cache: an archive already present *and
767/// matching its pin* is accepted without a fetcher, which is what lets a machine
768/// with no network re-run `prefetch` and get a clean bill rather than a refusal.
769/// An archive present but **not** matching is refused rather than re-fetched —
770/// silently replacing bytes that failed verification would turn a tamper signal
771/// into a retry.
772fn provision_archive(
773 spec: &AssetSpec,
774 archives: &'static [crate::runtime_pins::PinnedArchive],
775 target: &Path,
776 fetch: Option<&Fetcher<'_>>,
777) -> Result<String, AssetError> {
778 let archive = archive_for_host(spec, archives)?;
779
780 if target.is_file() {
781 // Present already: verify, and take it or refuse it. Either way no
782 // network is touched, which is the whole point of a warm cache.
783 return verify_archive(spec, archive, target, &target.display().to_string());
784 }
785
786 let Some(fetch) = fetch else {
787 return Err(AssetError::ArchiveMissing {
788 id: spec.id,
789 target: archive.target,
790 path: target.display().to_string(),
791 });
792 };
793
794 if let Some(parent) = target.parent() {
795 std::fs::create_dir_all(parent).map_err(|source| AssetError::Io {
796 path: parent.display().to_string(),
797 source,
798 })?;
799 }
800
801 // Stage beside the destination, verify, and only then rename. A body that
802 // fails its pin never appears at the path anything reads — so a failed
803 // provision leaves a cold cache rather than a poisoned one.
804 let partial = target.with_extension("partial");
805 std::fs::remove_file(&partial).ok();
806 fetch(archive.url, &partial).map_err(|message| {
807 std::fs::remove_file(&partial).ok();
808 AssetError::Fetch {
809 id: spec.id,
810 url: archive.url,
811 message,
812 }
813 })?;
814
815 let digest = match verify_archive(spec, archive, &partial, archive.url) {
816 Ok(digest) => digest,
817 Err(e) => {
818 std::fs::remove_file(&partial).ok();
819 return Err(e);
820 }
821 };
822
823 std::fs::rename(&partial, target).map_err(|source| {
824 std::fs::remove_file(&partial).ok();
825 AssetError::Io {
826 path: target.display().to_string(),
827 source,
828 }
829 })?;
830 Ok(digest)
831}
832
833/// Check a file against a pinned archive, returning its digest when it matches.
834///
835/// `origin` is what the failure message blames — a URL when the bytes just
836/// arrived from one, a path when they were already on disk.
837///
838/// # Errors
839/// Returns [`AssetError::DigestMismatch`] when the bytes are not the pinned
840/// bytes, or [`AssetError::Io`] when the file cannot be read.
841pub fn verify_archive(
842 spec: &AssetSpec,
843 archive: &crate::runtime_pins::PinnedArchive,
844 path: &Path,
845 origin: &str,
846) -> Result<String, AssetError> {
847 let bytes = std::fs::read(path).map_err(|source| AssetError::Io {
848 path: path.display().to_string(),
849 source,
850 })?;
851 let digest = sha256_hex(&bytes);
852 let actual_bytes = bytes.len() as u64;
853 if digest != archive.sha256 || actual_bytes != archive.bytes {
854 return Err(AssetError::DigestMismatch {
855 id: spec.id,
856 url: origin.to_owned(),
857 expected: archive.sha256,
858 expected_bytes: archive.bytes,
859 actual: digest,
860 actual_bytes,
861 });
862 }
863 Ok(digest)
864}
865
866/// Whether a declared install path stays inside the asset directory.
867///
868/// Compiled-in paths today, but they name where bytes from the network land, and
869/// the check costs nothing.
870fn is_safe_relative(path: &str) -> bool {
871 !path.is_empty()
872 && !Path::new(path).components().any(|component| {
873 matches!(
874 component,
875 std::path::Component::RootDir
876 | std::path::Component::Prefix(_)
877 | std::path::Component::ParentDir
878 )
879 })
880}
881
882/// The provisioning record for an asset, or `None` if it was never provisioned
883/// or the record is unreadable.
884///
885/// An unreadable record is treated as absent rather than as an error: the
886/// remedy is the same — run `prefetch` — and a corrupt cache file should not
887/// make `status` fail.
888#[must_use]
889pub fn installed(root: &Path, spec: &AssetSpec) -> Option<InstalledAsset> {
890 let bytes = std::fs::read(record_path(root, spec)).ok()?;
891 serde_json::from_slice(&bytes).ok()
892}
893
894/// The state of every asset this build knows about, for `roteiro security
895/// status`.
896#[must_use]
897pub fn status(root: &Path, analyzer: Option<&str>) -> Vec<AssetStatus> {
898 ASSETS
899 .iter()
900 .filter(|spec| analyzer.is_none_or(|name| spec.analyzer == name))
901 .map(|spec| {
902 let installed = installed(root, spec);
903 let now = rfc3339_utc(std::time::SystemTime::now());
904 let age_days = installed
905 .as_ref()
906 .and_then(|record| age_in_days(&record.fetched_at, &now));
907 // Re-digest what is on disk. A record that no longer matches the
908 // bytes is exactly the case a status command exists to surface, and
909 // reporting the record alone would hide it.
910 let verified = installed.as_ref().map(|record| {
911 current_digest(root, spec).as_deref() == Some(record.digest.as_str())
912 });
913 AssetStatus {
914 id: spec.id,
915 analyzer: spec.analyzer,
916 kind: spec.kind,
917 path: asset_path(root, spec).display().to_string(),
918 installed,
919 age_days,
920 verified,
921 }
922 })
923 .collect()
924}
925
926/// When the contents at `dir` were published, if that can be established.
927///
928/// A git checkout's `HEAD` commit time is the publication date. Anything that is
929/// not a git checkout has no such date, and `None` is reported rather than
930/// invented — a made-up publication date would make a stale database look fresh,
931/// which is the one failure mode this whole field exists to prevent.
932fn published_at(dir: &Path) -> Option<String> {
933 let repo = rto_graph::Repo::discover(dir).ok()?;
934 // `discover` walks upwards, so a directory that is merely *inside* a
935 // repository would otherwise be dated by that repository's HEAD.
936 if repo.workdir()? != dir {
937 return None;
938 }
939 let seconds = repo.head_commit_time().ok()?;
940 Some(rfc3339_utc(
941 std::time::UNIX_EPOCH + std::time::Duration::from_secs(u64::try_from(seconds).ok()?),
942 ))
943}
944
945/// The advisory-database evidence recorded for `analyzer` at provisioning time.
946///
947/// Supplied to a run so its results carry a database identity and publication
948/// date even though the analyzer itself reports neither.
949#[must_use]
950pub fn advisory_db_evidence(root: &Path, analyzer: &str) -> Option<rto_graph::AdvisoryDb> {
951 let spec = assets_for(analyzer)
952 .into_iter()
953 .find(|s| s.kind == AssetKind::AdvisoryDb)?;
954 let record = installed(root, spec)?;
955 Some(rto_graph::AdvisoryDb {
956 digest: record.digest,
957 published_at: record.published_at,
958 })
959}
960
961/// Digest of what is on disk right now, or `None` if it is not there.
962fn current_digest(root: &Path, spec: &AssetSpec) -> Option<String> {
963 let target = asset_path(root, spec);
964 match spec.source {
965 AssetSource::Vendored(_) | AssetSource::PinnedArchive { .. } => {
966 Some(sha256_hex(&std::fs::read(target).ok()?))
967 }
968 AssetSource::External { .. } | AssetSource::Download { .. } => {
969 digest_tree(&target).ok().map(|(digest, _)| digest)
970 }
971 }
972}
973
974/// Resolve every asset `analyzer` needs to a verified local path.
975///
976/// # Errors
977/// Returns [`ExecError::AssetsUnavailableOffline`] naming every asset that is
978/// missing or whose bytes no longer match what was recorded, together with the
979/// exact prefetch command. It never fetches, and it never falls back to a
980/// host-installed copy.
981pub fn resolve(root: &Path, analyzer: &str) -> Result<Vec<(&'static str, PathBuf)>, ExecError> {
982 let specs = assets_for(analyzer);
983 let mut resolved = Vec::with_capacity(specs.len());
984 let mut missing = Vec::new();
985
986 for spec in specs {
987 let path = asset_path(root, spec);
988 match (installed(root, spec), current_digest(root, spec)) {
989 // Provisioned, and the bytes still match what was recorded.
990 (Some(record), Some(digest)) if digest == record.digest => {
991 resolved.push((spec.id, path));
992 }
993 // Provisioned, but the bytes changed underneath the record. That is
994 // not a warning: a run would stamp a digest that does not describe
995 // what it read.
996 (Some(record), Some(_)) => missing.push(MissingAsset {
997 id: spec.id.to_owned(),
998 digest: record.digest.clone(),
999 reason: "the bytes on disk no longer match the recorded digest",
1000 }),
1001 (Some(record), None) => missing.push(MissingAsset {
1002 id: spec.id.to_owned(),
1003 digest: record.digest.clone(),
1004 reason: "recorded as provisioned, but nothing is there now",
1005 }),
1006 (None, _) => missing.push(MissingAsset {
1007 id: spec.id.to_owned(),
1008 digest: "not yet pinned".to_owned(),
1009 reason: "never provisioned",
1010 }),
1011 }
1012 }
1013
1014 if missing.is_empty() {
1015 Ok(resolved)
1016 } else {
1017 Err(ExecError::AssetsUnavailableOffline {
1018 analyzer: analyzer.to_owned(),
1019 missing,
1020 command: format!("roteiro security prefetch --analyzer {analyzer}"),
1021 })
1022 }
1023}
1024
1025/// One asset a run needed and did not have.
1026#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
1027pub struct MissingAsset {
1028 /// The asset id.
1029 pub id: String,
1030 /// The digest that was pinned for it, or a note that none is.
1031 pub digest: String,
1032 /// Why it could not be used.
1033 pub reason: &'static str,
1034}
1035
1036impl std::fmt::Display for MissingAsset {
1037 fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
1038 write!(f, "{} ({}; {})", self.id, self.digest, self.reason)
1039 }
1040}
1041
1042/// Digest of a directory tree, plus the number of files it covered.
1043///
1044/// The digest is over the sorted list of `(relative path, sha256(bytes))`, so it
1045/// is a function of the tree's contents alone: independent of directory
1046/// iteration order, of timestamps, and of where the tree happens to be mounted.
1047/// `.git` is skipped — it is bookkeeping, not advisory data, and including it
1048/// would make the digest churn on every fetch that changed nothing.
1049fn digest_tree(dir: &Path) -> Result<(String, usize), AssetError> {
1050 let mut entries: BTreeMap<String, String> = BTreeMap::new();
1051 walk(dir, dir, &mut entries)?;
1052 let mut manifest = String::new();
1053 for (path, digest) in &entries {
1054 use std::fmt::Write as _;
1055 let _ = writeln!(manifest, "{digest} {path}");
1056 }
1057 Ok((sha256_hex(manifest.as_bytes()), entries.len()))
1058}
1059
1060fn walk(root: &Path, dir: &Path, into: &mut BTreeMap<String, String>) -> Result<(), AssetError> {
1061 let read = std::fs::read_dir(dir).map_err(|source| AssetError::Io {
1062 path: dir.display().to_string(),
1063 source,
1064 })?;
1065 for entry in read {
1066 let entry = entry.map_err(|source| AssetError::Io {
1067 path: dir.display().to_string(),
1068 source,
1069 })?;
1070 let path = entry.path();
1071 // `symlink_metadata` rather than `metadata`: a symlink out of the tree
1072 // must not be followed into a file the digest has no business reading.
1073 let meta = std::fs::symlink_metadata(&path).map_err(|source| AssetError::Io {
1074 path: path.display().to_string(),
1075 source,
1076 })?;
1077 if meta.is_symlink() {
1078 continue;
1079 }
1080 if meta.is_dir() {
1081 if path.file_name().is_some_and(|n| n == ".git") {
1082 continue;
1083 }
1084 walk(root, &path, into)?;
1085 } else if meta.is_file() {
1086 let bytes = std::fs::read(&path).map_err(|source| AssetError::Io {
1087 path: path.display().to_string(),
1088 source,
1089 })?;
1090 let relative = path
1091 .strip_prefix(root)
1092 .unwrap_or(&path)
1093 .to_string_lossy()
1094 .replace('\\', "/");
1095 into.insert(relative, sha256_hex(&bytes));
1096 }
1097 }
1098 Ok(())
1099}
1100
1101/// Write `bytes` to `path` via a temp file and a rename, so a reader never sees
1102/// a half-written asset — the same discipline `rto_graph::download_verified`
1103/// applies to a model file.
1104fn write_atomically(path: &Path, bytes: &[u8]) -> Result<(), AssetError> {
1105 let io = |source| AssetError::Io {
1106 path: path.display().to_string(),
1107 source,
1108 };
1109 let tmp = path.with_extension("partial");
1110 std::fs::write(&tmp, bytes).map_err(io)?;
1111 if path.exists() {
1112 std::fs::remove_file(path).map_err(io)?;
1113 }
1114 std::fs::rename(&tmp, path).map_err(|source| {
1115 std::fs::remove_file(&tmp).ok();
1116 AssetError::Io {
1117 path: path.display().to_string(),
1118 source,
1119 }
1120 })
1121}
1122
1123#[cfg(test)]
1124mod tests {
1125 use super::{
1126 ASSETS, AssetError, AssetKind, AssetSource, SANDBOX, asset, asset_path, assets_for,
1127 installed, provision, resolve, status,
1128 };
1129 use crate::runner::ExecError;
1130 use std::path::PathBuf;
1131
1132 /// A throwaway cache root that removes itself.
1133 struct Cache(PathBuf);
1134
1135 impl Cache {
1136 fn new(name: &str) -> Self {
1137 let dir = std::env::temp_dir().join(format!("rto-exec-assets-{name}"));
1138 std::fs::remove_dir_all(&dir).ok();
1139 std::fs::create_dir_all(&dir).expect("create");
1140 Self(dir)
1141 }
1142 }
1143
1144 impl Drop for Cache {
1145 fn drop(&mut self) {
1146 std::fs::remove_dir_all(&self.0).ok();
1147 }
1148 }
1149
1150 fn rules() -> &'static super::AssetSpec {
1151 asset("semgrep-rules").expect("the baseline rule set is a known asset")
1152 }
1153
1154 fn advisory_db() -> &'static super::AssetSpec {
1155 asset("rustsec-advisory-db").expect("the advisory database is a known asset")
1156 }
1157
1158 /// Every asset is reachable from something that wants it — either an
1159 /// analyzer's adapter, or the shared sandbox, which no single analyzer owns.
1160 ///
1161 /// The `SANDBOX` arm is not a loophole: an asset that claims to belong to an
1162 /// analyzer and is not in that analyzer's `asset_ids` would be provisioned
1163 /// and never used, which is the case this test exists to catch.
1164 #[test]
1165 fn every_asset_belongs_to_an_analyzer_that_asked_for_it() {
1166 for spec in ASSETS {
1167 if spec.analyzer == super::SANDBOX {
1168 assert!(
1169 assets_for(spec.analyzer).is_empty(),
1170 "{} uses the shared-asset sentinel, so no adapter may claim it",
1171 spec.id
1172 );
1173 } else {
1174 assert!(
1175 assets_for(spec.analyzer).iter().any(|s| s.id == spec.id),
1176 "{} is not claimed by {}",
1177 spec.id,
1178 spec.analyzer
1179 );
1180 }
1181 assert!(!spec.licence.is_empty(), "{} discloses no licence", spec.id);
1182 }
1183 }
1184
1185 /// `--analyzer sandbox` selects the runtime archive, and selects only it.
1186 ///
1187 /// Both halves are asserted because the interesting behaviour is the join of
1188 /// them: `assets_for` returns nothing for the sandbox — no adapter declares
1189 /// the archive — which is what sends `run_security_prefetch` to its
1190 /// fallback, and the fallback selects by [`AssetSpec::analyzer`]. If either
1191 /// half moved, `prefetch --analyzer sandbox` would quietly start fetching
1192 /// either nothing or a quarter-gigabyte of advisory databases, and the
1193 /// bootstrap recipe in `build.rs`, `README.md` and `AGENTS.md` would be
1194 /// wrong without a single test going red.
1195 ///
1196 /// That is not hypothetical: the comment on the spec described the
1197 /// pre-fallback rule for long enough to ship a wrong recipe and nearly cost
1198 /// a correct review comment its adjudication (#362).
1199 #[test]
1200 fn the_sandbox_analyzer_selects_the_runtime_archive_alone() {
1201 assert!(
1202 assets_for(SANDBOX).is_empty(),
1203 "no adapter should declare the shared runtime; if one does, the fallback in \
1204 run_security_prefetch is no longer what selects it"
1205 );
1206
1207 let by_owner: Vec<&str> = ASSETS
1208 .iter()
1209 .filter(|spec| spec.analyzer == SANDBOX)
1210 .map(|spec| spec.id)
1211 .collect();
1212 assert_eq!(
1213 by_owner,
1214 vec![crate::runtime_pins::RUNTIME_ASSET],
1215 "`prefetch --analyzer sandbox` resolves by owner, so this is exactly what it \
1216 provisions — it must be the runtime archive and nothing else"
1217 );
1218 }
1219
1220 /// The sandbox runtime's disclosure must name every licence family in the
1221 /// archive, not flatten them into one word.
1222 ///
1223 /// Flattening is precisely how 25 MB of GPL binaries travelled through a
1224 /// licence gate that reported `licenses ok`. A reader of `prefetch`'s output
1225 /// is entitled to see what they are about to install.
1226 #[test]
1227 fn the_sandbox_runtime_discloses_every_licence_it_carries() {
1228 let spec = asset(crate::runtime_pins::RUNTIME_ASSET).expect("the runtime is a known asset");
1229 assert_eq!(spec.kind, AssetKind::SandboxRuntime);
1230 for family in ["Apache-2.0", "GPL-2.0", "LGPL-2.0"] {
1231 assert!(
1232 spec.licence.contains(family),
1233 "the disclosure does not mention {family}: {}",
1234 spec.licence
1235 );
1236 }
1237 assert!(
1238 spec.licence.contains("NOTICE-boxlite-runtime.md"),
1239 "the disclosure must point at the full record: {}",
1240 spec.licence
1241 );
1242 }
1243
1244 /// Every pinned archive must carry a full digest and a real size, and the
1245 /// set must cover exactly the platforms `runtime_target` claims — a target
1246 /// that maps to no archive would fail at build time with nothing to say.
1247 #[test]
1248 fn every_pinned_archive_is_complete_and_reachable() {
1249 use crate::runtime_pins::{RUNTIME_ARCHIVES, archive_for, runtime_target};
1250 assert!(!RUNTIME_ARCHIVES.is_empty());
1251 for archive in RUNTIME_ARCHIVES {
1252 assert_eq!(
1253 archive.sha256.len(),
1254 64,
1255 "{} has no full sha256",
1256 archive.target
1257 );
1258 assert!(
1259 archive
1260 .sha256
1261 .chars()
1262 .all(|c| c.is_ascii_hexdigit() && !c.is_uppercase()),
1263 "{} digest must be lowercase hex",
1264 archive.target
1265 );
1266 assert!(
1267 archive.bytes > 1_000_000,
1268 "{} size looks wrong",
1269 archive.target
1270 );
1271 assert!(
1272 archive.url.ends_with(".tar.gz") && archive.url.contains(archive.target),
1273 "{} url does not name the target it is for: {}",
1274 archive.target,
1275 archive.url
1276 );
1277 }
1278 for (os, arch) in [
1279 ("macos", "aarch64"),
1280 ("linux", "x86_64"),
1281 ("linux", "aarch64"),
1282 ] {
1283 let target = runtime_target(os, arch).expect("a pinned platform");
1284 let archive = archive_for(os, arch).expect("must resolve to an archive");
1285 assert_eq!(archive.target, target);
1286 }
1287 assert!(runtime_target("windows", "x86_64").is_none());
1288 assert!(archive_for("windows", "x86_64").is_none());
1289 }
1290
1291 /// A digest that does not match is refused, and the refusal says which
1292 /// bytes were expected — including the size, because a truncated body is
1293 /// the common failure and two unequal digests do not say so.
1294 #[test]
1295 fn a_pinned_archive_that_does_not_match_is_refused() {
1296 use crate::runtime_pins::PinnedArchive;
1297 let cache = Cache::new("pinned-mismatch");
1298 let spec = asset(crate::runtime_pins::RUNTIME_ASSET).expect("known asset");
1299 let archive = PinnedArchive {
1300 target: "test-target",
1301 url: "https://example.invalid/runtime.tar.gz",
1302 sha256: "0000000000000000000000000000000000000000000000000000000000000000",
1303 bytes: 999,
1304 };
1305 let path = cache.0.join("impostor.tar.gz");
1306 std::fs::write(&path, b"not the pinned bytes").expect("write");
1307
1308 let err = super::verify_archive(spec, &archive, &path, archive.url)
1309 .expect_err("bytes that do not match the pin must be refused");
1310 let message = err.to_string();
1311 assert!(matches!(err, AssetError::DigestMismatch { .. }));
1312 assert!(message.contains(archive.sha256), "{message}");
1313 assert!(message.contains("999 bytes"), "{message}");
1314 assert!(message.contains("20 bytes"), "{message}");
1315 }
1316
1317 /// Provisioning a pinned archive without a fetcher is refused by name, and
1318 /// names the command that fixes it — the same offline contract every other
1319 /// asset kind follows.
1320 #[test]
1321 fn a_pinned_archive_is_not_fetched_by_a_path_that_may_not_download() {
1322 let cache = Cache::new("pinned-cold");
1323 let spec = asset(crate::runtime_pins::RUNTIME_ASSET).expect("known asset");
1324 let err = provision(&cache.0, spec).expect_err("a cold cache must refuse");
1325 // A host with no pinned archive fails earlier, and differently — both
1326 // are correct refusals, and asserting the property rather than one
1327 // literal keeps this test honest on an unpinned platform.
1328 let message = err.to_string();
1329 match err {
1330 AssetError::ArchiveMissing { .. } => {
1331 assert!(message.contains("prefetch --allow-download"), "{message}");
1332 }
1333 AssetError::UnsupportedPlatform { .. } => {
1334 assert!(message.contains("pinned platforms are"), "{message}");
1335 }
1336 other => panic!("unexpected refusal: {other}"),
1337 }
1338 }
1339
1340 /// A spec pinned to `body`, for the host platform, without touching the
1341 /// shipped pins.
1342 ///
1343 /// Leaked because [`AssetSource::PinnedArchive`] holds `&'static` data — a
1344 /// few bytes per test process, and the alternative is either a fake entry in
1345 /// the real table or not exercising the pin at all.
1346 fn pinned_to(body: &[u8]) -> Option<&'static super::AssetSpec> {
1347 let target =
1348 crate::runtime_pins::runtime_target(std::env::consts::OS, std::env::consts::ARCH)?;
1349 let archives: &'static [crate::runtime_pins::PinnedArchive] =
1350 Box::leak(Box::new([crate::runtime_pins::PinnedArchive {
1351 target,
1352 url: "https://example.invalid/runtime.tar.gz",
1353 sha256: Box::leak(crate::sha256_hex(body).into_boxed_str()),
1354 bytes: body.len() as u64,
1355 }]));
1356 Some(Box::leak(Box::new(super::AssetSpec {
1357 id: "test-pinned-archive",
1358 analyzer: super::SANDBOX,
1359 kind: AssetKind::SandboxRuntime,
1360 source: AssetSource::PinnedArchive { archives },
1361 file: "fixture.tar.gz",
1362 licence: "test fixture",
1363 })))
1364 }
1365
1366 /// A warm cache provisions with **no fetcher at all**, and is still
1367 /// verified.
1368 ///
1369 /// This is what makes "no network, warm cache" a real claim rather than an
1370 /// aspiration — and the first half is the one that matters most: an archive
1371 /// already on disk whose bytes do not match the pin is *refused*, so a warm
1372 /// cache can never become a way around the pin.
1373 #[test]
1374 fn a_warm_pinned_archive_provisions_offline_and_is_still_verified() {
1375 let body = b"pretend this is a runtime archive".to_vec();
1376 let Some(spec) = pinned_to(&body) else {
1377 eprintln!(
1378 "SKIPPED: no sandbox runtime is pinned for {}/{}",
1379 std::env::consts::OS,
1380 std::env::consts::ARCH
1381 );
1382 return;
1383 };
1384 let cache = Cache::new("pinned-warm");
1385 let target = asset_path(&cache.0, spec);
1386 std::fs::create_dir_all(target.parent().expect("parent")).expect("mkdir");
1387
1388 // Right pin, wrong bytes: refused, without a fetcher ever being offered.
1389 std::fs::write(&target, b"tampered").expect("write");
1390 let err = provision(&cache.0, spec).expect_err("a warm cache is still verified");
1391 assert!(matches!(err, AssetError::DigestMismatch { .. }), "{err}");
1392
1393 // The pinned bytes: provisions offline, with no fetcher at all.
1394 std::fs::write(&target, &body).expect("write");
1395 let record = provision(&cache.0, spec).expect("a matching warm cache provisions offline");
1396 assert_eq!(record.kind, AssetKind::SandboxRuntime);
1397 assert_eq!(record.digest, crate::sha256_hex(&body));
1398
1399 // And `resolve`-style re-verification agrees the bytes are still right.
1400 assert_eq!(
1401 super::current_digest(&cache.0, spec).as_deref(),
1402 Some(record.digest.as_str())
1403 );
1404 }
1405
1406 /// A fetcher that returns success over the wrong bytes cannot poison the
1407 /// cache: the archive is verified *before* it is renamed into place, so a
1408 /// failed provision leaves a cold cache rather than a bad one.
1409 ///
1410 /// This is the case [`Fetcher`]'s contract cannot cover for a `Download`
1411 /// asset, and the reason `PinnedArchive` exists.
1412 #[test]
1413 fn a_lying_fetcher_cannot_install_a_pinned_archive() {
1414 let body = b"the real runtime archive".to_vec();
1415 let Some(spec) = pinned_to(&body) else {
1416 eprintln!("SKIPPED: no sandbox runtime is pinned for this platform");
1417 return;
1418 };
1419 let cache = Cache::new("pinned-lying-fetcher");
1420
1421 let liar: &super::Fetcher<'_> = &|_url: &str, dest: &std::path::Path| {
1422 std::fs::write(dest, b"truncated").map_err(|e| e.to_string())
1423 };
1424 let err = super::provision_with(&cache.0, spec, Some(liar))
1425 .expect_err("bytes that do not match the pin must be refused");
1426 assert!(matches!(err, AssetError::DigestMismatch { .. }), "{err}");
1427
1428 // Nothing was left behind at the path anything reads, and no staging
1429 // file survived to be folded into a later digest.
1430 let target = asset_path(&cache.0, spec);
1431 assert!(!target.exists(), "a refused archive must not be installed");
1432 assert!(
1433 !target.with_extension("partial").exists(),
1434 "staging file left behind"
1435 );
1436
1437 // An honest fetcher then provisions normally.
1438 let honest: &super::Fetcher<'_> = &|_url: &str, dest: &std::path::Path| {
1439 std::fs::write(dest, b"the real runtime archive").map_err(|e| e.to_string())
1440 };
1441 let record = super::provision_with(&cache.0, spec, Some(honest)).expect("provision");
1442 assert_eq!(record.digest, crate::sha256_hex(&body));
1443 }
1444
1445 #[test]
1446 fn provisioning_a_vendored_asset_installs_and_records_it() {
1447 let cache = Cache::new("vendored");
1448 let record = provision(&cache.0, rules()).expect("provision");
1449 assert_eq!(record.kind, AssetKind::Rules);
1450 assert_eq!(record.digest.len(), 64);
1451 assert!(!record.fetched_at.is_empty());
1452
1453 // The file is really there, and is really the vendored bytes.
1454 let AssetSource::Vendored(bytes) = rules().source else {
1455 panic!("the rule set is a vendored asset");
1456 };
1457 assert_eq!(
1458 std::fs::read(asset_path(&cache.0, rules())).expect("read"),
1459 bytes
1460 );
1461 assert_eq!(installed(&cache.0, rules()), Some(record));
1462 }
1463
1464 /// `prefetch` is a thing you run when unsure, so running it twice must be
1465 /// harmless and must not change what a run will read.
1466 #[test]
1467 fn provisioning_is_idempotent() {
1468 let cache = Cache::new("idempotent");
1469 let first = provision(&cache.0, rules()).expect("first");
1470 let second = provision(&cache.0, rules()).expect("second");
1471 assert_eq!(first.digest, second.digest);
1472 }
1473
1474 /// The headline offline contract: a cold cache fails, names what is missing,
1475 /// and prints the exact command that fixes it.
1476 #[test]
1477 fn a_cold_cache_fails_with_the_named_offline_error() {
1478 let cache = Cache::new("cold");
1479 let err = resolve(&cache.0, "semgrep").expect_err("a cold cache must fail");
1480 let ExecError::AssetsUnavailableOffline {
1481 analyzer,
1482 missing,
1483 command,
1484 } = &err
1485 else {
1486 panic!("expected the offline error, got {err:?}");
1487 };
1488 assert_eq!(analyzer, "semgrep");
1489 assert_eq!(missing.len(), 1);
1490 assert_eq!(missing[0].id, "semgrep-rules");
1491 assert_eq!(command, "roteiro security prefetch --analyzer semgrep");
1492
1493 // The rendered message has to carry all of it, because that is what a
1494 // user on a plane actually reads.
1495 let message = err.to_string();
1496 assert!(message.contains("assets-unavailable-offline"), "{message}");
1497 assert!(message.contains("semgrep-rules"), "{message}");
1498 assert!(
1499 message.contains("roteiro security prefetch --analyzer semgrep"),
1500 "{message}"
1501 );
1502 }
1503
1504 #[test]
1505 fn a_warm_cache_resolves_to_the_provisioned_path() {
1506 let cache = Cache::new("warm");
1507 provision(&cache.0, rules()).expect("provision");
1508 let resolved = resolve(&cache.0, "semgrep").expect("a warm cache must resolve");
1509 assert_eq!(resolved.len(), 1);
1510 assert_eq!(resolved[0].0, "semgrep-rules");
1511 assert_eq!(resolved[0].1, asset_path(&cache.0, rules()));
1512 }
1513
1514 /// A record that no longer describes the bytes is worse than no record: a
1515 /// run would stamp a `rules_digest` that does not match what it read.
1516 #[test]
1517 fn an_asset_edited_after_provisioning_is_refused_not_warned_about() {
1518 let cache = Cache::new("tampered");
1519 provision(&cache.0, rules()).expect("provision");
1520 std::fs::write(asset_path(&cache.0, rules()), b"rules: []\n").expect("tamper");
1521
1522 let err = resolve(&cache.0, "semgrep").expect_err("tampering must be refused");
1523 let ExecError::AssetsUnavailableOffline { missing, .. } = &err else {
1524 panic!("expected the offline error");
1525 };
1526 assert!(
1527 missing[0].reason.contains("no longer match"),
1528 "{}",
1529 missing[0].reason
1530 );
1531 }
1532
1533 /// Roteiro never fetches the advisory database. Absent, it says where it
1534 /// looked and what to run — and does not go and get it.
1535 #[test]
1536 fn an_absent_external_asset_is_explained_never_fetched() {
1537 let cache = Cache::new("external");
1538 let err = provision(&cache.0, advisory_db()).expect_err("must not be fetched");
1539 let AssetError::ExternalMissing { hint, analyzer, .. } = &err else {
1540 panic!("expected ExternalMissing, got {err:?}");
1541 };
1542 assert_eq!(*analyzer, "cargo-audit");
1543 assert!(hint.contains("advisory-db"), "{hint}");
1544 assert!(
1545 err.to_string().contains("roteiro security prefetch"),
1546 "{err}"
1547 );
1548 }
1549
1550 #[test]
1551 fn a_directory_asset_is_digested_by_content_not_by_layout() {
1552 let cache = Cache::new("tree");
1553 let db = asset_path(&cache.0, advisory_db());
1554 std::fs::create_dir_all(db.join("crates/openssl")).expect("create");
1555 std::fs::write(db.join("crates/openssl/RUSTSEC-2026-0031.md"), b"a").expect("write");
1556 std::fs::write(db.join("README.md"), b"b").expect("write");
1557
1558 let first = provision(&cache.0, advisory_db()).expect("provision");
1559 assert_eq!(first.files, Some(2));
1560
1561 // A `.git` directory is bookkeeping, not advisory data: adding one must
1562 // not move the digest.
1563 std::fs::create_dir_all(db.join(".git")).expect("create");
1564 std::fs::write(db.join(".git/HEAD"), b"ref: refs/heads/main").expect("write");
1565 assert_eq!(
1566 provision(&cache.0, advisory_db()).expect("again").digest,
1567 first.digest
1568 );
1569
1570 // Changing an advisory does move it.
1571 std::fs::write(db.join("README.md"), b"c").expect("write");
1572 assert_ne!(
1573 provision(&cache.0, advisory_db()).expect("third").digest,
1574 first.digest
1575 );
1576 }
1577
1578 #[test]
1579 fn status_reports_what_is_provisioned_and_what_is_not() {
1580 let cache = Cache::new("status");
1581 let cold = status(&cache.0, Some("semgrep"));
1582 assert_eq!(cold.len(), 1);
1583 assert!(cold[0].installed.is_none());
1584 assert!(cold[0].verified.is_none());
1585 assert!(cold[0].age_days.is_none());
1586
1587 provision(&cache.0, rules()).expect("provision");
1588 let warm = status(&cache.0, Some("semgrep"));
1589 assert_eq!(warm[0].verified, Some(true));
1590 assert_eq!(warm[0].age_days, Some(0));
1591 assert_eq!(warm[0].installed.as_ref().map(|r| r.digest.len()), Some(64));
1592
1593 // …and it notices when the bytes stop matching.
1594 std::fs::write(asset_path(&cache.0, rules()), b"rules: []\n").expect("tamper");
1595 assert_eq!(status(&cache.0, Some("semgrep"))[0].verified, Some(false));
1596 }
1597
1598 #[test]
1599 fn status_covers_every_analyzer_when_none_is_named() {
1600 let cache = Cache::new("status-all");
1601 assert_eq!(status(&cache.0, None).len(), ASSETS.len());
1602 assert!(status(&cache.0, Some("no-such-analyzer")).is_empty());
1603 }
1604
1605 /// The install paths of a downloadable asset name where bytes from the
1606 /// network are written, so a `..` in one would write outside the asset
1607 /// cache. They are compiled in today, which is exactly why the check is
1608 /// worth having: nothing else would notice a typo that escaped.
1609 #[test]
1610 fn a_download_path_that_escapes_the_asset_directory_is_refused() {
1611 static ESCAPING: &[super::DownloadFile] = &[super::DownloadFile {
1612 path: "../../outside.zip",
1613 url: "https://example.invalid/outside.zip",
1614 }];
1615 let cache = Cache::new("escape");
1616 let spec = super::AssetSpec {
1617 id: "escaping-asset",
1618 analyzer: "osv-scanner",
1619 kind: AssetKind::AdvisoryDb,
1620 source: AssetSource::Download { files: ESCAPING },
1621 file: "",
1622 licence: "n/a",
1623 };
1624 let fetched = std::cell::Cell::new(false);
1625 let fetch = |_: &str, _: &std::path::Path| {
1626 fetched.set(true);
1627 Ok(())
1628 };
1629 let err = super::provision_with(&cache.0, &spec, Some(&fetch))
1630 .expect_err("an escaping path must be refused");
1631 assert!(
1632 matches!(err, AssetError::UnsafeInstallPath { .. }),
1633 "{err:?}"
1634 );
1635 assert!(
1636 !fetched.get(),
1637 "the path is checked before anything is fetched"
1638 );
1639 }
1640
1641 /// An analyzer this build cannot run has no assets, and asking for them is
1642 /// not an error — it is simply an empty answer.
1643 #[test]
1644 fn an_unknown_analyzer_needs_nothing() {
1645 assert!(assets_for("no-such-analyzer").is_empty());
1646 let cache = Cache::new("unknown");
1647 assert!(
1648 resolve(&cache.0, "no-such-analyzer")
1649 .expect("no assets")
1650 .is_empty()
1651 );
1652 }
1653}