rto_exec/assets.rs
1//! Pinned analyzer assets: what a run needs before it can happen, and what
2//! happens when it is not there.
3//!
4//! ADR-0014's working model is *mostly offline, degrade gracefully, pre-download
5//! expected*. That is a provisioning contract, and this module is it:
6//!
7//! - **`roteiro security prefetch`** installs and verifies every pinned asset an
8//! analyzer needs, recording its digest and the time it was fetched.
9//! - **`roteiro security status`** reports each digest and fetch time.
10//! - **A run never provisions.** Cold cache fails with
11//! [`ExecError::AssetsUnavailableOffline`], which names the missing assets,
12//! their pinned digests, and the exact command to fix it. Never an implicit
13//! fetch; never a silent fall back to whatever the host happens to have
14//! installed.
15//!
16//! # The one rule that makes the rest work
17//!
18//! Provisioning writes; running reads. A run that quietly materialised its own
19//! inputs would make "did this machine have the pinned rules?" unanswerable
20//! after the fact — and the whole point of stamping `rules_digest` onto an
21//! [`rto_graph::AnalysisRun`] is that the question has an answer.
22//!
23//! # Four kinds of asset
24//!
25//! [`AssetSource::Vendored`] is compiled into the binary — the baseline semgrep
26//! rule set. Installing it needs no network at all, which is what makes a fresh
27//! machine on a plane able to run `prefetch` and then scan.
28//!
29//! [`AssetSource::External`] is a directory Roteiro does **not** fetch: the
30//! `RustSec` advisory database, which is a git checkout rather than a file with
31//! a stable URL. `prefetch` verifies it is there, digests it, and records it, so
32//! a run consults a database whose identity was pinned before it started —
33//! rather than whatever `~/.cargo/advisory-db` happened to contain. If it is
34//! absent, `prefetch` says exactly how to obtain it and refuses.
35//!
36//! [`AssetSource::Download`] is fetched by URL, and arrived with `osv-scanner`
37//! in Stage 22b. Earlier revisions of this module said there was deliberately no
38//! such source because "an unused fetch path is a security surface with no
39//! user"; OSV's per-ecosystem databases are that user. They are single files at
40//! stable URLs — exactly what a digest pin wants, and what the `RustSec` git
41//! checkout could never be. The enum being `#[non_exhaustive]` is what made
42//! adding it a non-breaking change.
43//!
44//! [`AssetSource::PinnedArchive`] is the one with a **compile-time digest**, and
45//! it exists for the sandbox runtime (Stage 24). The difference from `Download`
46//! is not the transport but the target: OSV rebuilds its databases daily, so the
47//! only pin that can be honoured there is the snapshot this machine provisioned.
48//! A published release artifact is immutable, so its correct bytes are knowable
49//! in advance — and where they are knowable, they are checked.
50//!
51//! That closes the gap the [`Fetcher`] contract has to leave open elsewhere. A
52//! fetcher that reports success over a truncated body can defeat a `Download`
53//! asset's pin, because there is nothing to contradict it; it cannot defeat a
54//! `PinnedArchive`, because the expected digest is compiled in and the archive
55//! is verified here, in this crate, before it is installed.
56//!
57//! **Fetching is still confined to provisioning.** The transport is not in this
58//! crate at all: [`provision_with`] takes the fetcher as an argument, and the
59//! plain [`provision`] passes one that refuses. A run resolves assets through
60//! [`resolve`], which has no fetcher to call even if it wanted one — so "a run
61//! never provisions" is a property of the signatures rather than a rule someone
62//! has to remember.
63//!
64//! @rto:0014
65
66use std::collections::BTreeMap;
67use std::path::{Path, PathBuf};
68
69use serde::{Deserialize, Serialize};
70
71use crate::adapter::adapter_for;
72use crate::clock::{age_in_days, rfc3339_utc};
73use crate::runner::ExecError;
74use crate::sha256_hex;
75
76/// The baseline semgrep rule set, compiled in.
77///
78/// Vendoring the bytes rather than reading a file at runtime means the asset is
79/// available on a machine that has only the binary — which is the case
80/// `prefetch` exists to serve.
81pub const BASELINE_RULES: &[u8] = include_bytes!(concat!(
82 env!("CARGO_MANIFEST_DIR"),
83 "/rules/roteiro-baseline.yml"
84));
85
86/// Where an asset comes from.
87#[derive(Debug, Clone, Copy, PartialEq, Eq)]
88#[non_exhaustive]
89pub enum AssetSource {
90 /// Bytes compiled into this binary, installed as a single file.
91 Vendored(&'static [u8]),
92 /// A directory the operator provisions, which Roteiro verifies and pins but
93 /// never fetches. `hint` is the exact command that obtains it.
94 External {
95 /// What to run to obtain it, quoted verbatim in every error.
96 hint: &'static str,
97 },
98 /// A set of files downloaded by URL into one directory, digest-pinned at
99 /// provisioning time.
100 ///
101 /// Downloading happens only in [`provision_with`], and only with a fetcher
102 /// the caller supplied. There is no compile-time digest because the upstream
103 /// files are republished continuously — OSV rebuilds its per-ecosystem
104 /// databases daily — so what is pinned is the snapshot this machine
105 /// provisioned, recorded in [`InstalledAsset::digest`] and re-checked on
106 /// every run. That is the same pin the `RustSec` checkout gets, and it is
107 /// the one that can actually be honoured.
108 Download {
109 /// Each file's path relative to the asset directory, and where it comes
110 /// from. Order is preserved so `prefetch` reports progress in a stable
111 /// sequence.
112 files: &'static [DownloadFile],
113 },
114 /// A single published release artifact with a **compile-time SHA-256**,
115 /// installed as one file and selected by host platform.
116 ///
117 /// Verified in this crate, before installation and again by `build.rs`
118 /// before anything is built against it — so neither a lying fetcher nor a
119 /// redirected URL can substitute different bytes. See
120 /// [`crate::runtime_pins`] for what is pinned and why it has to be.
121 PinnedArchive {
122 /// One entry per supported host platform. A host not listed here cannot
123 /// be provisioned, and is told which platforms are.
124 archives: &'static [crate::runtime_pins::PinnedArchive],
125 },
126}
127
128/// One file of an [`AssetSource::Download`] asset.
129#[derive(Debug, Clone, Copy, PartialEq, Eq)]
130pub struct DownloadFile {
131 /// Where it is installed, relative to the asset directory. Forward slashes;
132 /// never absolute and never containing `..`, which [`provision_with`]
133 /// enforces rather than trusts.
134 pub path: &'static str,
135 /// The URL it is fetched from.
136 pub url: &'static str,
137}
138
139/// What kind of input an asset is — the axis along which it goes stale.
140#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
141#[serde(rename_all = "snake_case")]
142pub enum AssetKind {
143 /// A rule set. Changes only when someone changes it.
144 Rules,
145 /// An advisory database. Changes continuously and independently of the
146 /// source tree, which is why results derived from it are labelled *possibly
147 /// stale* rather than *current*.
148 AdvisoryDb,
149 /// The prebuilt sandbox runtime an analyzer is executed inside.
150 ///
151 /// Unlike the other two it is **immutable for a given release**: one
152 /// published artifact with one correct digest, which is why it is the only
153 /// kind carrying a compile-time pin.
154 SandboxRuntime,
155}
156
157impl AssetKind {
158 /// Stable token for display and `--json`.
159 #[must_use]
160 pub fn as_str(self) -> &'static str {
161 match self {
162 Self::Rules => "rules",
163 Self::AdvisoryDb => "advisory-db",
164 Self::SandboxRuntime => "sandbox-runtime",
165 }
166 }
167}
168
169/// One pinned asset.
170#[derive(Debug, Clone, Copy)]
171pub struct AssetSpec {
172 /// Stable id, used as the cache directory name and in every error.
173 pub id: &'static str,
174 /// The analyzer that needs it.
175 pub analyzer: &'static str,
176 /// What it is.
177 pub kind: AssetKind,
178 /// Where it comes from.
179 pub source: AssetSource,
180 /// The file name it is installed under, for a [`AssetSource::Vendored`]
181 /// asset. Empty for a directory asset.
182 pub file: &'static str,
183 /// Licence of the asset's contents, disclosed by `prefetch` before it
184 /// installs anything — the same disclosure `roteiro model pull` makes.
185 pub licence: &'static str,
186}
187
188/// Every asset this build knows how to provision.
189pub static ASSETS: &[AssetSpec] = &[
190 AssetSpec {
191 id: crate::adapter::semgrep::RULES_ASSET,
192 analyzer: crate::adapter::semgrep::ANALYZER,
193 kind: AssetKind::Rules,
194 source: AssetSource::Vendored(BASELINE_RULES),
195 file: "roteiro-baseline.yml",
196 // Written for this repository; see the rule file's own header for why no
197 // Semgrep Registry rule is vendored.
198 licence: "MIT OR Apache-2.0 (written for this repository)",
199 },
200 AssetSpec {
201 id: crate::adapter::cargo_audit::ADVISORY_DB_ASSET,
202 analyzer: crate::adapter::cargo_audit::ANALYZER,
203 kind: AssetKind::AdvisoryDb,
204 source: AssetSource::External {
205 hint: "git clone --depth 1 https://github.com/RustSec/advisory-db \
206 ~/.roteiro/security/rustsec-advisory-db/db",
207 },
208 file: "",
209 licence: "CC0-1.0 (RustSec advisory database)",
210 },
211 AssetSpec {
212 id: crate::adapter::osv_scanner::DB_ASSET,
213 analyzer: crate::adapter::osv_scanner::ANALYZER,
214 kind: AssetKind::AdvisoryDb,
215 source: AssetSource::Download {
216 files: OSV_DATABASES,
217 },
218 file: "",
219 // OSV.dev aggregates upstream databases and does not relicense them; each
220 // record carries its own terms. The two that dominate this set are named
221 // rather than flattened into one claim, because `cargo deny` governs
222 // crates and would never have looked at an advisory file.
223 licence: "per-record, as published by OSV.dev \
224 (CC0-1.0 for RustSec, CC-BY-4.0 for the GitHub Advisory Database)",
225 },
226 AssetSpec {
227 id: crate::runtime_pins::RUNTIME_ASSET,
228 // Not an analyzer's asset: every analyzer run under the sandboxed
229 // backend needs the same one. No adapter declares it, so `assets_for`
230 // never returns it and `prefetch --analyzer <name>` never selects it;
231 // it is provisioned by a plain `prefetch`, and resolved directly by id.
232 analyzer: SANDBOX,
233 kind: AssetKind::SandboxRuntime,
234 source: AssetSource::PinnedArchive {
235 archives: crate::runtime_pins::RUNTIME_ARCHIVES,
236 },
237 file: crate::runtime_pins::RUNTIME_FILE,
238 // The archive is a bundle of separately-licensed executables, and
239 // flattening them into one claim is exactly what let 25 MB of GPL
240 // binaries through a licence gate unnoticed. Each is named, and the
241 // full record — including the source-offer duty this creates — is in
242 // `crates/rto-exec/NOTICE-boxlite-runtime.md`, disclosed before install.
243 licence: "mixed: Apache-2.0 (boxlite-shim, boxlite-guest), \
244 GPL-2.0 (mke2fs, debugfs, libkrunfw), \
245 LGPL-2.0-or-later (bwrap) — see NOTICE-boxlite-runtime.md",
246 },
247];
248
249/// The `analyzer` field for an asset that belongs to no single analyzer.
250///
251/// A sentinel rather than an empty string, so `status` prints something a reader
252/// can act on and `--analyzer <name>` cannot accidentally match it.
253pub const SANDBOX: &str = "sandbox";
254
255/// The OSV per-ecosystem databases this build provisions.
256///
257/// The layout is not ours to choose: `osv-scanner --local-db-path <dir>` looks
258/// for `<dir>/osv-scalibr/<ECOSYSTEM>/all.zip`, with the ecosystem spelled
259/// exactly as OSV spells it (`crates.io`, not `cargo`; `PyPI`, not `pypi`).
260///
261/// Four ecosystems, because that is what ADR-0018's matrix asks of this
262/// analyzer: Python, Java and Node are the gap it closes, and `crates.io` is
263/// what makes the Rust cross-reference with `cargo-audit` possible at all.
264/// **`npm/all.zip` alone is roughly 210 MB**, and the four together are around
265/// 260 MB — a real provisioning cost, disclosed by `prefetch` before it fetches
266/// anything.
267pub static OSV_DATABASES: &[DownloadFile] = &[
268 DownloadFile {
269 path: "osv-scalibr/crates.io/all.zip",
270 url: "https://osv-vulnerabilities.storage.googleapis.com/crates.io/all.zip",
271 },
272 DownloadFile {
273 path: "osv-scalibr/PyPI/all.zip",
274 url: "https://osv-vulnerabilities.storage.googleapis.com/PyPI/all.zip",
275 },
276 DownloadFile {
277 path: "osv-scalibr/Maven/all.zip",
278 url: "https://osv-vulnerabilities.storage.googleapis.com/Maven/all.zip",
279 },
280 DownloadFile {
281 path: "osv-scalibr/npm/all.zip",
282 url: "https://osv-vulnerabilities.storage.googleapis.com/npm/all.zip",
283 },
284];
285
286/// The spec for `id`, or `None`.
287#[must_use]
288pub fn asset(id: &str) -> Option<&'static AssetSpec> {
289 ASSETS.iter().find(|a| a.id == id)
290}
291
292/// Every asset `analyzer` needs, in the order its adapter declares them.
293#[must_use]
294pub fn assets_for(analyzer: &str) -> Vec<&'static AssetSpec> {
295 adapter_for(analyzer)
296 .map(|adapter| {
297 adapter
298 .asset_ids()
299 .iter()
300 .filter_map(|id| asset(id))
301 .collect()
302 })
303 .unwrap_or_default()
304}
305
306/// What was recorded about an asset when it was provisioned.
307///
308/// Persisted beside the asset as `installed.json`, so `status` reports what was
309/// actually verified rather than re-deriving it and hoping the answer matches.
310#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
311pub struct InstalledAsset {
312 /// The asset id.
313 pub id: String,
314 /// What it is.
315 pub kind: AssetKind,
316 /// SHA-256 of the asset as installed. For a directory it is a digest over
317 /// the sorted `(relative path, content digest)` list, so it changes when any
318 /// file in the tree changes and does not depend on directory iteration
319 /// order.
320 pub digest: String,
321 /// When `prefetch` verified and recorded it, RFC 3339 UTC.
322 pub fetched_at: String,
323 /// How many files the digest covers, for a directory asset.
324 #[serde(default, skip_serializing_if = "Option::is_none")]
325 pub files: Option<usize>,
326 /// When the asset's contents were published, RFC 3339 UTC — for an advisory
327 /// database that is a git checkout, its `HEAD` commit time.
328 ///
329 /// This is **not** `fetched_at`. Fetching an eight-month-old database today
330 /// does not make it current, and the difference between the two is exactly
331 /// what a *possibly stale* label is about.
332 ///
333 /// It is recorded here because the analyzer will not report it: `cargo audit`
334 /// returns `last-commit: null` and `last-updated: null` whenever it is
335 /// pointed at a database with `--db` instead of resolving one itself —
336 /// verified against cargo-audit 0.22.2, at both a shallow clone and its own
337 /// managed checkout. Pinning the database is what makes a run reproducible,
338 /// so the pinned configuration must not be the one that loses the staleness
339 /// evidence.
340 #[serde(default, skip_serializing_if = "Option::is_none")]
341 pub published_at: Option<String>,
342}
343
344/// An asset's state, as `roteiro security status` reports it.
345#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
346pub struct AssetStatus {
347 /// The asset id.
348 pub id: &'static str,
349 /// The analyzer that needs it.
350 pub analyzer: &'static str,
351 /// What it is.
352 pub kind: AssetKind,
353 /// Where it is (or would be) on disk.
354 pub path: String,
355 /// What was recorded at provisioning time, if it has been provisioned.
356 #[serde(default, skip_serializing_if = "Option::is_none")]
357 pub installed: Option<InstalledAsset>,
358 /// Whole days since it was provisioned, when that can be computed.
359 #[serde(default, skip_serializing_if = "Option::is_none")]
360 pub age_days: Option<i64>,
361 /// Whether the bytes on disk still match the recorded digest. `None` when
362 /// nothing is installed.
363 #[serde(default, skip_serializing_if = "Option::is_none")]
364 pub verified: Option<bool>,
365}
366
367/// Resolve the root of the asset cache from its inputs, without touching the
368/// environment — so it is testable.
369fn root_from(
370 security_root: Option<PathBuf>,
371 roteiro_home: Option<PathBuf>,
372 home: Option<PathBuf>,
373) -> PathBuf {
374 if let Some(dir) = security_root {
375 return dir;
376 }
377 if let Some(dir) = roteiro_home {
378 return dir.join("security");
379 }
380 home.unwrap_or_else(|| PathBuf::from("."))
381 .join(".roteiro")
382 .join("security")
383}
384
385/// Root of the asset cache (`~/.roteiro/security`), honouring
386/// `ROTEIRO_SECURITY_ASSETS` and then `ROTEIRO_HOME`.
387///
388/// It sits beside the model store rather than inside the repository: assets are
389/// per-user, are shared across every checkout, and must never be committed.
390#[must_use]
391pub fn asset_root() -> PathBuf {
392 root_from(
393 std::env::var_os("ROTEIRO_SECURITY_ASSETS").map(PathBuf::from),
394 std::env::var_os("ROTEIRO_HOME").map(PathBuf::from),
395 std::env::var_os("HOME")
396 .or_else(|| std::env::var_os("USERPROFILE"))
397 .map(PathBuf::from),
398 )
399}
400
401/// Directory a given asset lives in.
402#[must_use]
403pub fn asset_dir(root: &Path, spec: &AssetSpec) -> PathBuf {
404 root.join(spec.id)
405}
406
407/// The path an analyzer is pointed at for this asset: the installed file for a
408/// vendored asset, the directory itself for an external one.
409#[must_use]
410pub fn asset_path(root: &Path, spec: &AssetSpec) -> PathBuf {
411 let dir = asset_dir(root, spec);
412 match spec.source {
413 AssetSource::Vendored(_) | AssetSource::PinnedArchive { .. } => dir.join(spec.file),
414 AssetSource::External { .. } | AssetSource::Download { .. } => dir.join("db"),
415 }
416}
417
418/// Where the provisioning record is kept.
419fn record_path(root: &Path, spec: &AssetSpec) -> PathBuf {
420 asset_dir(root, spec).join("installed.json")
421}
422
423/// Errors raised while provisioning.
424#[derive(Debug, thiserror::Error)]
425#[non_exhaustive]
426pub enum AssetError {
427 /// An [`AssetSource::External`] asset is not present, and Roteiro will not
428 /// fetch it. The message names the command that obtains it.
429 #[error(
430 "asset {id:?} is not provisioned: expected a directory at {path}\n \
431 obtain it with: {hint}\n \
432 then run: roteiro security prefetch --analyzer {analyzer}"
433 )]
434 ExternalMissing {
435 /// The asset id.
436 id: &'static str,
437 /// Where it was expected.
438 path: String,
439 /// The command that obtains it.
440 hint: &'static str,
441 /// The analyzer that needs it.
442 analyzer: &'static str,
443 },
444 /// This build has no such asset.
445 #[error("unknown asset {0:?}")]
446 Unknown(String),
447 /// A downloadable asset was asked for without a fetcher, which is what every
448 /// path except `roteiro security prefetch` does.
449 ///
450 /// This is the offline contract stated as an error rather than as a comment:
451 /// a run that finds a cold cache is told what to run, and is never quietly
452 /// given a network connection instead.
453 #[error(
454 "asset {id:?} is not provisioned and this code path does not download \
455 ({files} file(s), starting with {first})\n \
456 fetch it with: roteiro security prefetch --analyzer {analyzer}"
457 )]
458 FetchNotPermitted {
459 /// The asset id.
460 id: &'static str,
461 /// How many files it is made of.
462 files: usize,
463 /// The first URL, so the message names something concrete.
464 first: &'static str,
465 /// The analyzer that needs it.
466 analyzer: &'static str,
467 },
468 /// A download failed. The message is the fetcher's, because it knows what
469 /// went wrong and this module deliberately knows no transport.
470 #[error("downloading {url} for asset {id:?}: {message}")]
471 Fetch {
472 /// The asset id.
473 id: &'static str,
474 /// The URL that failed.
475 url: &'static str,
476 /// What the fetcher reported.
477 message: String,
478 },
479 /// A [`DownloadFile::path`] is not a plain relative path.
480 ///
481 /// Checked rather than trusted: these paths are compiled in today, but they
482 /// name where bytes from the network are written, and a `..` in one would
483 /// write outside the asset cache.
484 #[error("asset {id:?} declares an unsafe install path {path:?}")]
485 UnsafeInstallPath {
486 /// The asset id.
487 id: &'static str,
488 /// The offending path.
489 path: &'static str,
490 },
491 /// No sandbox runtime is pinned for this host platform.
492 ///
493 /// Refused by name rather than left to fail as a link error later: a
494 /// platform Roteiro has not pinned is a platform whose runtime bytes nobody
495 /// has verified, and building against unverified bytes is the thing this
496 /// whole path exists to prevent.
497 #[error(
498 "asset {id:?} has no pinned archive for this host ({os}/{arch}); \
499 pinned platforms are: {supported}"
500 )]
501 UnsupportedPlatform {
502 /// The asset id.
503 id: &'static str,
504 /// `std::env::consts::OS` for the host.
505 os: &'static str,
506 /// `std::env::consts::ARCH` for the host.
507 arch: &'static str,
508 /// The platforms that do have a pin, comma-separated.
509 supported: String,
510 },
511 /// A pinned archive is not provisioned, and this code path does not
512 /// download.
513 #[error(
514 "asset {id:?} ({target}) is not provisioned and this code path does not download\n \
515 expected at: {path}\n \
516 fetch it with: roteiro security prefetch --allow-download"
517 )]
518 ArchiveMissing {
519 /// The asset id.
520 id: &'static str,
521 /// The host platform it would be fetched for.
522 target: &'static str,
523 /// Where it was expected.
524 path: String,
525 },
526 /// A pinned archive's bytes are not the bytes that were pinned.
527 ///
528 /// This is the check that makes the sandbox runtime reproducible, so it is a
529 /// hard failure with no override: a mismatch is either a truncated download,
530 /// a redirected URL, or a substituted artifact, and none of those is
531 /// something to carry on from. The size is reported alongside because a
532 /// short body is the common case and two unequal digests do not say so.
533 #[error(
534 "asset {id:?} does not match its pinned digest — refusing it\n \
535 from: {url}\n \
536 expected: {expected} ({expected_bytes} bytes)\n \
537 actual: {actual} ({actual_bytes} bytes)"
538 )]
539 DigestMismatch {
540 /// The asset id.
541 id: &'static str,
542 /// Where the bytes came from.
543 url: String,
544 /// The digest that was pinned.
545 expected: &'static str,
546 /// The size that was pinned.
547 expected_bytes: u64,
548 /// The digest of what arrived.
549 actual: String,
550 /// The size of what arrived.
551 actual_bytes: u64,
552 },
553 /// Reading or writing the cache failed.
554 #[error("asset cache I/O at {path}: {source}")]
555 Io {
556 /// What was being touched.
557 path: String,
558 /// The underlying failure.
559 source: std::io::Error,
560 },
561 /// The provisioning record could not be read or written.
562 #[error("asset record: {0}")]
563 Record(#[from] serde_json::Error),
564}
565
566/// How bytes at a URL are written to a local path.
567///
568/// The transport is the caller's: this crate has no HTTP dependency and is not
569/// going to acquire one for a single asset kind. `roteiro security prefetch`
570/// supplies an implementation over the `ureq` client already in the tree; tests
571/// supply one that writes fixture bytes and never opens a socket, which is how
572/// the download path is exercised without a network.
573///
574/// # The contract, and why it cannot be checked here
575///
576/// **An implementation must write the whole file or fail.** A truncated download
577/// that returned `Ok` would be renamed into place, digested, and recorded as the
578/// asset's pin — and [`AssetSource::Download`] has no compile-time digest to
579/// contradict it, so `status` would then report the short file as present and
580/// matching. Staging through a `.partial` file guards against a crash, not
581/// against a fetcher that misreports success.
582///
583/// Nothing in this crate can verify that: completeness is a property of the
584/// transport's framing, and this crate deliberately has no transport. The
585/// shipped implementation is `download_asset_file` in the CLI, which establishes
586/// it from the response's declared length and refuses a body whose length cannot
587/// be established at all.
588pub type Fetcher<'a> = dyn Fn(&str, &Path) -> Result<(), String> + 'a;
589
590/// Install and verify one asset, without any ability to download.
591///
592/// This is what every path except `roteiro security prefetch` calls. A
593/// [`AssetSource::Download`] asset that is not already present therefore fails
594/// with [`AssetError::FetchNotPermitted`] naming the prefetch command, which is
595/// the offline contract expressed as a signature.
596///
597/// It is idempotent: re-running it re-digests and re-stamps, which is what makes
598/// `prefetch` a safe thing to run whenever you are unsure.
599///
600/// # Errors
601/// Returns [`AssetError::ExternalMissing`] when an operator-provisioned asset is
602/// absent, [`AssetError::FetchNotPermitted`] when a downloadable one is, or
603/// [`AssetError::Io`] if the cache cannot be written.
604pub fn provision(root: &Path, spec: &AssetSpec) -> Result<InstalledAsset, AssetError> {
605 provision_with(root, spec, None)
606}
607
608/// Install and verify one asset, downloading through `fetch` where the asset
609/// needs it.
610///
611/// This is the **only** function that writes to the asset cache, and the only
612/// one that can cause a network request. `fetch` is `None` for every caller that
613/// must not fetch; see [`provision`].
614///
615/// # Errors
616/// As [`provision`], plus [`AssetError::Fetch`] if a download fails and
617/// [`AssetError::UnsafeInstallPath`] if a declared install path could escape the
618/// asset directory.
619pub fn provision_with(
620 root: &Path,
621 spec: &AssetSpec,
622 fetch: Option<&Fetcher<'_>>,
623) -> Result<InstalledAsset, AssetError> {
624 let dir = asset_dir(root, spec);
625 std::fs::create_dir_all(&dir).map_err(|source| AssetError::Io {
626 path: dir.display().to_string(),
627 source,
628 })?;
629 let target = asset_path(root, spec);
630
631 let (digest, files) = match spec.source {
632 AssetSource::Vendored(bytes) => {
633 write_atomically(&target, bytes)?;
634 (sha256_hex(bytes), None)
635 }
636 AssetSource::External { hint } => {
637 if !target.is_dir() {
638 return Err(AssetError::ExternalMissing {
639 id: spec.id,
640 path: target.display().to_string(),
641 hint,
642 analyzer: spec.analyzer,
643 });
644 }
645 let (digest, count) = digest_tree(&target)?;
646 (digest, Some(count))
647 }
648 AssetSource::Download { files } => {
649 download_all(spec, files, &target, fetch)?;
650 let (digest, count) = digest_tree(&target)?;
651 (digest, Some(count))
652 }
653 AssetSource::PinnedArchive { archives } => {
654 let digest = provision_archive(spec, archives, &target, fetch)?;
655 (digest, None)
656 }
657 };
658 let published_at = published_at(&target);
659
660 let record = InstalledAsset {
661 id: spec.id.to_owned(),
662 kind: spec.kind,
663 digest,
664 fetched_at: rfc3339_utc(std::time::SystemTime::now()),
665 files,
666 published_at,
667 };
668 let json = serde_json::to_vec_pretty(&record)?;
669 write_atomically(&record_path(root, spec), &json)?;
670 Ok(record)
671}
672
673/// Fetch every file of a [`AssetSource::Download`] asset into `target`.
674///
675/// With no fetcher this refuses unless the files are *already* all there, which
676/// is what makes `provision` idempotent for a downloadable asset without giving
677/// it a network: a second `prefetch --offline`-style call over a warm cache
678/// re-digests and re-stamps rather than failing.
679fn download_all(
680 spec: &AssetSpec,
681 files: &'static [DownloadFile],
682 target: &Path,
683 fetch: Option<&Fetcher<'_>>,
684) -> Result<(), AssetError> {
685 for file in files {
686 if !is_safe_relative(file.path) {
687 return Err(AssetError::UnsafeInstallPath {
688 id: spec.id,
689 path: file.path,
690 });
691 }
692 }
693
694 let missing: Vec<&DownloadFile> = files
695 .iter()
696 .filter(|file| !target.join(file.path).is_file())
697 .collect();
698 if missing.is_empty() {
699 return Ok(());
700 }
701 let Some(fetch) = fetch else {
702 return Err(AssetError::FetchNotPermitted {
703 id: spec.id,
704 files: missing.len(),
705 first: missing[0].url,
706 analyzer: spec.analyzer,
707 });
708 };
709
710 for file in missing {
711 let destination = target.join(file.path);
712 if let Some(parent) = destination.parent() {
713 std::fs::create_dir_all(parent).map_err(|source| AssetError::Io {
714 path: parent.display().to_string(),
715 source,
716 })?;
717 }
718 // Fetch beside the destination and rename, so an interrupted download
719 // never leaves a half-file at the path the analyzer reads.
720 //
721 // Staging protects the pin from a *crash*; it cannot protect it from a
722 // fetcher that returns `Ok` over a short body, because then the rename
723 // happens and the truncated file is what gets digested. That half of the
724 // contract is the fetcher's, and is stated on [`Fetcher`].
725 let partial = destination.with_extension("partial");
726 std::fs::remove_file(&partial).ok();
727 fetch(file.url, &partial).map_err(|message| {
728 // Leave nothing behind. The stray file is not at a path any analyzer
729 // reads, but `digest_tree` covers the whole asset directory — so a
730 // later successful provision (of the remaining files, or after the
731 // operator placed this one by hand) would fold these bytes into the
732 // recorded pin, and removing them afterwards would then read as
733 // tampering.
734 std::fs::remove_file(&partial).ok();
735 AssetError::Fetch {
736 id: spec.id,
737 url: file.url,
738 message,
739 }
740 })?;
741 std::fs::rename(&partial, &destination).map_err(|source| {
742 std::fs::remove_file(&partial).ok();
743 AssetError::Io {
744 path: destination.display().to_string(),
745 source,
746 }
747 })?;
748 }
749 Ok(())
750}
751
752/// The pinned archive for the host this is running on.
753///
754/// # Errors
755/// Returns [`AssetError::UnsupportedPlatform`] naming the platforms that are
756/// pinned, for a host that is not one of them.
757pub fn archive_for_host(
758 spec: &AssetSpec,
759 archives: &'static [crate::runtime_pins::PinnedArchive],
760) -> Result<&'static crate::runtime_pins::PinnedArchive, AssetError> {
761 // Searched in the slice the *spec* carries, not in the global table. They
762 // are the same slice in production, and keeping the lookup parameterised is
763 // what lets the pin be exercised without shipping a fake into the real one.
764 crate::runtime_pins::runtime_target(std::env::consts::OS, std::env::consts::ARCH)
765 .and_then(|target| archives.iter().find(|a| a.target == target))
766 .ok_or_else(|| AssetError::UnsupportedPlatform {
767 id: spec.id,
768 os: std::env::consts::OS,
769 arch: std::env::consts::ARCH,
770 supported: archives
771 .iter()
772 .map(|a| a.target)
773 .collect::<Vec<_>>()
774 .join(", "),
775 })
776}
777
778/// Install the host's pinned archive, verifying its digest before it counts.
779///
780/// Idempotent and offline over a warm cache: an archive already present *and
781/// matching its pin* is accepted without a fetcher, which is what lets a machine
782/// with no network re-run `prefetch` and get a clean bill rather than a refusal.
783/// An archive present but **not** matching is refused rather than re-fetched —
784/// silently replacing bytes that failed verification would turn a tamper signal
785/// into a retry.
786fn provision_archive(
787 spec: &AssetSpec,
788 archives: &'static [crate::runtime_pins::PinnedArchive],
789 target: &Path,
790 fetch: Option<&Fetcher<'_>>,
791) -> Result<String, AssetError> {
792 let archive = archive_for_host(spec, archives)?;
793
794 if target.is_file() {
795 // Present already: verify, and take it or refuse it. Either way no
796 // network is touched, which is the whole point of a warm cache.
797 return verify_archive(spec, archive, target, &target.display().to_string());
798 }
799
800 let Some(fetch) = fetch else {
801 return Err(AssetError::ArchiveMissing {
802 id: spec.id,
803 target: archive.target,
804 path: target.display().to_string(),
805 });
806 };
807
808 if let Some(parent) = target.parent() {
809 std::fs::create_dir_all(parent).map_err(|source| AssetError::Io {
810 path: parent.display().to_string(),
811 source,
812 })?;
813 }
814
815 // Stage beside the destination, verify, and only then rename. A body that
816 // fails its pin never appears at the path anything reads — so a failed
817 // provision leaves a cold cache rather than a poisoned one.
818 let partial = target.with_extension("partial");
819 std::fs::remove_file(&partial).ok();
820 fetch(archive.url, &partial).map_err(|message| {
821 std::fs::remove_file(&partial).ok();
822 AssetError::Fetch {
823 id: spec.id,
824 url: archive.url,
825 message,
826 }
827 })?;
828
829 let digest = match verify_archive(spec, archive, &partial, archive.url) {
830 Ok(digest) => digest,
831 Err(e) => {
832 std::fs::remove_file(&partial).ok();
833 return Err(e);
834 }
835 };
836
837 std::fs::rename(&partial, target).map_err(|source| {
838 std::fs::remove_file(&partial).ok();
839 AssetError::Io {
840 path: target.display().to_string(),
841 source,
842 }
843 })?;
844 Ok(digest)
845}
846
847/// Check a file against a pinned archive, returning its digest when it matches.
848///
849/// `origin` is what the failure message blames — a URL when the bytes just
850/// arrived from one, a path when they were already on disk.
851///
852/// # Errors
853/// Returns [`AssetError::DigestMismatch`] when the bytes are not the pinned
854/// bytes, or [`AssetError::Io`] when the file cannot be read.
855pub fn verify_archive(
856 spec: &AssetSpec,
857 archive: &crate::runtime_pins::PinnedArchive,
858 path: &Path,
859 origin: &str,
860) -> Result<String, AssetError> {
861 let bytes = std::fs::read(path).map_err(|source| AssetError::Io {
862 path: path.display().to_string(),
863 source,
864 })?;
865 let digest = sha256_hex(&bytes);
866 let actual_bytes = bytes.len() as u64;
867 if digest != archive.sha256 || actual_bytes != archive.bytes {
868 return Err(AssetError::DigestMismatch {
869 id: spec.id,
870 url: origin.to_owned(),
871 expected: archive.sha256,
872 expected_bytes: archive.bytes,
873 actual: digest,
874 actual_bytes,
875 });
876 }
877 Ok(digest)
878}
879
880/// Whether a declared install path stays inside the asset directory.
881///
882/// Compiled-in paths today, but they name where bytes from the network land, and
883/// the check costs nothing.
884fn is_safe_relative(path: &str) -> bool {
885 !path.is_empty()
886 && !Path::new(path).components().any(|component| {
887 matches!(
888 component,
889 std::path::Component::RootDir
890 | std::path::Component::Prefix(_)
891 | std::path::Component::ParentDir
892 )
893 })
894}
895
896/// The provisioning record for an asset, or `None` if it was never provisioned
897/// or the record is unreadable.
898///
899/// An unreadable record is treated as absent rather than as an error: the
900/// remedy is the same — run `prefetch` — and a corrupt cache file should not
901/// make `status` fail.
902#[must_use]
903pub fn installed(root: &Path, spec: &AssetSpec) -> Option<InstalledAsset> {
904 let bytes = std::fs::read(record_path(root, spec)).ok()?;
905 serde_json::from_slice(&bytes).ok()
906}
907
908/// The state of every asset this build knows about, for `roteiro security
909/// status`.
910#[must_use]
911pub fn status(root: &Path, analyzer: Option<&str>) -> Vec<AssetStatus> {
912 ASSETS
913 .iter()
914 .filter(|spec| analyzer.is_none_or(|name| spec.analyzer == name))
915 .map(|spec| {
916 let installed = installed(root, spec);
917 let now = rfc3339_utc(std::time::SystemTime::now());
918 let age_days = installed
919 .as_ref()
920 .and_then(|record| age_in_days(&record.fetched_at, &now));
921 // Re-digest what is on disk. A record that no longer matches the
922 // bytes is exactly the case a status command exists to surface, and
923 // reporting the record alone would hide it.
924 let verified = installed.as_ref().map(|record| {
925 current_digest(root, spec).as_deref() == Some(record.digest.as_str())
926 });
927 AssetStatus {
928 id: spec.id,
929 analyzer: spec.analyzer,
930 kind: spec.kind,
931 path: asset_path(root, spec).display().to_string(),
932 installed,
933 age_days,
934 verified,
935 }
936 })
937 .collect()
938}
939
940/// When the contents at `dir` were published, if that can be established.
941///
942/// A git checkout's `HEAD` commit time is the publication date. Anything that is
943/// not a git checkout has no such date, and `None` is reported rather than
944/// invented — a made-up publication date would make a stale database look fresh,
945/// which is the one failure mode this whole field exists to prevent.
946fn published_at(dir: &Path) -> Option<String> {
947 let repo = rto_graph::Repo::discover(dir).ok()?;
948 // `discover` walks upwards, so a directory that is merely *inside* a
949 // repository would otherwise be dated by that repository's HEAD.
950 if repo.workdir()? != dir {
951 return None;
952 }
953 let seconds = repo.head_commit_time().ok()?;
954 Some(rfc3339_utc(
955 std::time::UNIX_EPOCH + std::time::Duration::from_secs(u64::try_from(seconds).ok()?),
956 ))
957}
958
959/// The advisory-database evidence recorded for `analyzer` at provisioning time.
960///
961/// Supplied to a run so its results carry a database identity and publication
962/// date even though the analyzer itself reports neither.
963#[must_use]
964pub fn advisory_db_evidence(root: &Path, analyzer: &str) -> Option<rto_graph::AdvisoryDb> {
965 let spec = assets_for(analyzer)
966 .into_iter()
967 .find(|s| s.kind == AssetKind::AdvisoryDb)?;
968 let record = installed(root, spec)?;
969 Some(rto_graph::AdvisoryDb {
970 digest: record.digest,
971 published_at: record.published_at,
972 })
973}
974
975/// Digest of what is on disk right now, or `None` if it is not there.
976fn current_digest(root: &Path, spec: &AssetSpec) -> Option<String> {
977 let target = asset_path(root, spec);
978 match spec.source {
979 AssetSource::Vendored(_) | AssetSource::PinnedArchive { .. } => {
980 Some(sha256_hex(&std::fs::read(target).ok()?))
981 }
982 AssetSource::External { .. } | AssetSource::Download { .. } => {
983 digest_tree(&target).ok().map(|(digest, _)| digest)
984 }
985 }
986}
987
988/// Resolve every asset `analyzer` needs to a verified local path.
989///
990/// # Errors
991/// Returns [`ExecError::AssetsUnavailableOffline`] naming every asset that is
992/// missing or whose bytes no longer match what was recorded, together with the
993/// exact prefetch command. It never fetches, and it never falls back to a
994/// host-installed copy.
995pub fn resolve(root: &Path, analyzer: &str) -> Result<Vec<(&'static str, PathBuf)>, ExecError> {
996 let specs = assets_for(analyzer);
997 let mut resolved = Vec::with_capacity(specs.len());
998 let mut missing = Vec::new();
999
1000 for spec in specs {
1001 let path = asset_path(root, spec);
1002 match (installed(root, spec), current_digest(root, spec)) {
1003 // Provisioned, and the bytes still match what was recorded.
1004 (Some(record), Some(digest)) if digest == record.digest => {
1005 resolved.push((spec.id, path));
1006 }
1007 // Provisioned, but the bytes changed underneath the record. That is
1008 // not a warning: a run would stamp a digest that does not describe
1009 // what it read.
1010 (Some(record), Some(_)) => missing.push(MissingAsset {
1011 id: spec.id.to_owned(),
1012 digest: record.digest.clone(),
1013 reason: "the bytes on disk no longer match the recorded digest",
1014 }),
1015 (Some(record), None) => missing.push(MissingAsset {
1016 id: spec.id.to_owned(),
1017 digest: record.digest.clone(),
1018 reason: "recorded as provisioned, but nothing is there now",
1019 }),
1020 (None, _) => missing.push(MissingAsset {
1021 id: spec.id.to_owned(),
1022 digest: "not yet pinned".to_owned(),
1023 reason: "never provisioned",
1024 }),
1025 }
1026 }
1027
1028 if missing.is_empty() {
1029 Ok(resolved)
1030 } else {
1031 Err(ExecError::AssetsUnavailableOffline {
1032 analyzer: analyzer.to_owned(),
1033 missing,
1034 command: format!("roteiro security prefetch --analyzer {analyzer}"),
1035 })
1036 }
1037}
1038
1039/// One asset a run needed and did not have.
1040#[derive(Debug, Clone, PartialEq, Eq, Serialize)]
1041pub struct MissingAsset {
1042 /// The asset id.
1043 pub id: String,
1044 /// The digest that was pinned for it, or a note that none is.
1045 pub digest: String,
1046 /// Why it could not be used.
1047 pub reason: &'static str,
1048}
1049
1050impl std::fmt::Display for MissingAsset {
1051 fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
1052 write!(f, "{} ({}; {})", self.id, self.digest, self.reason)
1053 }
1054}
1055
1056/// Digest of a directory tree, plus the number of files it covered.
1057///
1058/// The digest is over the sorted list of `(relative path, sha256(bytes))`, so it
1059/// is a function of the tree's contents alone: independent of directory
1060/// iteration order, of timestamps, and of where the tree happens to be mounted.
1061/// `.git` is skipped — it is bookkeeping, not advisory data, and including it
1062/// would make the digest churn on every fetch that changed nothing.
1063fn digest_tree(dir: &Path) -> Result<(String, usize), AssetError> {
1064 let mut entries: BTreeMap<String, String> = BTreeMap::new();
1065 walk(dir, dir, &mut entries)?;
1066 let mut manifest = String::new();
1067 for (path, digest) in &entries {
1068 use std::fmt::Write as _;
1069 let _ = writeln!(manifest, "{digest} {path}");
1070 }
1071 Ok((sha256_hex(manifest.as_bytes()), entries.len()))
1072}
1073
1074fn walk(root: &Path, dir: &Path, into: &mut BTreeMap<String, String>) -> Result<(), AssetError> {
1075 let read = std::fs::read_dir(dir).map_err(|source| AssetError::Io {
1076 path: dir.display().to_string(),
1077 source,
1078 })?;
1079 for entry in read {
1080 let entry = entry.map_err(|source| AssetError::Io {
1081 path: dir.display().to_string(),
1082 source,
1083 })?;
1084 let path = entry.path();
1085 // `symlink_metadata` rather than `metadata`: a symlink out of the tree
1086 // must not be followed into a file the digest has no business reading.
1087 let meta = std::fs::symlink_metadata(&path).map_err(|source| AssetError::Io {
1088 path: path.display().to_string(),
1089 source,
1090 })?;
1091 if meta.is_symlink() {
1092 continue;
1093 }
1094 if meta.is_dir() {
1095 if path.file_name().is_some_and(|n| n == ".git") {
1096 continue;
1097 }
1098 walk(root, &path, into)?;
1099 } else if meta.is_file() {
1100 let bytes = std::fs::read(&path).map_err(|source| AssetError::Io {
1101 path: path.display().to_string(),
1102 source,
1103 })?;
1104 let relative = path
1105 .strip_prefix(root)
1106 .unwrap_or(&path)
1107 .to_string_lossy()
1108 .replace('\\', "/");
1109 into.insert(relative, sha256_hex(&bytes));
1110 }
1111 }
1112 Ok(())
1113}
1114
1115/// Write `bytes` to `path` via a temp file and a rename, so a reader never sees
1116/// a half-written asset — the same discipline `rto_graph::download_verified`
1117/// applies to a model file.
1118fn write_atomically(path: &Path, bytes: &[u8]) -> Result<(), AssetError> {
1119 let io = |source| AssetError::Io {
1120 path: path.display().to_string(),
1121 source,
1122 };
1123 let tmp = path.with_extension("partial");
1124 std::fs::write(&tmp, bytes).map_err(io)?;
1125 if path.exists() {
1126 std::fs::remove_file(path).map_err(io)?;
1127 }
1128 std::fs::rename(&tmp, path).map_err(|source| {
1129 std::fs::remove_file(&tmp).ok();
1130 AssetError::Io {
1131 path: path.display().to_string(),
1132 source,
1133 }
1134 })
1135}
1136
1137#[cfg(test)]
1138mod tests {
1139 use super::{
1140 ASSETS, AssetError, AssetKind, AssetSource, asset, asset_path, assets_for, installed,
1141 provision, resolve, root_from, status,
1142 };
1143 use crate::runner::ExecError;
1144 use std::path::PathBuf;
1145
1146 /// A throwaway cache root that removes itself.
1147 struct Cache(PathBuf);
1148
1149 impl Cache {
1150 fn new(name: &str) -> Self {
1151 let dir = std::env::temp_dir().join(format!("rto-exec-assets-{name}"));
1152 std::fs::remove_dir_all(&dir).ok();
1153 std::fs::create_dir_all(&dir).expect("create");
1154 Self(dir)
1155 }
1156 }
1157
1158 impl Drop for Cache {
1159 fn drop(&mut self) {
1160 std::fs::remove_dir_all(&self.0).ok();
1161 }
1162 }
1163
1164 fn rules() -> &'static super::AssetSpec {
1165 asset("semgrep-rules").expect("the baseline rule set is a known asset")
1166 }
1167
1168 fn advisory_db() -> &'static super::AssetSpec {
1169 asset("rustsec-advisory-db").expect("the advisory database is a known asset")
1170 }
1171
1172 /// Every asset is reachable from something that wants it — either an
1173 /// analyzer's adapter, or the shared sandbox, which no single analyzer owns.
1174 ///
1175 /// The `SANDBOX` arm is not a loophole: an asset that claims to belong to an
1176 /// analyzer and is not in that analyzer's `asset_ids` would be provisioned
1177 /// and never used, which is the case this test exists to catch.
1178 #[test]
1179 fn every_asset_belongs_to_an_analyzer_that_asked_for_it() {
1180 for spec in ASSETS {
1181 if spec.analyzer == super::SANDBOX {
1182 assert!(
1183 assets_for(spec.analyzer).is_empty(),
1184 "{} uses the shared-asset sentinel, so no adapter may claim it",
1185 spec.id
1186 );
1187 } else {
1188 assert!(
1189 assets_for(spec.analyzer).iter().any(|s| s.id == spec.id),
1190 "{} is not claimed by {}",
1191 spec.id,
1192 spec.analyzer
1193 );
1194 }
1195 assert!(!spec.licence.is_empty(), "{} discloses no licence", spec.id);
1196 }
1197 }
1198
1199 /// The sandbox runtime's disclosure must name every licence family in the
1200 /// archive, not flatten them into one word.
1201 ///
1202 /// Flattening is precisely how 25 MB of GPL binaries travelled through a
1203 /// licence gate that reported `licenses ok`. A reader of `prefetch`'s output
1204 /// is entitled to see what they are about to install.
1205 #[test]
1206 fn the_sandbox_runtime_discloses_every_licence_it_carries() {
1207 let spec = asset(crate::runtime_pins::RUNTIME_ASSET).expect("the runtime is a known asset");
1208 assert_eq!(spec.kind, AssetKind::SandboxRuntime);
1209 for family in ["Apache-2.0", "GPL-2.0", "LGPL-2.0"] {
1210 assert!(
1211 spec.licence.contains(family),
1212 "the disclosure does not mention {family}: {}",
1213 spec.licence
1214 );
1215 }
1216 assert!(
1217 spec.licence.contains("NOTICE-boxlite-runtime.md"),
1218 "the disclosure must point at the full record: {}",
1219 spec.licence
1220 );
1221 }
1222
1223 /// Every pinned archive must carry a full digest and a real size, and the
1224 /// set must cover exactly the platforms `runtime_target` claims — a target
1225 /// that maps to no archive would fail at build time with nothing to say.
1226 #[test]
1227 fn every_pinned_archive_is_complete_and_reachable() {
1228 use crate::runtime_pins::{RUNTIME_ARCHIVES, archive_for, runtime_target};
1229 assert!(!RUNTIME_ARCHIVES.is_empty());
1230 for archive in RUNTIME_ARCHIVES {
1231 assert_eq!(
1232 archive.sha256.len(),
1233 64,
1234 "{} has no full sha256",
1235 archive.target
1236 );
1237 assert!(
1238 archive
1239 .sha256
1240 .chars()
1241 .all(|c| c.is_ascii_hexdigit() && !c.is_uppercase()),
1242 "{} digest must be lowercase hex",
1243 archive.target
1244 );
1245 assert!(
1246 archive.bytes > 1_000_000,
1247 "{} size looks wrong",
1248 archive.target
1249 );
1250 assert!(
1251 archive.url.ends_with(".tar.gz") && archive.url.contains(archive.target),
1252 "{} url does not name the target it is for: {}",
1253 archive.target,
1254 archive.url
1255 );
1256 }
1257 for (os, arch) in [
1258 ("macos", "aarch64"),
1259 ("linux", "x86_64"),
1260 ("linux", "aarch64"),
1261 ] {
1262 let target = runtime_target(os, arch).expect("a pinned platform");
1263 let archive = archive_for(os, arch).expect("must resolve to an archive");
1264 assert_eq!(archive.target, target);
1265 }
1266 assert!(runtime_target("windows", "x86_64").is_none());
1267 assert!(archive_for("windows", "x86_64").is_none());
1268 }
1269
1270 /// A digest that does not match is refused, and the refusal says which
1271 /// bytes were expected — including the size, because a truncated body is
1272 /// the common failure and two unequal digests do not say so.
1273 #[test]
1274 fn a_pinned_archive_that_does_not_match_is_refused() {
1275 use crate::runtime_pins::PinnedArchive;
1276 let cache = Cache::new("pinned-mismatch");
1277 let spec = asset(crate::runtime_pins::RUNTIME_ASSET).expect("known asset");
1278 let archive = PinnedArchive {
1279 target: "test-target",
1280 url: "https://example.invalid/runtime.tar.gz",
1281 sha256: "0000000000000000000000000000000000000000000000000000000000000000",
1282 bytes: 999,
1283 };
1284 let path = cache.0.join("impostor.tar.gz");
1285 std::fs::write(&path, b"not the pinned bytes").expect("write");
1286
1287 let err = super::verify_archive(spec, &archive, &path, archive.url)
1288 .expect_err("bytes that do not match the pin must be refused");
1289 let message = err.to_string();
1290 assert!(matches!(err, AssetError::DigestMismatch { .. }));
1291 assert!(message.contains(archive.sha256), "{message}");
1292 assert!(message.contains("999 bytes"), "{message}");
1293 assert!(message.contains("20 bytes"), "{message}");
1294 }
1295
1296 /// Provisioning a pinned archive without a fetcher is refused by name, and
1297 /// names the command that fixes it — the same offline contract every other
1298 /// asset kind follows.
1299 #[test]
1300 fn a_pinned_archive_is_not_fetched_by_a_path_that_may_not_download() {
1301 let cache = Cache::new("pinned-cold");
1302 let spec = asset(crate::runtime_pins::RUNTIME_ASSET).expect("known asset");
1303 let err = provision(&cache.0, spec).expect_err("a cold cache must refuse");
1304 // A host with no pinned archive fails earlier, and differently — both
1305 // are correct refusals, and asserting the property rather than one
1306 // literal keeps this test honest on an unpinned platform.
1307 let message = err.to_string();
1308 match err {
1309 AssetError::ArchiveMissing { .. } => {
1310 assert!(message.contains("prefetch --allow-download"), "{message}");
1311 }
1312 AssetError::UnsupportedPlatform { .. } => {
1313 assert!(message.contains("pinned platforms are"), "{message}");
1314 }
1315 other => panic!("unexpected refusal: {other}"),
1316 }
1317 }
1318
1319 /// A spec pinned to `body`, for the host platform, without touching the
1320 /// shipped pins.
1321 ///
1322 /// Leaked because [`AssetSource::PinnedArchive`] holds `&'static` data — a
1323 /// few bytes per test process, and the alternative is either a fake entry in
1324 /// the real table or not exercising the pin at all.
1325 fn pinned_to(body: &[u8]) -> Option<&'static super::AssetSpec> {
1326 let target =
1327 crate::runtime_pins::runtime_target(std::env::consts::OS, std::env::consts::ARCH)?;
1328 let archives: &'static [crate::runtime_pins::PinnedArchive] =
1329 Box::leak(Box::new([crate::runtime_pins::PinnedArchive {
1330 target,
1331 url: "https://example.invalid/runtime.tar.gz",
1332 sha256: Box::leak(crate::sha256_hex(body).into_boxed_str()),
1333 bytes: body.len() as u64,
1334 }]));
1335 Some(Box::leak(Box::new(super::AssetSpec {
1336 id: "test-pinned-archive",
1337 analyzer: super::SANDBOX,
1338 kind: AssetKind::SandboxRuntime,
1339 source: AssetSource::PinnedArchive { archives },
1340 file: "fixture.tar.gz",
1341 licence: "test fixture",
1342 })))
1343 }
1344
1345 /// A warm cache provisions with **no fetcher at all**, and is still
1346 /// verified.
1347 ///
1348 /// This is what makes "no network, warm cache" a real claim rather than an
1349 /// aspiration — and the first half is the one that matters most: an archive
1350 /// already on disk whose bytes do not match the pin is *refused*, so a warm
1351 /// cache can never become a way around the pin.
1352 #[test]
1353 fn a_warm_pinned_archive_provisions_offline_and_is_still_verified() {
1354 let body = b"pretend this is a runtime archive".to_vec();
1355 let Some(spec) = pinned_to(&body) else {
1356 eprintln!(
1357 "SKIPPED: no sandbox runtime is pinned for {}/{}",
1358 std::env::consts::OS,
1359 std::env::consts::ARCH
1360 );
1361 return;
1362 };
1363 let cache = Cache::new("pinned-warm");
1364 let target = asset_path(&cache.0, spec);
1365 std::fs::create_dir_all(target.parent().expect("parent")).expect("mkdir");
1366
1367 // Right pin, wrong bytes: refused, without a fetcher ever being offered.
1368 std::fs::write(&target, b"tampered").expect("write");
1369 let err = provision(&cache.0, spec).expect_err("a warm cache is still verified");
1370 assert!(matches!(err, AssetError::DigestMismatch { .. }), "{err}");
1371
1372 // The pinned bytes: provisions offline, with no fetcher at all.
1373 std::fs::write(&target, &body).expect("write");
1374 let record = provision(&cache.0, spec).expect("a matching warm cache provisions offline");
1375 assert_eq!(record.kind, AssetKind::SandboxRuntime);
1376 assert_eq!(record.digest, crate::sha256_hex(&body));
1377
1378 // And `resolve`-style re-verification agrees the bytes are still right.
1379 assert_eq!(
1380 super::current_digest(&cache.0, spec).as_deref(),
1381 Some(record.digest.as_str())
1382 );
1383 }
1384
1385 /// A fetcher that returns success over the wrong bytes cannot poison the
1386 /// cache: the archive is verified *before* it is renamed into place, so a
1387 /// failed provision leaves a cold cache rather than a bad one.
1388 ///
1389 /// This is the case [`Fetcher`]'s contract cannot cover for a `Download`
1390 /// asset, and the reason `PinnedArchive` exists.
1391 #[test]
1392 fn a_lying_fetcher_cannot_install_a_pinned_archive() {
1393 let body = b"the real runtime archive".to_vec();
1394 let Some(spec) = pinned_to(&body) else {
1395 eprintln!("SKIPPED: no sandbox runtime is pinned for this platform");
1396 return;
1397 };
1398 let cache = Cache::new("pinned-lying-fetcher");
1399
1400 let liar: &super::Fetcher<'_> = &|_url: &str, dest: &std::path::Path| {
1401 std::fs::write(dest, b"truncated").map_err(|e| e.to_string())
1402 };
1403 let err = super::provision_with(&cache.0, spec, Some(liar))
1404 .expect_err("bytes that do not match the pin must be refused");
1405 assert!(matches!(err, AssetError::DigestMismatch { .. }), "{err}");
1406
1407 // Nothing was left behind at the path anything reads, and no staging
1408 // file survived to be folded into a later digest.
1409 let target = asset_path(&cache.0, spec);
1410 assert!(!target.exists(), "a refused archive must not be installed");
1411 assert!(
1412 !target.with_extension("partial").exists(),
1413 "staging file left behind"
1414 );
1415
1416 // An honest fetcher then provisions normally.
1417 let honest: &super::Fetcher<'_> = &|_url: &str, dest: &std::path::Path| {
1418 std::fs::write(dest, b"the real runtime archive").map_err(|e| e.to_string())
1419 };
1420 let record = super::provision_with(&cache.0, spec, Some(honest)).expect("provision");
1421 assert_eq!(record.digest, crate::sha256_hex(&body));
1422 }
1423
1424 #[test]
1425 fn the_cache_root_prefers_the_explicit_override_then_roteiro_home() {
1426 assert_eq!(
1427 root_from(
1428 Some("/explicit".into()),
1429 Some("/home/.roteiro".into()),
1430 None
1431 ),
1432 PathBuf::from("/explicit")
1433 );
1434 assert_eq!(
1435 root_from(None, Some("/home/.roteiro".into()), None),
1436 PathBuf::from("/home/.roteiro/security")
1437 );
1438 assert_eq!(
1439 root_from(None, None, Some("/home/me".into())),
1440 PathBuf::from("/home/me/.roteiro/security")
1441 );
1442 }
1443
1444 #[test]
1445 fn provisioning_a_vendored_asset_installs_and_records_it() {
1446 let cache = Cache::new("vendored");
1447 let record = provision(&cache.0, rules()).expect("provision");
1448 assert_eq!(record.kind, AssetKind::Rules);
1449 assert_eq!(record.digest.len(), 64);
1450 assert!(!record.fetched_at.is_empty());
1451
1452 // The file is really there, and is really the vendored bytes.
1453 let AssetSource::Vendored(bytes) = rules().source else {
1454 panic!("the rule set is a vendored asset");
1455 };
1456 assert_eq!(
1457 std::fs::read(asset_path(&cache.0, rules())).expect("read"),
1458 bytes
1459 );
1460 assert_eq!(installed(&cache.0, rules()), Some(record));
1461 }
1462
1463 /// `prefetch` is a thing you run when unsure, so running it twice must be
1464 /// harmless and must not change what a run will read.
1465 #[test]
1466 fn provisioning_is_idempotent() {
1467 let cache = Cache::new("idempotent");
1468 let first = provision(&cache.0, rules()).expect("first");
1469 let second = provision(&cache.0, rules()).expect("second");
1470 assert_eq!(first.digest, second.digest);
1471 }
1472
1473 /// The headline offline contract: a cold cache fails, names what is missing,
1474 /// and prints the exact command that fixes it.
1475 #[test]
1476 fn a_cold_cache_fails_with_the_named_offline_error() {
1477 let cache = Cache::new("cold");
1478 let err = resolve(&cache.0, "semgrep").expect_err("a cold cache must fail");
1479 let ExecError::AssetsUnavailableOffline {
1480 analyzer,
1481 missing,
1482 command,
1483 } = &err
1484 else {
1485 panic!("expected the offline error, got {err:?}");
1486 };
1487 assert_eq!(analyzer, "semgrep");
1488 assert_eq!(missing.len(), 1);
1489 assert_eq!(missing[0].id, "semgrep-rules");
1490 assert_eq!(command, "roteiro security prefetch --analyzer semgrep");
1491
1492 // The rendered message has to carry all of it, because that is what a
1493 // user on a plane actually reads.
1494 let message = err.to_string();
1495 assert!(message.contains("assets-unavailable-offline"), "{message}");
1496 assert!(message.contains("semgrep-rules"), "{message}");
1497 assert!(
1498 message.contains("roteiro security prefetch --analyzer semgrep"),
1499 "{message}"
1500 );
1501 }
1502
1503 #[test]
1504 fn a_warm_cache_resolves_to_the_provisioned_path() {
1505 let cache = Cache::new("warm");
1506 provision(&cache.0, rules()).expect("provision");
1507 let resolved = resolve(&cache.0, "semgrep").expect("a warm cache must resolve");
1508 assert_eq!(resolved.len(), 1);
1509 assert_eq!(resolved[0].0, "semgrep-rules");
1510 assert_eq!(resolved[0].1, asset_path(&cache.0, rules()));
1511 }
1512
1513 /// A record that no longer describes the bytes is worse than no record: a
1514 /// run would stamp a `rules_digest` that does not match what it read.
1515 #[test]
1516 fn an_asset_edited_after_provisioning_is_refused_not_warned_about() {
1517 let cache = Cache::new("tampered");
1518 provision(&cache.0, rules()).expect("provision");
1519 std::fs::write(asset_path(&cache.0, rules()), b"rules: []\n").expect("tamper");
1520
1521 let err = resolve(&cache.0, "semgrep").expect_err("tampering must be refused");
1522 let ExecError::AssetsUnavailableOffline { missing, .. } = &err else {
1523 panic!("expected the offline error");
1524 };
1525 assert!(
1526 missing[0].reason.contains("no longer match"),
1527 "{}",
1528 missing[0].reason
1529 );
1530 }
1531
1532 /// Roteiro never fetches the advisory database. Absent, it says where it
1533 /// looked and what to run — and does not go and get it.
1534 #[test]
1535 fn an_absent_external_asset_is_explained_never_fetched() {
1536 let cache = Cache::new("external");
1537 let err = provision(&cache.0, advisory_db()).expect_err("must not be fetched");
1538 let AssetError::ExternalMissing { hint, analyzer, .. } = &err else {
1539 panic!("expected ExternalMissing, got {err:?}");
1540 };
1541 assert_eq!(*analyzer, "cargo-audit");
1542 assert!(hint.contains("advisory-db"), "{hint}");
1543 assert!(
1544 err.to_string().contains("roteiro security prefetch"),
1545 "{err}"
1546 );
1547 }
1548
1549 #[test]
1550 fn a_directory_asset_is_digested_by_content_not_by_layout() {
1551 let cache = Cache::new("tree");
1552 let db = asset_path(&cache.0, advisory_db());
1553 std::fs::create_dir_all(db.join("crates/openssl")).expect("create");
1554 std::fs::write(db.join("crates/openssl/RUSTSEC-2026-0031.md"), b"a").expect("write");
1555 std::fs::write(db.join("README.md"), b"b").expect("write");
1556
1557 let first = provision(&cache.0, advisory_db()).expect("provision");
1558 assert_eq!(first.files, Some(2));
1559
1560 // A `.git` directory is bookkeeping, not advisory data: adding one must
1561 // not move the digest.
1562 std::fs::create_dir_all(db.join(".git")).expect("create");
1563 std::fs::write(db.join(".git/HEAD"), b"ref: refs/heads/main").expect("write");
1564 assert_eq!(
1565 provision(&cache.0, advisory_db()).expect("again").digest,
1566 first.digest
1567 );
1568
1569 // Changing an advisory does move it.
1570 std::fs::write(db.join("README.md"), b"c").expect("write");
1571 assert_ne!(
1572 provision(&cache.0, advisory_db()).expect("third").digest,
1573 first.digest
1574 );
1575 }
1576
1577 #[test]
1578 fn status_reports_what_is_provisioned_and_what_is_not() {
1579 let cache = Cache::new("status");
1580 let cold = status(&cache.0, Some("semgrep"));
1581 assert_eq!(cold.len(), 1);
1582 assert!(cold[0].installed.is_none());
1583 assert!(cold[0].verified.is_none());
1584 assert!(cold[0].age_days.is_none());
1585
1586 provision(&cache.0, rules()).expect("provision");
1587 let warm = status(&cache.0, Some("semgrep"));
1588 assert_eq!(warm[0].verified, Some(true));
1589 assert_eq!(warm[0].age_days, Some(0));
1590 assert_eq!(warm[0].installed.as_ref().map(|r| r.digest.len()), Some(64));
1591
1592 // …and it notices when the bytes stop matching.
1593 std::fs::write(asset_path(&cache.0, rules()), b"rules: []\n").expect("tamper");
1594 assert_eq!(status(&cache.0, Some("semgrep"))[0].verified, Some(false));
1595 }
1596
1597 #[test]
1598 fn status_covers_every_analyzer_when_none_is_named() {
1599 let cache = Cache::new("status-all");
1600 assert_eq!(status(&cache.0, None).len(), ASSETS.len());
1601 assert!(status(&cache.0, Some("no-such-analyzer")).is_empty());
1602 }
1603
1604 /// The install paths of a downloadable asset name where bytes from the
1605 /// network are written, so a `..` in one would write outside the asset
1606 /// cache. They are compiled in today, which is exactly why the check is
1607 /// worth having: nothing else would notice a typo that escaped.
1608 #[test]
1609 fn a_download_path_that_escapes_the_asset_directory_is_refused() {
1610 static ESCAPING: &[super::DownloadFile] = &[super::DownloadFile {
1611 path: "../../outside.zip",
1612 url: "https://example.invalid/outside.zip",
1613 }];
1614 let cache = Cache::new("escape");
1615 let spec = super::AssetSpec {
1616 id: "escaping-asset",
1617 analyzer: "osv-scanner",
1618 kind: AssetKind::AdvisoryDb,
1619 source: AssetSource::Download { files: ESCAPING },
1620 file: "",
1621 licence: "n/a",
1622 };
1623 let fetched = std::cell::Cell::new(false);
1624 let fetch = |_: &str, _: &std::path::Path| {
1625 fetched.set(true);
1626 Ok(())
1627 };
1628 let err = super::provision_with(&cache.0, &spec, Some(&fetch))
1629 .expect_err("an escaping path must be refused");
1630 assert!(
1631 matches!(err, AssetError::UnsafeInstallPath { .. }),
1632 "{err:?}"
1633 );
1634 assert!(
1635 !fetched.get(),
1636 "the path is checked before anything is fetched"
1637 );
1638 }
1639
1640 /// An analyzer this build cannot run has no assets, and asking for them is
1641 /// not an error — it is simply an empty answer.
1642 #[test]
1643 fn an_unknown_analyzer_needs_nothing() {
1644 assert!(assets_for("no-such-analyzer").is_empty());
1645 let cache = Cache::new("unknown");
1646 assert!(
1647 resolve(&cache.0, "no-such-analyzer")
1648 .expect("no assets")
1649 .is_empty()
1650 );
1651 }
1652}