rto_exec/sandbox_store.rs
1//! The **sandbox image store**: what it is holding, and dropping it safely.
2//!
3//! `roteiro security prefetch` obtains and `roteiro security status` reports.
4//! Nothing removed, so the store only grew — 2.9 GB when issue #433 was filed,
5//! 8.7 GB on the machine this module was written on, 12 GB after one afternoon
6//! of trying candidate builder images. ADR-0014 v1.6 gives provisioning its third
7//! verb, and this module is the half of it that knows what is on disk.
8//!
9//! # The store has no names in it, so deletion starts at the index
10//!
11//! Everything under `<asset-root>/boxlite-home/images/` is keyed by digest —
12//! `layers/`, `configs/`, `manifests/`, `extracted/`, `disk-images/` — and not one
13//! path carries an image's name. The name → digest mapping lives only in
14//! `db/boxlite.db`'s `image_index` table. **There is no filesystem-only route**:
15//! a walk can tell you the store is 8.5 GB and cannot tell you which image any of
16//! it belongs to.
17//!
18//! So [`status`] and [`clear`] both start by reading that table, and every path
19//! this module touches is derived from a row in it. A file it cannot derive is
20//! never deleted on a per-image request and is reported by name — see
21//! [`SandboxStatus::unattributed`].
22//!
23//! # Deletion is a set difference, never a walk
24//!
25//! Blobs are shared. Two images can share a base layer, and one image's layer
26//! list can name the same digest twice — `cimg/rust`'s does, in the store this
27//! was written against. Dropping image A must therefore remove only the digests
28//! **no surviving image references**, which is a set difference over the whole
29//! index rather than a walk of A's own object list.
30//!
31//! Getting this wrong is invisible until it isn't. The first pair of images with
32//! a common layer turns a naive per-image delete into a *broken surviving image*
33//! — not an error, not a warning, just a `security run` that fails much later
34//! with a missing blob. [`plan`] is the only place the difference is computed and
35//! [`clear`] cannot delete anything [`plan`] did not put in the doomed set.
36//!
37//! # The derived artifacts, and the association that was thought to be missing
38//!
39//! `images/disk-images/*.ext4` is the largest thing in the store — 3.7 GB of 8.7
40//! on this machine — and issue #433's hand-clearing notes record that nothing
41//! mapped an image to the disk image built from it, so they were separated by
42//! **mtime** and the note says a shipped `clear` must not do that.
43//!
44//! It does not have to. `boxlite`'s `ImageObject::compute_image_digest` keys a
45//! disk image by the SHA-256 of its layer digest strings concatenated in manifest
46//! order, and `ImageDiskManager::disk_path` writes it as `{digest}.ext4` with the
47//! `:` turned into `-`. That is [`image_digest`], it is computed from
48//! `image_index.layers` alone, and it reproduces all three filenames in the live
49//! store exactly. The base rootfs in `bases/` is keyed off the same value:
50//! `base_disk.name` is `{image_digest[..12]}-{guest_binary_hash[..12]}`.
51//!
52//! **It is a re-derivation of another crate's private function, so it is checked
53//! rather than trusted.** [`Attribution`] records whether every disk image on
54//! disk was claimed by some indexed image; when one is not, it becomes
55//! `unattributed` and a per-image `clear` leaves it alone. A `boxlite` upgrade
56//! that changed the key would show up as unattributed bytes in `status`, which is
57//! a visible wrong number rather than a silent wrong deletion.
58//!
59//! # What may be dropped, and the two things here that may not
60//!
61//! ADR-0014 v1.6's permission and its limit are one property: everything under
62//! the asset cache is re-obtainable from a pinned digest, so clearing costs time
63//! and never information — **and the verb may therefore never reach anything that
64//! is not**. Under this store root, two things are not:
65//!
66//! - **A `base_disk` row of kind `snapshot` or `clone_base`.** A `rootfs` base is
67//! rebuilt from the image; a snapshot is the state of a box somebody ran, and no
68//! digest re-obtains it. They are never deleted, and they are reported —
69//! [`ClearReport::preserved`].
70//! - **Anything under the store root this module does not recognise.** A new
71//! `boxlite` layout directory is not known to be re-obtainable just because it
72//! turned up in a cache, so [`clear`] refuses rather than guesses:
73//! [`StoreError::UnrecognisedEntry`].
74//!
75//! Nothing outside `<asset-root>/boxlite-home` is reachable from here at all. The
76//! findings layers, the memory records and `graph.db` live in the repository's
77//! store, which this module has no path to and does not link against.
78//!
79//! # Two prefixes that look like one
80//!
81//! The index stores `sha256:abc…`; the filesystem writes `sha256-abc…`. Issue
82//! #433's first hand-clearing pass matched nothing because of it. [`blob_name`]
83//! is the single translation, and every path in this module goes through it.
84//!
85//! @rto:0014
86//! @rto:0013
87
88use std::collections::{BTreeMap, BTreeSet};
89use std::path::{Path, PathBuf};
90
91use serde::Serialize;
92
93use crate::sha256_hex;
94
95/// Schema tag for the sandbox-store status document.
96pub const SANDBOX_STATUS_SCHEMA: &str = "roteiro.sandbox.status/v1";
97
98/// Schema tag for the sandbox-store clear document.
99pub const SANDBOX_CLEAR_SCHEMA: &str = "roteiro.sandbox.clear/v1";
100
101/// The sandbox store's directory under the asset cache root.
102///
103/// `boxlite.rs` spells the same literal where it builds a runtime's `home_dir`,
104/// and cannot share this constant without `exec-boxlite` becoming a condition of
105/// being able to *clear* a store a previous build filled. The two are held
106/// together by `the_store_directory_is_the_one_boxlite_is_pointed_at` instead.
107pub const SANDBOX_STORE_DIR: &str = "boxlite-home";
108
109/// The index `boxlite` keeps its name → digest mapping in, under the store root.
110const INDEX_DB: &str = "db/boxlite.db";
111
112/// Top-level entries [`clear`] knows the disposition of.
113///
114/// Anything else under the store root stops [`clear`] with
115/// [`StoreError::UnrecognisedEntry`], because "it appeared in a cache" is not
116/// evidence that a digest re-obtains it (ADR-0014 v1.6).
117const KNOWN_ENTRIES: &[&str] = &[".lock", "bases", "boxes", "db", "images", "locks", "tmp"];
118
119/// The digest-keyed object directories under `images/`.
120const IMAGE_DIRS: &[&str] = &["configs", "disk-images", "extracted", "layers", "manifests"];
121
122/// Turn an index digest (`sha256:abc…`) into the name the filesystem uses
123/// (`sha256-abc…`).
124///
125/// The single place that translation happens. It is one character and it is the
126/// reason issue #433's first hand-clearing pass matched nothing at all.
127#[must_use]
128pub fn blob_name(digest: &str) -> String {
129 digest.replace(':', "-")
130}
131
132/// The cache key `boxlite` derives a disk image and a base rootfs from: the
133/// SHA-256 of the layer digest strings, concatenated in manifest order.
134///
135/// A re-derivation of `boxlite`'s `ImageObject::compute_image_digest`, which is
136/// private to that crate. It is what removes the mtime heuristic issue #433's
137/// notes fell back to, and [`Attribution`] is what keeps it honest if `boxlite`
138/// ever changes it.
139///
140/// Duplicates are **not** removed and order is **not** normalised: this hashes the
141/// layer list as the manifest wrote it, because that is what the other side does.
142/// `cimg/rust` names one digest twice and still resolves to the filename on disk.
143#[must_use]
144pub fn image_digest(layers: &[String]) -> String {
145 let joined: String = layers.concat();
146 format!("sha256:{}", sha256_hex(joined.as_bytes()))
147}
148
149/// What went wrong, in terms of what to do about it.
150#[derive(Debug, thiserror::Error)]
151#[non_exhaustive]
152pub enum StoreError {
153 /// The index could not be read. Without it nothing in the store has a name,
154 /// so neither verb can proceed — see this module's documentation.
155 #[error("cannot read the sandbox image index at {path}: {message}")]
156 Index {
157 /// The database that could not be read.
158 path: String,
159 /// What the database layer said.
160 message: String,
161 },
162 /// A filesystem operation failed.
163 #[error("{action} {path}: {message}")]
164 Io {
165 /// What was being attempted, as a verb phrase.
166 action: &'static str,
167 /// The path it was attempted on.
168 path: String,
169 /// What the operating system said.
170 message: String,
171 },
172 /// A per-image request named an image the store is not holding.
173 ///
174 /// Carries what it *is* holding, because the likely cause is a tag written
175 /// where the index has a digest reference, and a listing is the way forward.
176 #[error("the sandbox store is not holding `{reference}`; it is holding: {known}")]
177 UnknownImage {
178 /// What was asked for.
179 reference: String,
180 /// The references the index does have, comma-separated.
181 known: String,
182 },
183 /// A box is registered in the store, so something may be using these bytes.
184 ///
185 /// `boxlite` takes an exclusive `flock` on `<store>/.lock` for the lifetime of
186 /// a runtime, which this crate cannot take back: `unsafe_code = "forbid"`
187 /// rules out the `libc::flock` call that acquires it. So the guard is the
188 /// evidence a *lock* would have protected — a registered box — and it is
189 /// checked rather than assumed absent.
190 #[error(
191 "the sandbox store has {boxes} registered box(es); \
192 stop them before clearing, or the bytes a running box is reading go away underneath it"
193 )]
194 LiveBoxes {
195 /// How many boxes are registered.
196 boxes: usize,
197 },
198 /// Something under the store root that this module does not recognise.
199 ///
200 /// ADR-0014 v1.6's limit, enforced rather than trusted: `clear` may drop what
201 /// a pinned digest re-obtains and may drop nothing else, and an unknown entry
202 /// is not known to be re-obtainable.
203 #[error(
204 "the sandbox store holds `{entry}`, which this version of Roteiro does not recognise; \
205 it will not be cleared, and nothing else was cleared either — \
206 report it on issue #433, because an entry a digest does not re-obtain does not belong here"
207 )]
208 UnrecognisedEntry {
209 /// The entry's name under the store root.
210 entry: String,
211 },
212 /// A `base_disk` row points outside the store root.
213 ///
214 /// `base_path` is an absolute path recorded when the base was built, so it is
215 /// data rather than a derivation, and data can name anywhere. This is the
216 /// check that a row cannot aim deletion at a path outside the asset cache.
217 #[error("the sandbox index has a base disk at {path}, which is outside the store root {root}")]
218 BaseOutsideStore {
219 /// Where the row pointed.
220 path: String,
221 /// The root it had to be under.
222 root: String,
223 },
224}
225
226/// Which images a [`clear`] is being asked for.
227///
228/// Two variants rather than an `Option<String>`, because ADR-0014 v1.6 requires
229/// that "clear this image" and "clear everything" be **different arguments** — a
230/// caller asking for one must not be able to receive the other by supplying
231/// nothing.
232#[derive(Debug, Clone, PartialEq, Eq)]
233pub enum Scope {
234 /// Every cached image, and the unattributed bytes alongside them.
235 Everything,
236 /// One image, by the reference the index holds it under. Anything a surviving
237 /// image still references stays.
238 Image(String),
239}
240
241impl Scope {
242 /// The token this serialises as, for a report that names what was asked for.
243 #[must_use]
244 pub fn as_str(&self) -> &str {
245 match self {
246 Self::Everything => "everything",
247 Self::Image(reference) => reference,
248 }
249 }
250}
251
252/// Whether the store's derived artifacts were all claimed by an indexed image.
253///
254/// [`image_digest`] re-derives a key that is private to `boxlite`, so this is the
255/// evidence that the re-derivation still matches. `Complete` means every
256/// `disk-images/*.ext4` in the store was claimed; `Partial` names how many were
257/// not, and those bytes are reported as unattributed rather than deleted.
258#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)]
259#[serde(rename_all = "kebab-case")]
260pub enum Attribution {
261 /// Every derived artifact belongs to an indexed image.
262 Complete,
263 /// Some do not. A per-image `clear` will not touch them.
264 Partial,
265}
266
267/// One image's byte accounting.
268///
269/// `total` is everything the image references; `exclusive` is what dropping this
270/// image *alone* would free. They differ exactly when another cached image shares
271/// a blob, which is the case the set difference exists for — and reporting only
272/// `total` would promise bytes back that a shared layer keeps.
273#[derive(Debug, Clone, Copy, Default, Serialize)]
274pub struct ImageBytes {
275 /// The manifest and config JSON.
276 pub metadata: u64,
277 /// The compressed layer tarballs under `images/layers/`.
278 pub layers: u64,
279 /// The unpacked layer trees under `images/extracted/`.
280 pub extracted: u64,
281 /// The ext4 disk image built from the layer stack, if one has been built.
282 pub disk_image: u64,
283 /// The guest rootfs base built from that disk image, if one has been built.
284 pub base_disk: u64,
285 /// Everything above.
286 pub total: u64,
287 /// What dropping this image alone would actually free — `total` minus every
288 /// byte another cached image also references.
289 pub exclusive: u64,
290}
291
292/// How much of an image's **pulled** content is on disk.
293///
294/// Manifest, config and one entry per unique layer digest — the objects a pull
295/// produces and a run consumes. Deliberately *not* counting the extracted trees,
296/// the disk image or the base rootfs: those are built lazily on first run and are
297/// a cache below this cache, so an image that has only ever been pulled is
298/// complete without them.
299///
300/// The unit matches what issue #433's hand verification counted: 15/15 for
301/// `semgrep` (one manifest, one config, thirteen layers) and 3/3 for `debian`.
302#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)]
303pub struct Objects {
304 /// How many objects the index says this image has.
305 pub expected: usize,
306 /// How many of them are on disk.
307 pub present: usize,
308}
309
310impl Objects {
311 /// Whether every object the index names is on disk.
312 #[must_use]
313 pub fn complete(self) -> bool {
314 self.expected == self.present
315 }
316}
317
318/// One image in the store.
319#[derive(Debug, Clone, Serialize)]
320pub struct CachedImage {
321 /// The reference the index holds it under — a tag, or a digest-pinned name.
322 pub reference: String,
323 /// The manifest digest, as the index spells it (`sha256:…`).
324 pub manifest_digest: String,
325 /// The config digest, as the index spells it.
326 pub config_digest: String,
327 /// The derived key its disk image and base rootfs are stored under
328 /// ([`image_digest`]).
329 pub image_digest: String,
330 /// When the pull completed, as the index recorded it.
331 pub cached_at: String,
332 /// Whether the index considers the pull finished. A `false` here is a partial
333 /// pull, not a corrupt one, and `security prefetch` completes it.
334 pub pull_complete: bool,
335 /// How many **distinct** layer digests it has. Below the manifest's layer
336 /// count when a layer is named twice.
337 pub layers: usize,
338 /// Its byte accounting.
339 pub bytes: ImageBytes,
340 /// How much of its pulled content is on disk.
341 pub objects: Objects,
342 /// Whether a disk image has been built from it.
343 pub disk_image_built: bool,
344 /// Whether a guest rootfs base has been built from it.
345 pub base_disk_built: bool,
346}
347
348/// Bytes under the store root that no indexed image claims.
349#[derive(Debug, Clone, Serialize)]
350pub struct Unattributed {
351 /// Where it is, relative to the store root.
352 pub path: String,
353 /// Its size in bytes.
354 pub bytes: u64,
355}
356
357/// Something under the store root that [`clear`] deliberately leaves alone, and
358/// why.
359#[derive(Debug, Clone, Serialize)]
360pub struct Preserved {
361 /// Where it is, relative to the store root.
362 pub path: String,
363 /// Why it is not re-obtainable from a pinned digest, in one sentence.
364 pub reason: String,
365}
366
367/// What the sandbox store is holding.
368///
369/// # Why `scope` is a field and not a sentence in a doc comment
370///
371/// The same reason [`crate::tool_security::MachineScope`] carries one: an asset
372/// root is a property of the **machine**, and a caller who selected a project
373/// (ADR-0008) will otherwise read a store size as that project's. There is one
374/// sandbox store per asset root and every repository on the host shares it, so
375/// the document says so in a field that travels with any half of it that gets
376/// quoted.
377#[derive(Debug, Clone, Serialize)]
378pub struct SandboxStatus {
379 /// Stable schema tag ([`SANDBOX_STATUS_SCHEMA`]).
380 pub schema: &'static str,
381 /// Always `"machine"`. The store is shared by every repository on this host.
382 pub scope: &'static str,
383 /// The store root these numbers describe.
384 pub store: String,
385 /// Whether there is a store there at all. `false` means nothing is cached,
386 /// which is a different fact from an empty index.
387 pub present: bool,
388 /// Every image the index is holding, largest first.
389 pub images: Vec<CachedImage>,
390 /// Whether every derived artifact was claimed by one of them.
391 pub attribution: Attribution,
392 /// Bytes no indexed image claims. A per-image `clear` never touches these.
393 pub unattributed: Vec<Unattributed>,
394 /// State a digest does not re-obtain, which `clear` will not remove.
395 pub preserved: Vec<Preserved>,
396 /// How many boxes are registered. Non-zero blocks a `clear`.
397 pub live_boxes: usize,
398 /// Every byte under the store root.
399 pub total_bytes: u64,
400}
401
402/// One image [`clear`] removed, and what removing it freed.
403#[derive(Debug, Clone, Serialize)]
404pub struct RemovedImage {
405 /// The reference it was held under.
406 pub reference: String,
407 /// Bytes freed by removing it — its exclusive bytes, never its total.
408 pub freed_bytes: u64,
409 /// How many objects were removed from the filesystem.
410 pub objects_removed: usize,
411}
412
413/// A surviving image, re-checked against the filesystem after the deletion.
414///
415/// The assertion that matters. A set-difference bug does not present as an error;
416/// it presents as an image whose blobs are gone, discovered on the next run. So
417/// `clear` re-resolves every survivor's manifest, config and layers against the
418/// filesystem *after* deleting, and says so in its report — issue #433's "a
419/// `clear` that cannot demonstrate the surviving images are still complete is one
420/// nobody trusts twice".
421#[derive(Debug, Clone, Serialize)]
422pub struct VerifiedImage {
423 /// The reference it is held under.
424 pub reference: String,
425 /// Its object tally after the deletion.
426 pub objects: Objects,
427 /// Whether every object is still there.
428 pub complete: bool,
429}
430
431/// What a [`clear`] would do, or did.
432#[derive(Debug, Clone, Serialize)]
433pub struct ClearReport {
434 /// Stable schema tag ([`SANDBOX_CLEAR_SCHEMA`]).
435 pub schema: &'static str,
436 /// Always `"machine"`, for the reason [`SandboxStatus::scope`] gives.
437 pub scope: &'static str,
438 /// The store root that was cleared.
439 pub store: String,
440 /// What was asked for — a reference, or `everything`.
441 pub requested: String,
442 /// Whether this is a plan or a completed removal.
443 pub applied: bool,
444 /// The images removed.
445 pub removed: Vec<RemovedImage>,
446 /// The unattributed bytes removed. Only ever populated for
447 /// [`Scope::Everything`].
448 pub removed_unattributed: Vec<Unattributed>,
449 /// Bytes accounted for by the objects removed.
450 pub freed_bytes: u64,
451 /// Every byte under the store root before the deletion.
452 pub store_bytes_before: u64,
453 /// Every byte under the store root after it. Equal to `store_bytes_before` on
454 /// a plan.
455 pub store_bytes_after: u64,
456 /// The survivors, re-checked blob by blob after the deletion.
457 pub retained: Vec<VerifiedImage>,
458 /// State a digest does not re-obtain, left alone and named.
459 pub preserved: Vec<Preserved>,
460}
461
462impl ClearReport {
463 /// The bytes the filesystem actually gave back.
464 ///
465 /// Reported alongside [`ClearReport::freed_bytes`] rather than instead of it:
466 /// the accounted figure is what this module believes it removed, the measured
467 /// one is what the store shrank by, and the two are checkable against each
468 /// other by anyone holding a `du`.
469 ///
470 /// They are not identical, and the gap is one thing: the index moves under a
471 /// `clear` without being part of what was removed. **`SQLite` does not shrink
472 /// a file when rows are deleted** — it frees pages inside it — and a `DELETE`
473 /// under a write transaction can add a page, while a checkpoint can hand back
474 /// a write-ahead log. So the gap runs in **both** directions, it is kilobytes
475 /// against a clear measured in gigabytes, and
476 /// `the_accounted_bytes_and_the_measured_bytes_differ_only_by_the_index` is
477 /// what keeps that claim true. Anything larger is a defect, which is why both
478 /// numbers are reported rather than one.
479 ///
480 /// Measured on the real store: dropping an image that shared every layer with
481 /// a survivor accounted for 0 bytes and measured 32 KiB back, all of it the
482 /// index.
483 #[must_use]
484 pub fn measured_freed_bytes(&self) -> u64 {
485 self.store_bytes_before
486 .saturating_sub(self.store_bytes_after)
487 }
488
489 /// Whether every surviving image is still complete.
490 #[must_use]
491 pub fn survivors_intact(&self) -> bool {
492 self.retained.iter().all(|image| image.complete)
493 }
494}
495
496// ---------------------------------------------------------------------------
497// The index
498// ---------------------------------------------------------------------------
499
500/// One `image_index` row, with its layer list parsed.
501#[derive(Debug, Clone)]
502struct IndexRow {
503 reference: String,
504 manifest_digest: String,
505 config_digest: String,
506 /// In manifest order and **with duplicates**, because [`image_digest`] hashes
507 /// the list as the manifest wrote it.
508 layers: Vec<String>,
509 cached_at: String,
510 complete: bool,
511}
512
513impl IndexRow {
514 /// The distinct layer digests, for the object lists that are keyed by digest.
515 fn unique_layers(&self) -> BTreeSet<String> {
516 self.layers.iter().cloned().collect()
517 }
518}
519
520/// One `base_disk` row.
521#[derive(Debug, Clone)]
522struct BaseRow {
523 name: String,
524 kind: String,
525 path: PathBuf,
526}
527
528/// What the index says, read in one pass.
529#[derive(Debug, Default)]
530struct Index {
531 images: Vec<IndexRow>,
532 /// Base disks that live under the store root. **Only** these; a row naming a
533 /// path elsewhere is in `escaped` and no derivation here can reach it.
534 bases: Vec<BaseRow>,
535 /// Base-disk rows pointing outside the store root.
536 ///
537 /// `base_path` is an absolute path recorded when the base was built, so it is
538 /// data rather than a derivation — and data can name anywhere, including the
539 /// repository store this verb must never reach. Split off at the point it is
540 /// read, so no later code has to remember to check: a relocated store makes
541 /// every row here stale at once, and the guarantee wanted is that nothing
542 /// outside the root is measured, listed **or** deleted, not only the last of
543 /// those.
544 escaped: Vec<PathBuf>,
545 boxes: usize,
546}
547
548/// Read `image_index`, `base_disk` and the box count.
549///
550/// Opened read-only: [`status`] must not be able to write to a store it is only
551/// describing, and [`clear`] does its own writing through a separate connection
552/// once it has decided what to do.
553fn read_index(store: &Path) -> Result<Index, StoreError> {
554 let path = store.join(INDEX_DB);
555 if !path.exists() {
556 return Ok(Index::default());
557 }
558 let fail = |message: String| StoreError::Index {
559 path: path.display().to_string(),
560 message,
561 };
562 let db = rusqlite::Connection::open_with_flags(
563 &path,
564 rusqlite::OpenFlags::SQLITE_OPEN_READ_ONLY | rusqlite::OpenFlags::SQLITE_OPEN_URI,
565 )
566 .map_err(|error| fail(error.to_string()))?;
567
568 let mut images = Vec::new();
569 {
570 let mut statement = db
571 .prepare(
572 "SELECT reference, manifest_digest, config_digest, layers, cached_at, complete \
573 FROM image_index ORDER BY reference",
574 )
575 .map_err(|error| fail(error.to_string()))?;
576 let rows = statement
577 .query_map([], |row| {
578 Ok((
579 row.get::<_, String>(0)?,
580 row.get::<_, String>(1)?,
581 row.get::<_, String>(2)?,
582 row.get::<_, String>(3)?,
583 row.get::<_, String>(4)?,
584 row.get::<_, i64>(5)?,
585 ))
586 })
587 .map_err(|error| fail(error.to_string()))?;
588 for row in rows {
589 let (reference, manifest_digest, config_digest, layers, cached_at, complete) =
590 row.map_err(|error| fail(error.to_string()))?;
591 // A layer list that will not parse is a row this module cannot derive
592 // paths from, and inventing an empty one would make its blobs look
593 // unreferenced — which is how a set difference deletes a live image.
594 // Refusing is the only safe reading.
595 let layers: Vec<String> = serde_json::from_str(&layers).map_err(|error| {
596 fail(format!(
597 "image_index row `{reference}` has an unreadable layer list: {error}"
598 ))
599 })?;
600 images.push(IndexRow {
601 reference,
602 manifest_digest,
603 config_digest,
604 layers,
605 cached_at,
606 complete: complete != 0,
607 });
608 }
609 }
610
611 let mut bases = Vec::new();
612 let mut escaped = Vec::new();
613 {
614 let mut statement = db
615 .prepare("SELECT name, kind, base_path FROM base_disk ORDER BY id")
616 .map_err(|error| fail(error.to_string()))?;
617 let rows = statement
618 .query_map([], |row| {
619 Ok((
620 row.get::<_, Option<String>>(0)?,
621 row.get::<_, String>(1)?,
622 row.get::<_, String>(2)?,
623 ))
624 })
625 .map_err(|error| fail(error.to_string()))?;
626 for row in rows {
627 let (name, kind, path) = row.map_err(|error| fail(error.to_string()))?;
628 let path = PathBuf::from(path);
629 if path.starts_with(store) {
630 bases.push(BaseRow {
631 name: name.unwrap_or_default(),
632 kind,
633 path,
634 });
635 } else {
636 escaped.push(path);
637 }
638 }
639 }
640
641 let boxes = db
642 .query_row("SELECT COUNT(*) FROM box_config", [], |row| {
643 row.get::<_, i64>(0)
644 })
645 .map_err(|error| fail(error.to_string()))?;
646
647 Ok(Index {
648 images,
649 bases,
650 escaped,
651 boxes: usize::try_from(boxes).unwrap_or(usize::MAX),
652 })
653}
654
655// ---------------------------------------------------------------------------
656// Object lists
657// ---------------------------------------------------------------------------
658
659/// The `bases/` prefix a `rootfs` base disk belonging to `digest` is named with.
660///
661/// `boxlite`'s `GuestRootfsManager::version_key` is
662/// `{image_digest[..12]}-{guest_binary_hash[..12]}`, so the image half is a
663/// prefix match and the guest half — which changes when the runtime does — is not
664/// something this crate has to know.
665fn base_name_prefix(image_digest: &str) -> String {
666 let bare = image_digest.strip_prefix("sha256:").unwrap_or(image_digest);
667 format!("{}-", &bare[..12.min(bare.len())])
668}
669
670/// Every path one image references, split into the pulled objects and the
671/// derived ones.
672#[derive(Debug, Default)]
673struct ImageObjects {
674 /// Manifest, config and layer tarballs — what a pull produces. The unit
675 /// [`Objects`] counts.
676 pulled: Vec<PathBuf>,
677 /// Extracted trees, the disk image and the base rootfs — built lazily on
678 /// first run, and a cache below this cache.
679 derived: Vec<PathBuf>,
680 /// How many `rootfs` base disks the index attributes to this image.
681 bases: usize,
682}
683
684impl ImageObjects {
685 fn all(&self) -> impl Iterator<Item = &PathBuf> {
686 self.pulled.iter().chain(self.derived.iter())
687 }
688}
689
690/// Resolve one index row to the paths it references.
691///
692/// The index manifest — the `application/vnd.oci.image.index.v1+json` a tag or a
693/// digest reference resolves through — is *not* here. It is claimed separately by
694/// [`index_manifests_for`], because a tag reference does not record which index it
695/// came from and the association has to be read out of the files themselves.
696fn objects_for(store: &Path, row: &IndexRow, bases: &[BaseRow]) -> ImageObjects {
697 let images = store.join("images");
698 let mut objects = ImageObjects::default();
699
700 objects.pulled.push(
701 images
702 .join("manifests")
703 .join(format!("{}.json", blob_name(&row.manifest_digest))),
704 );
705 objects.pulled.push(
706 images
707 .join("configs")
708 .join(format!("{}.json", blob_name(&row.config_digest))),
709 );
710 for layer in row.unique_layers() {
711 let name = blob_name(&layer);
712 objects
713 .pulled
714 .push(images.join("layers").join(format!("{name}.tar.gz")));
715 objects.derived.push(images.join("extracted").join(name));
716 }
717
718 let digest = image_digest(&row.layers);
719 objects.derived.push(
720 images
721 .join("disk-images")
722 .join(format!("{}.ext4", blob_name(&digest))),
723 );
724 let prefix = base_name_prefix(&digest);
725 for base in bases {
726 if base.kind == "rootfs" && base.name.starts_with(&prefix) {
727 objects.derived.push(base.path.clone());
728 objects.bases += 1;
729 }
730 }
731 objects
732}
733
734/// The index manifests that resolve to any of `retained` platform manifests.
735///
736/// `docker.io/library/debian:bookworm-slim` is held under a **tag**, so its index
737/// digest appears nowhere in `image_index` — only the platform manifest it
738/// resolved to does. Deleting every manifest file the index does not name would
739/// therefore take the index file of a surviving image with it. Reading the files
740/// and keeping any that lists a retained manifest is the association the database
741/// does not carry.
742fn index_manifests_for(store: &Path, retained: &BTreeSet<String>) -> BTreeSet<PathBuf> {
743 let dir = store.join("images").join("manifests");
744 let mut keep = BTreeSet::new();
745 let Ok(entries) = std::fs::read_dir(&dir) else {
746 return keep;
747 };
748 for entry in entries.flatten() {
749 let path = entry.path();
750 let Ok(bytes) = std::fs::read(&path) else {
751 continue;
752 };
753 let Ok(document) = serde_json::from_slice::<serde_json::Value>(&bytes) else {
754 continue;
755 };
756 let Some(children) = document.get("manifests").and_then(|value| value.as_array()) else {
757 continue;
758 };
759 if children
760 .iter()
761 .filter_map(|child| child.get("digest").and_then(|value| value.as_str()))
762 .any(|digest| retained.contains(digest))
763 {
764 keep.insert(path);
765 }
766 }
767 keep
768}
769
770// ---------------------------------------------------------------------------
771// Sizes
772// ---------------------------------------------------------------------------
773
774/// Bytes **allocated** to one file — what removing it gives back.
775///
776/// Not its length. The ext4 disks in this store are sparse, and by a margin
777/// nobody would call rounding: `sha256-0a89fbeb….ext4` in the store measured for
778/// issue #433 is 1.23 GiB long and occupies 1.09 GiB, and the base rootfs beside
779/// it is 256 MiB long and occupies 110 MiB. Reporting the length would tell
780/// somebody a clear had freed 9.1 GiB where `du` says 8.7 GiB, and the number
781/// this verb exists to report would be the one number in it that could not be
782/// checked.
783///
784/// So it is `st_blocks`, which is what `du` counts and what the filesystem
785/// actually returns. Off Unix there is no such field and the length is the best
786/// available answer; nothing in the store is sparse on a filesystem without it.
787fn allocated(metadata: &std::fs::Metadata) -> u64 {
788 #[cfg(unix)]
789 {
790 use std::os::unix::fs::MetadataExt as _;
791 metadata.blocks() * 512
792 }
793 #[cfg(not(unix))]
794 {
795 metadata.len()
796 }
797}
798
799/// Bytes at `path`: one file's allocation, or the sum of every regular file
800/// beneath it.
801///
802/// See [`allocated`] for why this is not the apparent size.
803fn size_of(path: &Path) -> u64 {
804 let Ok(metadata) = std::fs::symlink_metadata(path) else {
805 return 0;
806 };
807 if !metadata.is_dir() {
808 return allocated(&metadata);
809 }
810 let mut total = allocated(&metadata);
811 let mut stack = vec![path.to_path_buf()];
812 while let Some(dir) = stack.pop() {
813 let Ok(entries) = std::fs::read_dir(&dir) else {
814 continue;
815 };
816 for entry in entries.flatten() {
817 let Ok(metadata) = entry.metadata() else {
818 continue;
819 };
820 total += allocated(&metadata);
821 if metadata.is_dir() {
822 stack.push(entry.path());
823 }
824 }
825 }
826 total
827}
828
829/// Measure every path once, so a digest shared by two images is not weighed twice.
830fn measure(paths: impl IntoIterator<Item = PathBuf>) -> BTreeMap<PathBuf, u64> {
831 let mut sizes = BTreeMap::new();
832 for path in paths {
833 sizes.entry(path).or_insert_with_key(|path| size_of(path));
834 }
835 sizes
836}
837
838// ---------------------------------------------------------------------------
839// status
840// ---------------------------------------------------------------------------
841
842/// Where the sandbox store lives under an asset cache root.
843#[must_use]
844pub fn store_root(asset_root: &Path) -> PathBuf {
845 asset_root.join(SANDBOX_STORE_DIR)
846}
847
848/// Report what the sandbox store is holding, per image, with sizes.
849///
850/// Enough to decide what to drop, which is ADR-0014 v1.6's third rule: a
851/// destructive verb with no way to see what it will destroy is invoked blind.
852///
853/// # Errors
854///
855/// [`StoreError::Index`] when the index exists and cannot be read. An **absent**
856/// store is not an error — it is `present: false`, because "nothing is cached" is
857/// an answer rather than a failure.
858pub fn status(asset_root: &Path) -> Result<SandboxStatus, StoreError> {
859 let store = store_root(asset_root);
860 let mut report = SandboxStatus {
861 schema: SANDBOX_STATUS_SCHEMA,
862 scope: "machine",
863 store: store.display().to_string(),
864 present: store.is_dir(),
865 images: Vec::new(),
866 attribution: Attribution::Complete,
867 unattributed: Vec::new(),
868 preserved: Vec::new(),
869 live_boxes: 0,
870 total_bytes: 0,
871 };
872 if !report.present {
873 return Ok(report);
874 }
875
876 let index = read_index(&store)?;
877 report.live_boxes = index.boxes;
878 report.total_bytes = size_of(&store);
879
880 let objects: Vec<(usize, ImageObjects)> = index
881 .images
882 .iter()
883 .enumerate()
884 .map(|(at, row)| (at, objects_for(&store, row, &index.bases)))
885 .collect();
886
887 // Every path any image references, measured once. `references` counts how
888 // many images claim each path, which is what turns a total into an exclusive
889 // figure without measuring anything twice.
890 let sizes = measure(
891 objects
892 .iter()
893 .flat_map(|(_, object)| object.all().cloned())
894 .collect::<Vec<_>>(),
895 );
896 let mut references: BTreeMap<&PathBuf, usize> = BTreeMap::new();
897 for (_, object) in &objects {
898 for path in object.all() {
899 *references.entry(path).or_default() += 1;
900 }
901 }
902
903 for (at, object) in &objects {
904 let row = &index.images[*at];
905 report
906 .images
907 .push(cached_image(&store, row, object, &sizes, &references));
908 }
909 report
910 .images
911 .sort_by_key(|image| std::cmp::Reverse(image.bytes.total));
912
913 let claimed: BTreeSet<PathBuf> = objects
914 .iter()
915 .flat_map(|(_, object)| object.all().cloned())
916 .chain(index_manifests_for(
917 &store,
918 &index
919 .images
920 .iter()
921 .map(|row| row.manifest_digest.clone())
922 .collect(),
923 ))
924 .chain(preserved_paths(&index))
925 .collect();
926 report.unattributed = unattributed(&store, &claimed);
927 if !report.unattributed.is_empty() {
928 report.attribution = Attribution::Partial;
929 }
930 report.preserved = preserved(&index);
931 Ok(report)
932}
933
934/// Build one image's row from its object list and the shared size table.
935fn cached_image(
936 store: &Path,
937 row: &IndexRow,
938 object: &ImageObjects,
939 sizes: &BTreeMap<PathBuf, u64>,
940 references: &BTreeMap<&PathBuf, usize>,
941) -> CachedImage {
942 let images = store.join("images");
943 let mut bytes = ImageBytes::default();
944 for path in object.all() {
945 let size = sizes.get(path).copied().unwrap_or_default();
946 bytes.total += size;
947 if references.get(path).copied().unwrap_or(1) == 1 {
948 bytes.exclusive += size;
949 }
950 if path.starts_with(images.join("layers")) {
951 bytes.layers += size;
952 } else if path.starts_with(images.join("extracted")) {
953 bytes.extracted += size;
954 } else if path.starts_with(images.join("disk-images")) {
955 bytes.disk_image += size;
956 } else if path.starts_with(images.join("manifests"))
957 || path.starts_with(images.join("configs"))
958 {
959 bytes.metadata += size;
960 } else {
961 bytes.base_disk += size;
962 }
963 }
964
965 let digest = image_digest(&row.layers);
966 let disk = images
967 .join("disk-images")
968 .join(format!("{}.ext4", blob_name(&digest)));
969 CachedImage {
970 reference: row.reference.clone(),
971 manifest_digest: row.manifest_digest.clone(),
972 config_digest: row.config_digest.clone(),
973 image_digest: digest,
974 cached_at: row.cached_at.clone(),
975 pull_complete: row.complete,
976 layers: row.unique_layers().len(),
977 bytes,
978 objects: Objects {
979 expected: object.pulled.len(),
980 present: object.pulled.iter().filter(|path| path.exists()).count(),
981 },
982 disk_image_built: disk.exists(),
983 base_disk_built: object.bases > 0,
984 }
985}
986
987/// Digest-keyed objects that no indexed image claims.
988///
989/// One entry per top-level object rather than a single total, because "146 MB of
990/// extracted layer nobody references" is actionable and "146 MB unaccounted" is
991/// not. The live store had exactly one — an extracted tree whose layer tarball
992/// and index row are both gone.
993///
994/// `claimed` must already carry the [`preserved`] paths as well as the objects
995/// every indexed image references: a snapshot's base disk is claimed by nobody
996/// and is not therefore spare.
997fn unattributed(store: &Path, claimed: &BTreeSet<PathBuf>) -> Vec<Unattributed> {
998 let images = store.join("images");
999 let mut scanned: Vec<PathBuf> = IMAGE_DIRS.iter().map(|dir| images.join(dir)).collect();
1000 scanned.push(store.join("bases"));
1001 let mut found = Vec::new();
1002 for dir in scanned {
1003 let Ok(entries) = std::fs::read_dir(&dir) else {
1004 continue;
1005 };
1006 for entry in entries.flatten() {
1007 let path = entry.path();
1008 if claimed.contains(&path) {
1009 continue;
1010 }
1011 found.push(Unattributed {
1012 path: path
1013 .strip_prefix(store)
1014 .unwrap_or(&path)
1015 .display()
1016 .to_string(),
1017 bytes: size_of(&path),
1018 });
1019 }
1020 }
1021 found.sort_by_key(|entry| std::cmp::Reverse(entry.bytes));
1022 found
1023}
1024
1025/// State under the store root that no pinned digest re-obtains.
1026///
1027/// A `rootfs` base is rebuilt from its image; a `snapshot` or a `clone_base` is
1028/// the state of a box somebody ran. ADR-0014 v1.6's permission does not extend to
1029/// it, so it is listed rather than cleared.
1030fn preserved(index: &Index) -> Vec<Preserved> {
1031 index
1032 .bases
1033 .iter()
1034 .filter(|base| base.kind != "rootfs")
1035 .map(|base| Preserved {
1036 path: base.path.display().to_string(),
1037 reason: format!(
1038 "a `{}` base disk is the state of a box that ran, which no digest re-obtains",
1039 base.kind
1040 ),
1041 })
1042 .chain(index.escaped.iter().map(|path| {
1043 Preserved {
1044 path: path.display().to_string(),
1045 reason: "the index names this base disk outside the store root, so nothing \
1046 here measures, lists or removes it"
1047 .to_owned(),
1048 }
1049 }))
1050 .collect()
1051}
1052
1053/// The paths [`preserved`] names, as a set.
1054///
1055/// Folded into `claimed` before [`unattributed`] runs, so that state a digest
1056/// does not re-obtain is never reported as spare and never reaches the doomed
1057/// set. Nobody references a snapshot's base disk, which is exactly why an
1058/// unclaimed-means-spare rule would delete it.
1059fn preserved_paths(index: &Index) -> BTreeSet<PathBuf> {
1060 index
1061 .bases
1062 .iter()
1063 .filter(|base| base.kind != "rootfs")
1064 .map(|base| base.path.clone())
1065 .collect()
1066}
1067
1068// ---------------------------------------------------------------------------
1069// clear
1070// ---------------------------------------------------------------------------
1071
1072/// Split the index into the rows a scope drops and the rows it keeps.
1073///
1074/// Returns them in that order — doomed first — because that is the order the two
1075/// are used in, and a pair whose halves can be swapped by a careless edit is a
1076/// set difference computed the wrong way round.
1077fn select<'a>(
1078 index: &'a Index,
1079 scope: &Scope,
1080) -> Result<(Vec<&'a IndexRow>, Vec<&'a IndexRow>), StoreError> {
1081 match scope {
1082 Scope::Everything => Ok((index.images.iter().collect(), Vec::new())),
1083 Scope::Image(reference) => {
1084 if !index.images.iter().any(|row| &row.reference == reference) {
1085 return Err(StoreError::UnknownImage {
1086 reference: reference.clone(),
1087 known: index
1088 .images
1089 .iter()
1090 .map(|row| row.reference.as_str())
1091 .collect::<Vec<_>>()
1092 .join(", "),
1093 });
1094 }
1095 Ok(index
1096 .images
1097 .iter()
1098 .partition(|row| &row.reference == reference))
1099 }
1100 }
1101}
1102
1103/// Every path a set of index rows references, including the index manifests they
1104/// resolve through.
1105///
1106/// The other half of the set difference, and the reason it is a named function:
1107/// applied to the **surviving** rows it is the retained set, and anything not in
1108/// it is what may go. Computing it from the doomed rows instead is the walk this
1109/// module's documentation says a deletion must never be.
1110fn paths_for(store: &Path, rows: &[&IndexRow], bases: &[BaseRow]) -> BTreeSet<PathBuf> {
1111 rows.iter()
1112 .flat_map(|row| {
1113 objects_for(store, row, bases)
1114 .all()
1115 .cloned()
1116 .collect::<Vec<_>>()
1117 })
1118 .chain(index_manifests_for(
1119 store,
1120 &rows.iter().map(|row| row.manifest_digest.clone()).collect(),
1121 ))
1122 .collect()
1123}
1124
1125/// What a [`clear`] would remove, without removing it.
1126///
1127/// Separate from [`clear`] so the set difference has one implementation and the
1128/// tests can assert on the doomed set directly rather than inferring it from what
1129/// survived a deletion.
1130///
1131/// # Errors
1132///
1133/// [`StoreError::UnknownImage`] when a per-image scope names a reference the index
1134/// does not hold, [`StoreError::LiveBoxes`] when a box is registered,
1135/// [`StoreError::UnrecognisedEntry`] when the store root holds something this
1136/// module cannot classify, [`StoreError::BaseOutsideStore`] when a base-disk row
1137/// points outside the store, and [`StoreError::Index`] when the index cannot be
1138/// read.
1139pub fn plan(asset_root: &Path, scope: &Scope) -> Result<(ClearReport, Vec<PathBuf>), StoreError> {
1140 let store = store_root(asset_root);
1141 let mut report = ClearReport {
1142 schema: SANDBOX_CLEAR_SCHEMA,
1143 scope: "machine",
1144 store: store.display().to_string(),
1145 requested: scope.as_str().to_owned(),
1146 applied: false,
1147 removed: Vec::new(),
1148 removed_unattributed: Vec::new(),
1149 freed_bytes: 0,
1150 store_bytes_before: 0,
1151 store_bytes_after: 0,
1152 retained: Vec::new(),
1153 preserved: Vec::new(),
1154 };
1155 if !store.is_dir() {
1156 return Ok((report, Vec::new()));
1157 }
1158
1159 let index = read_index(&store)?;
1160 if index.boxes > 0 {
1161 return Err(StoreError::LiveBoxes { boxes: index.boxes });
1162 }
1163 guard_entries(&store)?;
1164 guard_bases(&store, &index)?;
1165
1166 let (doomed_rows, surviving_rows) = select(&index, scope)?;
1167 let retained = paths_for(&store, &surviving_rows, &index.bases);
1168
1169 report.store_bytes_before = size_of(&store);
1170 report.preserved = preserved(&index);
1171
1172 let mut doomed: Vec<PathBuf> = Vec::new();
1173 for row in &doomed_rows {
1174 let objects = objects_for(&store, row, &index.bases);
1175 let mine: Vec<PathBuf> = objects
1176 .all()
1177 .filter(|path| !retained.contains(*path))
1178 .filter(|path| path.exists())
1179 .cloned()
1180 .collect();
1181 let freed = mine.iter().map(|path| size_of(path)).sum();
1182 report.removed.push(RemovedImage {
1183 reference: row.reference.clone(),
1184 freed_bytes: freed,
1185 objects_removed: mine.len(),
1186 });
1187 report.freed_bytes += freed;
1188 doomed.extend(mine);
1189 }
1190
1191 // The index manifest a doomed image resolved through goes with it, unless a
1192 // survivor resolves through the same one.
1193 for path in index_manifests_for(
1194 &store,
1195 &doomed_rows
1196 .iter()
1197 .map(|row| row.manifest_digest.clone())
1198 .collect(),
1199 ) {
1200 if !retained.contains(&path) && path.exists() {
1201 report.freed_bytes += size_of(&path);
1202 doomed.push(path);
1203 }
1204 }
1205
1206 // Unattributed bytes are only ever in scope for `everything`. A per-image
1207 // request has no evidence they belong to the image it named, and this module
1208 // does not delete on a guess.
1209 if matches!(scope, Scope::Everything) {
1210 let claimed: BTreeSet<PathBuf> = doomed
1211 .iter()
1212 .cloned()
1213 .chain(preserved_paths(&index))
1214 .collect();
1215 report.removed_unattributed = unattributed(&store, &claimed);
1216 for entry in &report.removed_unattributed {
1217 report.freed_bytes += entry.bytes;
1218 doomed.push(store.join(&entry.path));
1219 }
1220 }
1221
1222 doomed.sort();
1223 doomed.dedup();
1224 report.store_bytes_after = report.store_bytes_before;
1225 Ok((report, doomed))
1226}
1227
1228/// Remove what [`plan`] named, then re-check every surviving image against the
1229/// filesystem.
1230///
1231/// # Errors
1232///
1233/// Everything [`plan`] can return, plus [`StoreError::Io`] if a removal or the
1234/// index update fails.
1235pub fn clear(asset_root: &Path, scope: &Scope) -> Result<ClearReport, StoreError> {
1236 let store = store_root(asset_root);
1237 let (mut report, doomed) = plan(asset_root, scope)?;
1238 if !store.is_dir() {
1239 report.applied = true;
1240 return Ok(report);
1241 }
1242
1243 for path in &doomed {
1244 remove(path)?;
1245 }
1246 // The rows go with the blobs. Leaving them behind makes `status` report images
1247 // whose bytes are gone, which is issue #433's fourth trap and reads as a
1248 // corrupt store rather than a cleared one.
1249 forget(&store, &report.removed)?;
1250
1251 report.applied = true;
1252 report.store_bytes_after = size_of(&store);
1253 report.retained = verify(&store)?;
1254 Ok(report)
1255}
1256
1257/// Delete a file or a directory tree.
1258fn remove(path: &Path) -> Result<(), StoreError> {
1259 let metadata = match std::fs::symlink_metadata(path) {
1260 Ok(metadata) => metadata,
1261 Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(()),
1262 Err(error) => {
1263 return Err(StoreError::Io {
1264 action: "inspecting",
1265 path: path.display().to_string(),
1266 message: error.to_string(),
1267 });
1268 }
1269 };
1270 let outcome = if metadata.is_dir() {
1271 std::fs::remove_dir_all(path)
1272 } else {
1273 std::fs::remove_file(path)
1274 };
1275 outcome.map_err(|error| StoreError::Io {
1276 action: "removing",
1277 path: path.display().to_string(),
1278 message: error.to_string(),
1279 })
1280}
1281
1282/// Drop the index rows for what was removed.
1283///
1284/// Two different rules, because the two tables are keyed differently. An
1285/// `image_index` row is named by the request, so it goes by reference. A
1286/// `base_disk` row is named by nothing the request knows, so it goes **by whether
1287/// its file is still there** — which means the set difference that spared a base
1288/// disk a surviving image shares also spares its row, with no second chance to get
1289/// the difference wrong.
1290fn forget(store: &Path, removed: &[RemovedImage]) -> Result<(), StoreError> {
1291 let path = store.join(INDEX_DB);
1292 if !path.exists() {
1293 return Ok(());
1294 }
1295 let fail = |message: String| StoreError::Index {
1296 path: path.display().to_string(),
1297 message,
1298 };
1299 let db = rusqlite::Connection::open(&path).map_err(|error| fail(error.to_string()))?;
1300 // `IMMEDIATE` takes the write lock up front, so a concurrent `boxlite` pull
1301 // fails to start rather than interleaving with this. It is not the `flock`
1302 // that crate holds — see `StoreError::LiveBoxes` for why this crate cannot
1303 // take that one — and it is the strongest guard available without `unsafe`.
1304 db.execute_batch("BEGIN IMMEDIATE")
1305 .map_err(|error| fail(error.to_string()))?;
1306 for image in removed {
1307 db.execute(
1308 "DELETE FROM image_index WHERE reference = ?1",
1309 [&image.reference],
1310 )
1311 .map_err(|error| fail(error.to_string()))?;
1312 }
1313 let orphaned: Vec<String> = {
1314 let mut statement = db
1315 .prepare("SELECT base_path FROM base_disk WHERE kind = 'rootfs'")
1316 .map_err(|error| fail(error.to_string()))?;
1317 let rows = statement
1318 .query_map([], |row| row.get::<_, String>(0))
1319 .map_err(|error| fail(error.to_string()))?;
1320 rows.filter_map(Result::ok)
1321 .filter(|base_path| !Path::new(base_path).exists())
1322 .collect()
1323 };
1324 for base_path in &orphaned {
1325 db.execute(
1326 "DELETE FROM base_disk WHERE kind = 'rootfs' AND base_path = ?1",
1327 [base_path],
1328 )
1329 .map_err(|error| fail(error.to_string()))?;
1330 }
1331 db.execute_batch("COMMIT")
1332 .map_err(|error| fail(error.to_string()))?;
1333 Ok(())
1334}
1335
1336/// Re-resolve every image the index still holds against the filesystem.
1337fn verify(store: &Path) -> Result<Vec<VerifiedImage>, StoreError> {
1338 let index = read_index(store)?;
1339 Ok(index
1340 .images
1341 .iter()
1342 .map(|row| {
1343 let objects = objects_for(store, row, &index.bases);
1344 let tally = Objects {
1345 expected: objects.pulled.len(),
1346 present: objects.pulled.iter().filter(|path| path.exists()).count(),
1347 };
1348 VerifiedImage {
1349 reference: row.reference.clone(),
1350 objects: tally,
1351 complete: tally.complete(),
1352 }
1353 })
1354 .collect())
1355}
1356
1357/// Refuse if the store root holds something this module cannot classify.
1358fn guard_entries(store: &Path) -> Result<(), StoreError> {
1359 let entries = std::fs::read_dir(store).map_err(|error| StoreError::Io {
1360 action: "reading",
1361 path: store.display().to_string(),
1362 message: error.to_string(),
1363 })?;
1364 for entry in entries.flatten() {
1365 let name = entry.file_name().to_string_lossy().into_owned();
1366 if !KNOWN_ENTRIES.contains(&name.as_str()) {
1367 return Err(StoreError::UnrecognisedEntry { entry: name });
1368 }
1369 }
1370 let images = store.join("images");
1371 if !images.is_dir() {
1372 return Ok(());
1373 }
1374 let entries = std::fs::read_dir(&images).map_err(|error| StoreError::Io {
1375 action: "reading",
1376 path: images.display().to_string(),
1377 message: error.to_string(),
1378 })?;
1379 for entry in entries.flatten() {
1380 let name = entry.file_name().to_string_lossy().into_owned();
1381 if !IMAGE_DIRS.contains(&name.as_str()) {
1382 return Err(StoreError::UnrecognisedEntry {
1383 entry: format!("images/{name}"),
1384 });
1385 }
1386 }
1387 Ok(())
1388}
1389
1390/// Refuse if any base-disk row points outside the store root.
1391///
1392/// [`read_index`] has already made those rows unreachable, so this cannot be what
1393/// prevents a deletion outside the root — it is what makes the situation
1394/// *visible* instead of quietly halving the store's inventory. The usual cause is
1395/// a store that has been moved, where every row is stale at once and the honest
1396/// answer is to say so rather than to clear what is left.
1397fn guard_bases(store: &Path, index: &Index) -> Result<(), StoreError> {
1398 if let Some(path) = index.escaped.first() {
1399 return Err(StoreError::BaseOutsideStore {
1400 path: path.display().to_string(),
1401 root: store.display().to_string(),
1402 });
1403 }
1404 Ok(())
1405}
1406
1407#[cfg(test)]
1408mod tests {
1409 use super::{
1410 Attribution, IMAGE_DIRS, INDEX_DB, SANDBOX_STORE_DIR, Scope, StoreError, blob_name, clear,
1411 image_digest, plan, status,
1412 };
1413 use std::path::PathBuf;
1414
1415 /// The three `boxlite` 0.9.7 tables this module reads, declared as that crate
1416 /// declares them.
1417 ///
1418 /// A restatement, and deliberately a literal one: issue #433's hand-clearing
1419 /// notes describe an `images` table and a `disk-images/` directory, and the
1420 /// live schema is `image_index` / `base_disk` / `base_disk_ref` / `snapshot`.
1421 /// A procedure written against a remembered schema goes stale silently, so
1422 /// what the fixture builds is what was read off the store rather than what
1423 /// anyone recalled about it.
1424 const SCHEMA: &str = "
1425 CREATE TABLE image_index (
1426 reference TEXT PRIMARY KEY NOT NULL,
1427 manifest_digest TEXT NOT NULL,
1428 config_digest TEXT NOT NULL,
1429 layers TEXT NOT NULL,
1430 cached_at TEXT NOT NULL,
1431 complete INTEGER NOT NULL DEFAULT 0
1432 );
1433 CREATE TABLE base_disk (
1434 id TEXT PRIMARY KEY NOT NULL,
1435 source_box_id TEXT NOT NULL,
1436 name TEXT,
1437 kind TEXT NOT NULL CHECK(kind IN ('snapshot', 'clone_base', 'rootfs')),
1438 base_path TEXT NOT NULL,
1439 created_at INTEGER NOT NULL,
1440 json TEXT NOT NULL,
1441 UNIQUE(source_box_id, name)
1442 );
1443 CREATE TABLE box_config (
1444 id TEXT PRIMARY KEY NOT NULL,
1445 name TEXT UNIQUE,
1446 created_at INTEGER NOT NULL,
1447 json TEXT NOT NULL
1448 );
1449 ";
1450
1451 /// A plausible digest for a seed word, so fixtures read like the real store.
1452 fn digest(seed: &str) -> String {
1453 format!("sha256:{}", crate::sha256_hex(seed.as_bytes()))
1454 }
1455
1456 /// An asset root holding a sandbox store, built object by object.
1457 struct Fixture {
1458 root: PathBuf,
1459 }
1460
1461 impl Fixture {
1462 fn new(name: &str) -> Self {
1463 let root = std::env::temp_dir()
1464 .join(format!("rto-exec-sandbox-{name}-{}", std::process::id()));
1465 let _ = std::fs::remove_dir_all(&root);
1466 let store = root.join(SANDBOX_STORE_DIR);
1467 for dir in IMAGE_DIRS {
1468 std::fs::create_dir_all(store.join("images").join(dir)).expect("image dir");
1469 }
1470 std::fs::create_dir_all(store.join("bases")).expect("bases dir");
1471 std::fs::create_dir_all(store.join("db")).expect("db dir");
1472 std::fs::write(store.join(".lock"), []).expect("lock file");
1473 let db = rusqlite::Connection::open(store.join(INDEX_DB)).expect("open index");
1474 db.execute_batch(SCHEMA).expect("index schema");
1475 Self { root }
1476 }
1477
1478 fn store(&self) -> PathBuf {
1479 self.root.join(SANDBOX_STORE_DIR)
1480 }
1481
1482 fn db(&self) -> rusqlite::Connection {
1483 rusqlite::Connection::open(self.store().join(INDEX_DB)).expect("open index")
1484 }
1485
1486 fn write(&self, relative: &str, bytes: usize) -> PathBuf {
1487 let path = self.store().join(relative);
1488 std::fs::create_dir_all(path.parent().expect("a parent")).expect("parent dir");
1489 std::fs::write(&path, vec![b'x'; bytes]).expect("write object");
1490 path
1491 }
1492
1493 /// Add an image: its index row, its pulled objects, its extracted trees
1494 /// and the disk image derived from its layer list.
1495 fn image(&self, reference: &str, layers: &[&str], layer_bytes: usize) {
1496 let manifest = digest(&format!("{reference} manifest"));
1497 let config = digest(&format!("{reference} config"));
1498 let digests: Vec<String> = layers.iter().map(|seed| digest(seed)).collect();
1499 self.db()
1500 .execute(
1501 "INSERT INTO image_index
1502 (reference, manifest_digest, config_digest, layers, cached_at, complete)
1503 VALUES (?1, ?2, ?3, ?4, ?5, 1)",
1504 rusqlite::params![
1505 reference,
1506 manifest,
1507 config,
1508 serde_json::to_string(&digests).expect("layer list"),
1509 "2026-08-19T00:00:00+00:00",
1510 ],
1511 )
1512 .expect("insert image row");
1513 self.write(
1514 &format!("images/manifests/{}.json", blob_name(&manifest)),
1515 64,
1516 );
1517 self.write(&format!("images/configs/{}.json", blob_name(&config)), 32);
1518 for layer in &digests {
1519 self.write(
1520 &format!("images/layers/{}.tar.gz", blob_name(layer)),
1521 layer_bytes,
1522 );
1523 self.write(
1524 &format!("images/extracted/{}/rootfs", blob_name(layer)),
1525 layer_bytes,
1526 );
1527 }
1528 self.write(
1529 &format!(
1530 "images/disk-images/{}.ext4",
1531 blob_name(&image_digest(&digests))
1532 ),
1533 layer_bytes * 4,
1534 );
1535 }
1536
1537 /// The layer tarball a seed word resolves to.
1538 fn layer(&self, seed: &str) -> PathBuf {
1539 self.store()
1540 .join("images/layers")
1541 .join(format!("{}.tar.gz", blob_name(&digest(seed))))
1542 }
1543
1544 /// Add a `base_disk` row and the file it points at.
1545 fn base(&self, id: &str, kind: &str, name: &str) -> PathBuf {
1546 let path = self.write(&format!("bases/{id}.ext4"), 512);
1547 self.db()
1548 .execute(
1549 "INSERT INTO base_disk
1550 (id, source_box_id, name, kind, base_path, created_at, json)
1551 VALUES (?1, '__global__', ?2, ?3, ?4, 0, '{}')",
1552 rusqlite::params![id, name, kind, path.display().to_string()],
1553 )
1554 .expect("insert base row");
1555 path
1556 }
1557 }
1558
1559 /// The disk-image filename is **derived**, and the vector is the live store.
1560 ///
1561 /// `boxlite` keys `images/disk-images/*.ext4` by the SHA-256 of the layer
1562 /// digest strings concatenated in manifest order. Issue #433's hand-clearing
1563 /// notes record that no such association existed and fell back to **mtime**,
1564 /// with the instruction that a shipped `clear` must not. This is the
1565 /// assertion that it does not have to: both vectors are read off the store
1566 /// measured for that issue, and `cimg/rust` is the one that names a digest
1567 /// twice — so a de-duplicating or re-ordering derivation fails here rather
1568 /// than in a wrong deletion.
1569 #[test]
1570 fn the_disk_image_filename_is_derived_from_the_layer_list() {
1571 let debian = [
1572 "sha256:0f5d7465a5bb9d419f60c93d126a161286c73a1ede4a8b2e46bd5e7ad5782cc7".to_owned(),
1573 ];
1574 assert_eq!(
1575 image_digest(&debian),
1576 "sha256:2674b856eab71e6d70f5d8ad573394d1b90f40da02593e3af3a31c17b8de1d97",
1577 "the derivation no longer reproduces the disk image the live store holds for \
1578 docker.io/library/debian:bookworm-slim"
1579 );
1580
1581 let cimg: Vec<String> = [
1582 "c36472b3458398be28ecbfebbaac44143c040eae73411baded48a22060d3055b",
1583 "fee4d731b9208f65a65b57345c4945de0d8eccf9a9f8729e796be8911bd3131c",
1584 "fec6be0b4b4a6668684b8cc97d59c44998ed49004a5e18954caa3f58986549a6",
1585 "943b99e461484cf70776207df97a17783fc424cfeafd5e1bdffab309f42fe84f",
1586 "4dc664574997cda0756a223b9e39e9c4cac313e72dbd59adef0fa723ac8ffc5f",
1587 "cd344ce4edc31b84f799cfd1ff435b61345534535f474fcb8a7f2e9d2ddb209d",
1588 "ab2260fc0eee2ac435fe045b63e2fc28c19cf56cb707101e8eb77601bcf7cdb3",
1589 "4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1",
1590 "5b96641132bf37840e483d28ac60942c3b7b26c2382322c8fa94e62e83b86523",
1591 "4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1",
1592 ]
1593 .iter()
1594 .map(|hex| format!("sha256:{hex}"))
1595 .collect();
1596 assert_eq!(
1597 image_digest(&cimg),
1598 "sha256:b038ce43bcc84d230823ec62558e9a5926f9057206e3eb5ec704da92677fcc0d",
1599 "the derivation no longer reproduces the disk image the live store holds for \
1600 docker.io/cimg/rust, whose layer list names one digest twice"
1601 );
1602 }
1603
1604 /// The index writes `sha256:`; the filesystem writes `sha256-`.
1605 ///
1606 /// One character, and it is why issue #433's first hand-clearing pass matched
1607 /// nothing at all.
1608 #[test]
1609 fn the_on_disk_name_is_not_the_index_digest() {
1610 assert_eq!(blob_name("sha256:abc123"), "sha256-abc123");
1611 }
1612
1613 /// Dropping one image must not take a layer another image still references.
1614 ///
1615 /// The correctness bug this module exists to prevent, and it does not present
1616 /// as an error: it presents as a **surviving image whose blobs are gone**,
1617 /// discovered on some later run. So the assertion is made twice — the shared
1618 /// layer is absent from the doomed set, and the survivor is verified blob by
1619 /// blob after the deletion actually happened.
1620 #[test]
1621 fn a_layer_two_images_share_survives_dropping_one_of_them() {
1622 let fixture = Fixture::new("shared-layer");
1623 fixture.image("registry/a:1", &["common", "only-a"], 4096);
1624 fixture.image("registry/b:1", &["common", "only-b"], 4096);
1625
1626 let scope = Scope::Image("registry/a:1".to_owned());
1627 let (_, doomed) = plan(&fixture.root, &scope).expect("plan");
1628 assert!(
1629 !doomed.contains(&fixture.layer("common")),
1630 "a layer `registry/b:1` still references was put in the doomed set"
1631 );
1632 assert!(
1633 doomed.contains(&fixture.layer("only-a")),
1634 "the layer only the dropped image references was not in the doomed set"
1635 );
1636
1637 let report = clear(&fixture.root, &scope).expect("clear");
1638 assert!(
1639 fixture.layer("common").exists(),
1640 "the shared layer was deleted, so `registry/b:1` is now broken"
1641 );
1642 assert!(!fixture.layer("only-a").exists());
1643 assert_eq!(report.retained.len(), 1);
1644 assert!(
1645 report.survivors_intact(),
1646 "a surviving image is incomplete after the clear: {:?}",
1647 report.retained
1648 );
1649 assert_eq!(report.retained[0].reference, "registry/b:1");
1650 assert_eq!(report.retained[0].objects.expected, 4);
1651 assert_eq!(report.retained[0].objects.present, 4);
1652 }
1653
1654 /// A shared blob is reported as shared, rather than promised back.
1655 ///
1656 /// `total` is what the image references and `exclusive` is what dropping it
1657 /// alone would free. Reporting only the total would offer bytes that a
1658 /// surviving image keeps, which is a `clear` whose number nobody can check.
1659 #[test]
1660 fn the_status_row_separates_what_an_image_references_from_what_it_would_free() {
1661 let fixture = Fixture::new("exclusive-bytes");
1662 fixture.image("registry/a:1", &["common", "only-a"], 4096);
1663 fixture.image("registry/b:1", &["common", "only-b"], 4096);
1664
1665 let report = status(&fixture.root).expect("status");
1666 let image = report
1667 .images
1668 .iter()
1669 .find(|image| image.reference == "registry/a:1")
1670 .expect("the image is listed");
1671 assert!(
1672 image.bytes.exclusive < image.bytes.total,
1673 "a shared layer was counted as reclaimable: {:?}",
1674 image.bytes
1675 );
1676 assert_eq!(image.objects.expected, 4, "manifest, config and two layers");
1677 assert_eq!(image.objects.present, 4);
1678 }
1679
1680 /// `everything` empties the store and the index together, and the two byte
1681 /// figures agree to within the index itself.
1682 ///
1683 /// The rows go with the blobs. Leaving a row behind reports an image whose
1684 /// bytes are gone, which reads as a corrupt store rather than a cleared one —
1685 /// issue #433's fourth trap.
1686 ///
1687 /// It also holds [`super::ClearReport::measured_freed_bytes`]'s claim: the
1688 /// only thing here that can change size without having been removed is the
1689 /// `SQLite` index, which frees pages inside its file rather than shrinking it.
1690 #[test]
1691 fn the_accounted_bytes_and_the_measured_bytes_differ_only_by_the_index() {
1692 let fixture = Fixture::new("everything");
1693 fixture.image("registry/a:1", &["common", "only-a"], 4096);
1694 fixture.image("registry/b:1", &["common", "only-b"], 4096);
1695
1696 let before = status(&fixture.root).expect("status").total_bytes;
1697 let report = clear(&fixture.root, &Scope::Everything).expect("clear");
1698 assert_eq!(report.removed.len(), 2);
1699 assert!(report.retained.is_empty());
1700 assert_eq!(report.store_bytes_before, before);
1701 assert!(
1702 report.store_bytes_after < report.store_bytes_before,
1703 "the store did not shrink"
1704 );
1705 let index = super::size_of(&fixture.store().join("db"));
1706 assert!(
1707 report.freed_bytes.abs_diff(report.measured_freed_bytes()) <= index,
1708 "the accounted bytes ({}) and the bytes the filesystem gave back ({}) differ by \
1709 more than the index ({index}) — in either direction, which is the only thing in \
1710 the store that can change size without having been removed",
1711 report.freed_bytes,
1712 report.measured_freed_bytes()
1713 );
1714
1715 let after = status(&fixture.root).expect("status");
1716 assert!(
1717 after.images.is_empty(),
1718 "the index still reports images whose blobs are gone: {:?}",
1719 after.images
1720 );
1721 }
1722
1723 /// An index manifest a survivor resolves through is kept.
1724 ///
1725 /// A tag reference — `debian:bookworm-slim` in the live store — records only
1726 /// the platform manifest it resolved to, never the index digest above it. A
1727 /// rule of "delete every manifest file no row names" would therefore take a
1728 /// surviving image's index with it, so the association is read out of the
1729 /// files themselves.
1730 #[test]
1731 fn an_index_manifest_a_survivor_resolves_through_is_kept() {
1732 let fixture = Fixture::new("index-manifest");
1733 fixture.image("registry/a:1", &["only-a"], 4096);
1734 fixture.image("registry/b:1", &["only-b"], 4096);
1735
1736 let survivor = digest("registry/b:1 manifest");
1737 let dropped = digest("registry/a:1 manifest");
1738 let keep = fixture.write("images/manifests/sha256-keep.json", 0);
1739 std::fs::write(
1740 &keep,
1741 serde_json::json!({ "manifests": [{ "digest": survivor }] }).to_string(),
1742 )
1743 .expect("write index manifest");
1744 let go = fixture.write("images/manifests/sha256-go.json", 0);
1745 std::fs::write(
1746 &go,
1747 serde_json::json!({ "manifests": [{ "digest": dropped }] }).to_string(),
1748 )
1749 .expect("write index manifest");
1750
1751 // The load-bearing assertion, and it is this one rather than "the file is
1752 // still there after a per-image clear" — nothing in a per-image clear
1753 // could ever reach it, so that assertion would pass with the association
1754 // removed entirely. What the association actually decides is whether these
1755 // files are **claimed**: an index manifest nobody claims is spare, and
1756 // `everything` removes spare bytes.
1757 let before = status(&fixture.root).expect("status");
1758 assert!(
1759 before.unattributed.is_empty(),
1760 "an index manifest a cached image resolves through was reported as \
1761 unattributed, which is one `--everything` away from being deleted while its \
1762 image survives: {:?}",
1763 before.unattributed
1764 );
1765
1766 clear(&fixture.root, &Scope::Image("registry/a:1".to_owned())).expect("clear");
1767 assert!(!go.exists(), "the dropped image's index manifest was kept");
1768 let after = status(&fixture.root).expect("status");
1769 assert!(
1770 after.unattributed.is_empty(),
1771 "the survivor's index manifest stopped being attributed once the other image \
1772 was dropped: {:?}",
1773 after.unattributed
1774 );
1775 assert!(
1776 keep.exists(),
1777 "the index manifest `registry/b:1` resolves through was deleted"
1778 );
1779 }
1780
1781 /// Something under the store root this module cannot classify stops the clear.
1782 ///
1783 /// ADR-0014 v1.6's limit, enforced rather than trusted: the verb may drop what
1784 /// a pinned digest re-obtains and nothing else, and turning up in a cache is
1785 /// not evidence of being re-obtainable. Nothing else is cleared either — a
1786 /// partial clear alongside a refusal is the worst of both.
1787 #[test]
1788 fn an_unrecognised_entry_stops_the_clear_without_removing_anything() {
1789 let fixture = Fixture::new("unrecognised");
1790 fixture.image("registry/a:1", &["only-a"], 4096);
1791 std::fs::create_dir_all(fixture.store().join("provenance")).expect("mystery dir");
1792
1793 let error = clear(&fixture.root, &Scope::Everything)
1794 .expect_err("an entry this module cannot classify must stop the clear");
1795 assert!(
1796 matches!(&error, StoreError::UnrecognisedEntry { entry } if entry == "provenance"),
1797 "expected the unrecognised entry to be named, got: {error}"
1798 );
1799 assert!(
1800 fixture.layer("only-a").exists(),
1801 "the refusal removed objects on its way out"
1802 );
1803 }
1804
1805 /// A `base_disk` row cannot aim a deletion outside the store root.
1806 ///
1807 /// `base_path` is an absolute path recorded when the base was built, so it is
1808 /// data rather than a derivation — and data can name anywhere, including the
1809 /// repository store this verb must never reach.
1810 #[test]
1811 fn a_base_disk_row_pointing_outside_the_store_is_refused() {
1812 let fixture = Fixture::new("escape");
1813 fixture.image("registry/a:1", &["only-a"], 4096);
1814 let outside = fixture.root.join("graph.db");
1815 std::fs::write(&outside, b"not re-obtainable").expect("write");
1816 fixture
1817 .db()
1818 .execute(
1819 "INSERT INTO base_disk
1820 (id, source_box_id, name, kind, base_path, created_at, json)
1821 VALUES ('esc', '__global__', 'escapee', 'rootfs', ?1, 0, '{}')",
1822 rusqlite::params![outside.display().to_string()],
1823 )
1824 .expect("insert base row");
1825
1826 let error = clear(&fixture.root, &Scope::Everything)
1827 .expect_err("a base disk outside the store root must stop the clear");
1828 assert!(
1829 matches!(&error, StoreError::BaseOutsideStore { path, .. } if path.contains("graph.db")),
1830 "expected the escaping path to be named, got: {error}"
1831 );
1832 assert!(outside.exists(), "the clear reached outside the store root");
1833
1834 // And `status` does not reach it either. The containment is at the point
1835 // the row is read, not at the point it is deleted, so a path outside the
1836 // root is never measured or listed — only named as something left alone.
1837 let report = status(&fixture.root).expect("status");
1838 assert!(
1839 report
1840 .preserved
1841 .iter()
1842 .any(|entry| entry.path.contains("graph.db")),
1843 "the escaping row was not named in the status document"
1844 );
1845 let image = &report.images[0];
1846 assert_eq!(
1847 image.bytes.base_disk, 0,
1848 "a file outside the store root was measured into an image's size"
1849 );
1850 assert!(!image.base_disk_built);
1851 }
1852
1853 /// The bytes reported are the bytes `du` reports, because the store is sparse.
1854 ///
1855 /// The ext4 disks are sparse by a margin nobody would call rounding: in the
1856 /// store measured for issue #433, `sha256-0a89fbeb….ext4` is 1.23 GiB long and
1857 /// occupies 1.09 GiB, and the base rootfs is 256 MiB long and occupies 110
1858 /// MiB. A `clear` that reported lengths would claim 9.1 GiB freed where `du`
1859 /// says 8.7 GiB — and the one number this verb exists to produce would be the
1860 /// one number in it that could not be checked.
1861 #[cfg(unix)]
1862 #[test]
1863 fn a_sparse_disk_image_is_counted_by_what_it_occupies() {
1864 use std::io::{Seek as _, Write as _};
1865
1866 let fixture = Fixture::new("sparse");
1867 let path = fixture.write("images/disk-images/sha256-sparse.ext4", 0);
1868 let mut file = std::fs::File::create(&path).expect("create");
1869 file.seek(std::io::SeekFrom::Start(64 << 20)).expect("seek");
1870 file.write_all(b"end").expect("write");
1871 drop(file);
1872
1873 let apparent = std::fs::metadata(&path).expect("metadata").len();
1874 let occupied = super::size_of(&path);
1875 assert!(
1876 apparent > 64 << 20,
1877 "the fixture file is not long enough to be worth measuring"
1878 );
1879 assert!(
1880 occupied < apparent,
1881 "a sparse file was counted by its length ({apparent}) rather than by what it \
1882 occupies ({occupied}), so a clear would over-report what it freed"
1883 );
1884 }
1885
1886 /// A registered box blocks the clear.
1887 ///
1888 /// `boxlite` holds an exclusive `flock` on `<store>/.lock` for a runtime's
1889 /// lifetime, which this crate cannot take back — `unsafe_code = "forbid"`
1890 /// rules out the call that acquires it. So the guard is the evidence such a
1891 /// lock would have protected, and it is checked rather than assumed absent.
1892 #[test]
1893 fn a_registered_box_blocks_the_clear() {
1894 let fixture = Fixture::new("live-box");
1895 fixture.image("registry/a:1", &["only-a"], 4096);
1896 fixture
1897 .db()
1898 .execute(
1899 "INSERT INTO box_config (id, name, created_at, json)
1900 VALUES ('box1', 'running', 0, '{}')",
1901 [],
1902 )
1903 .expect("insert box row");
1904
1905 let error = clear(&fixture.root, &Scope::Everything)
1906 .expect_err("a registered box must stop the clear");
1907 assert!(
1908 matches!(error, StoreError::LiveBoxes { boxes: 1 }),
1909 "expected a live-box refusal, got: {error}"
1910 );
1911 assert!(fixture.layer("only-a").exists());
1912 }
1913
1914 /// A snapshot base disk is preserved, named, and never counted as spare.
1915 ///
1916 /// A `rootfs` base is rebuilt from its image; a `snapshot` is the state of a
1917 /// box somebody ran, and no digest re-obtains it. It is referenced by no
1918 /// image, which is exactly why an unclaimed-means-spare rule would delete it.
1919 #[test]
1920 fn a_snapshot_base_disk_is_preserved_rather_than_treated_as_spare() {
1921 let fixture = Fixture::new("snapshot");
1922 fixture.image("registry/a:1", &["only-a"], 4096);
1923 let snapshot = fixture.base("snap1", "snapshot", "a-snapshot");
1924
1925 let before = status(&fixture.root).expect("status");
1926 assert!(
1927 !before
1928 .unattributed
1929 .iter()
1930 .any(|entry| snapshot.ends_with(&entry.path)),
1931 "a snapshot base disk was reported as unattributed bytes"
1932 );
1933 assert_eq!(before.preserved.len(), 1);
1934
1935 let report = clear(&fixture.root, &Scope::Everything).expect("clear");
1936 assert!(
1937 snapshot.exists(),
1938 "the clear removed a snapshot, which no digest re-obtains"
1939 );
1940 assert_eq!(report.preserved.len(), 1);
1941 }
1942
1943 /// A per-image request that names nothing in the store says what is in it.
1944 ///
1945 /// The likely cause is a tag typed where the index holds a digest reference,
1946 /// so the listing is the way forward rather than a courtesy.
1947 #[test]
1948 fn an_unknown_reference_names_what_the_store_is_holding() {
1949 let fixture = Fixture::new("unknown-image");
1950 fixture.image("registry/a:1", &["only-a"], 4096);
1951
1952 let error = clear(&fixture.root, &Scope::Image("registry/a:2".to_owned()))
1953 .expect_err("a reference the store is not holding must be refused");
1954 assert!(
1955 matches!(&error, StoreError::UnknownImage { known, .. } if known == "registry/a:1"),
1956 "expected the cached references to be named, got: {error}"
1957 );
1958 }
1959
1960 /// Bytes no index row claims survive a per-image clear and go with everything.
1961 ///
1962 /// The live store had one: a 146 MB extracted layer tree whose tarball and
1963 /// index row are both gone. A per-image request has no evidence it belongs to
1964 /// the image it named, so it is reported rather than deleted on a guess.
1965 #[test]
1966 fn unattributed_bytes_survive_a_per_image_clear_and_go_with_everything() {
1967 let fixture = Fixture::new("unattributed");
1968 fixture.image("registry/a:1", &["only-a"], 4096);
1969 fixture.image("registry/b:1", &["only-b"], 4096);
1970 let orphan = fixture.write("images/extracted/sha256-orphan/rootfs", 8192);
1971
1972 let before = status(&fixture.root).expect("status");
1973 assert_eq!(before.attribution, Attribution::Partial);
1974 assert_eq!(before.unattributed.len(), 1);
1975 assert!(before.unattributed[0].path.ends_with("sha256-orphan"));
1976 // At least the tree it holds, and deliberately not an exact figure: this
1977 // entry is a **directory**, and a directory's own allocation is a property
1978 // of the filesystem rather than of the store. APFS reports none for one;
1979 // ext4 reports a 4 KiB block, so an `== 8192` here passed on the machine
1980 // this was written on and failed on CI.
1981 assert!(
1982 before.unattributed[0].bytes >= 8192,
1983 "the unattributed tree was measured as smaller than the file in it: {:?}",
1984 before.unattributed[0]
1985 );
1986
1987 clear(&fixture.root, &Scope::Image("registry/a:1".to_owned())).expect("clear");
1988 assert!(
1989 orphan.exists(),
1990 "a per-image clear deleted bytes it could not attribute to that image"
1991 );
1992
1993 let report = clear(&fixture.root, &Scope::Everything).expect("clear");
1994 assert!(
1995 !orphan.exists(),
1996 "`everything` left unattributed bytes behind"
1997 );
1998 assert_eq!(report.removed_unattributed.len(), 1);
1999 }
2000
2001 /// An absent store is an answer, not a failure.
2002 #[test]
2003 fn an_absent_store_is_reported_rather_than_failing() {
2004 let root =
2005 std::env::temp_dir().join(format!("rto-exec-sandbox-absent-{}", std::process::id()));
2006 let _ = std::fs::remove_dir_all(&root);
2007 let report = status(&root).expect("status");
2008 assert!(!report.present);
2009 assert!(report.images.is_empty());
2010 assert_eq!(report.total_bytes, 0);
2011
2012 let cleared = clear(&root, &Scope::Everything).expect("clear");
2013 assert_eq!(cleared.freed_bytes, 0);
2014 assert!(cleared.applied);
2015 }
2016
2017 /// This module and `boxlite.rs` name the same directory.
2018 ///
2019 /// They cannot share the constant: `boxlite.rs` is behind `exec-boxlite`, and
2020 /// making that feature a condition of *clearing* a store some earlier build
2021 /// filled is the shape of the bootstrap problem that moved provisioning off
2022 /// the backend features in the first place. So the two literals are checked
2023 /// against each other instead, by reading the source — the only way to compare
2024 /// a constant with a `join` argument.
2025 #[test]
2026 fn the_store_directory_is_the_one_boxlite_is_pointed_at() {
2027 let source = include_str!("boxlite.rs");
2028 let marker = "home_dir: assets_root.join(\"";
2029 let named: Vec<&str> = source
2030 .match_indices(marker)
2031 .map(|(at, _)| {
2032 source[at + marker.len()..]
2033 .split_once('"')
2034 .expect("a join argument is closed on the same line")
2035 .0
2036 })
2037 .collect();
2038 assert!(
2039 !named.is_empty(),
2040 "no `home_dir: assets_root.join(..)` was found to check against"
2041 );
2042 for directory in named {
2043 assert_eq!(
2044 directory, SANDBOX_STORE_DIR,
2045 "boxlite.rs points a runtime at `{directory}` and this module clears \
2046 `{SANDBOX_STORE_DIR}`"
2047 );
2048 }
2049 }
2050
2051 /// The document says whose store it is describing.
2052 ///
2053 /// One sandbox store per asset root, shared by every repository on the host —
2054 /// the same hazard `MachineScope` carries a `scope` field for, and the same
2055 /// remedy: a caller who selected a project must not read a machine-global
2056 /// figure as that project's.
2057 #[test]
2058 fn the_status_document_labels_its_scope_as_the_machine() {
2059 let fixture = Fixture::new("scope");
2060 fixture.image("registry/a:1", &["only-a"], 16);
2061 let document =
2062 serde_json::to_value(status(&fixture.root).expect("status")).expect("serialise");
2063 assert_eq!(document["scope"], "machine");
2064 assert_eq!(document["schema"], super::SANDBOX_STATUS_SCHEMA);
2065 assert!(
2066 document["store"]
2067 .as_str()
2068 .expect("a store path")
2069 .ends_with(SANDBOX_STORE_DIR)
2070 );
2071 }
2072}