rto_exec/sandbox_store.rs
1//! The **sandbox image store**: what it is holding, and dropping it safely.
2//!
3//! Ungated, like [`crate::assets`], and the argument is the same one that moved
4//! provisioning off the backend features: reclaiming the bytes a previous build
5//! cached must not require rebuilding with the backend that cached them. Nothing
6//! here executes anything — it reads an index, measures files, and removes what
7//! a pinned digest re-obtains (ADR-0014 v1.6).
8//!
9//! `roteiro security prefetch` obtains and `roteiro security status` reports.
10//! Nothing removed, so the store only grew — 2.9 GB when issue #433 was filed,
11//! 8.7 GB on the machine this module was written on, 12 GB after one afternoon
12//! of trying candidate builder images. ADR-0014 v1.6 gives provisioning its third
13//! verb, and this module is the half of it that knows what is on disk.
14//!
15//! # The store has no names in it, so deletion starts at the index
16//!
17//! Everything under `<asset-root>/boxlite-home/images/` is keyed by digest —
18//! `layers/`, `configs/`, `manifests/`, `extracted/`, `disk-images/` — and not one
19//! path carries an image's name. The name → digest mapping lives only in
20//! `db/boxlite.db`'s `image_index` table. **There is no filesystem-only route**:
21//! a walk can tell you the store is 8.5 GB and cannot tell you which image any of
22//! it belongs to.
23//!
24//! So [`status`] and [`clear`] both start by reading that table, and every path
25//! this module touches is derived from a row in it. A file it cannot derive is
26//! never deleted on a per-image request and is reported by name — see
27//! [`SandboxStatus::unattributed`].
28//!
29//! # Deletion is a set difference, never a walk
30//!
31//! Blobs are shared. Two images can share a base layer, and one image's layer
32//! list can name the same digest twice — `cimg/rust`'s does, in the store this
33//! was written against. Dropping image A must therefore remove only the digests
34//! **no surviving image references**, which is a set difference over the whole
35//! index rather than a walk of A's own object list.
36//!
37//! Getting this wrong is invisible until it isn't. The first pair of images with
38//! a common layer turns a naive per-image delete into a *broken surviving image*
39//! — not an error, not a warning, just a `security run` that fails much later
40//! with a missing blob. [`plan`] is the only place the difference is computed and
41//! [`clear`] cannot delete anything [`plan`] did not put in the doomed set.
42//!
43//! # The derived artifacts, and the association that was thought to be missing
44//!
45//! `images/disk-images/*.ext4` is the largest thing in the store — 3.7 GB of 8.7
46//! on this machine — and issue #433's hand-clearing notes record that nothing
47//! mapped an image to the disk image built from it, so they were separated by
48//! **mtime** and the note says a shipped `clear` must not do that.
49//!
50//! It does not have to. `boxlite`'s `ImageObject::compute_image_digest` keys a
51//! disk image by the SHA-256 of its layer digest strings concatenated in manifest
52//! order, and `ImageDiskManager::disk_path` writes it as `{digest}.ext4` with the
53//! `:` turned into `-`. That is [`image_digest`], it is computed from
54//! `image_index.layers` alone, and it reproduces all three filenames in the live
55//! store exactly. The base rootfs in `bases/` is keyed off the same value:
56//! `base_disk.name` is `{image_digest[..12]}-{guest_binary_hash[..12]}`.
57//!
58//! **It is a re-derivation of another crate's private function, so it is checked
59//! rather than trusted.** [`Attribution`] records whether every disk image on
60//! disk was claimed by some indexed image; when one is not, it becomes
61//! `unattributed` and a per-image `clear` leaves it alone. A `boxlite` upgrade
62//! that changed the key would show up as unattributed bytes in `status`, which is
63//! a visible wrong number rather than a silent wrong deletion.
64//!
65//! # What may be dropped, and the two things here that may not
66//!
67//! ADR-0014 v1.6's permission and its limit are one property: everything under
68//! the asset cache is re-obtainable from a pinned digest, so clearing costs time
69//! and never information — **and the verb may therefore never reach anything that
70//! is not**. Under this store root, two things are not:
71//!
72//! - **A `base_disk` row of kind `snapshot` or `clone_base`.** A `rootfs` base is
73//! rebuilt from the image; a snapshot is the state of a box somebody ran, and no
74//! digest re-obtains it. They are never deleted, and they are reported —
75//! [`ClearReport::preserved`].
76//! - **Anything under the store root this module does not recognise.** A new
77//! `boxlite` layout directory is not known to be re-obtainable just because it
78//! turned up in a cache, so [`clear`] refuses rather than guesses:
79//! [`StoreError::UnrecognisedEntry`].
80//!
81//! Nothing outside `<asset-root>/boxlite-home` is reachable from here at all. The
82//! findings layers, the memory records and `graph.db` live in the repository's
83//! store, which this module has no path to and does not link against.
84//!
85//! # Two prefixes that look like one
86//!
87//! The index stores `sha256:abc…`; the filesystem writes `sha256-abc…`. Issue
88//! #433's first hand-clearing pass matched nothing because of it. [`blob_name`]
89//! is the single translation, and every path in this module goes through it.
90//!
91//! @rto:0014
92//! @rto:0013
93
94use std::collections::{BTreeMap, BTreeSet};
95use std::path::{Path, PathBuf};
96
97use serde::Serialize;
98
99use crate::sha256_hex;
100
101/// Schema tag for the sandbox-store status document.
102pub const SANDBOX_STATUS_SCHEMA: &str = "roteiro.sandbox.status/v1";
103
104/// Schema tag for the sandbox-store clear document.
105pub const SANDBOX_CLEAR_SCHEMA: &str = "roteiro.sandbox.clear/v1";
106
107/// The sandbox store's directory under the asset cache root.
108///
109/// `boxlite.rs` spells the same literal where it builds a runtime's `home_dir`,
110/// and cannot share this constant without `exec-boxlite` becoming a condition of
111/// being able to *clear* a store a previous build filled. The two are held
112/// together by `the_store_directory_is_the_one_boxlite_is_pointed_at` instead.
113pub const SANDBOX_STORE_DIR: &str = "boxlite-home";
114
115/// The index `boxlite` keeps its name → digest mapping in, under the store root.
116const INDEX_DB: &str = "db/boxlite.db";
117
118/// Top-level entries [`clear`] knows the disposition of.
119///
120/// Anything else under the store root stops [`clear`] with
121/// [`StoreError::UnrecognisedEntry`], because "it appeared in a cache" is not
122/// evidence that a digest re-obtains it (ADR-0014 v1.6).
123const KNOWN_ENTRIES: &[&str] = &[".lock", "bases", "boxes", "db", "images", "locks", "tmp"];
124
125/// The digest-keyed object directories under `images/`.
126const IMAGE_DIRS: &[&str] = &["configs", "disk-images", "extracted", "layers", "manifests"];
127
128/// Turn an index digest (`sha256:abc…`) into the name the filesystem uses
129/// (`sha256-abc…`).
130///
131/// The single place that translation happens. It is one character and it is the
132/// reason issue #433's first hand-clearing pass matched nothing at all.
133#[must_use]
134pub fn blob_name(digest: &str) -> String {
135 digest.replace(':', "-")
136}
137
138/// The cache key `boxlite` derives a disk image and a base rootfs from: the
139/// SHA-256 of the layer digest strings, concatenated in manifest order.
140///
141/// A re-derivation of `boxlite`'s `ImageObject::compute_image_digest`, which is
142/// private to that crate. It is what removes the mtime heuristic issue #433's
143/// notes fell back to, and [`Attribution`] is what keeps it honest if `boxlite`
144/// ever changes it.
145///
146/// Duplicates are **not** removed and order is **not** normalised: this hashes the
147/// layer list as the manifest wrote it, because that is what the other side does.
148/// `cimg/rust` names one digest twice and still resolves to the filename on disk.
149#[must_use]
150pub fn image_digest(layers: &[String]) -> String {
151 let joined: String = layers.concat();
152 format!("sha256:{}", sha256_hex(joined.as_bytes()))
153}
154
155/// What went wrong, in terms of what to do about it.
156#[derive(Debug, thiserror::Error)]
157#[non_exhaustive]
158pub enum StoreError {
159 /// The index could not be read. Without it nothing in the store has a name,
160 /// so neither verb can proceed — see this module's documentation.
161 #[error("cannot read the sandbox image index at {path}: {message}")]
162 Index {
163 /// The database that could not be read.
164 path: String,
165 /// What the database layer said.
166 message: String,
167 },
168 /// A filesystem operation failed.
169 #[error("{action} {path}: {message}")]
170 Io {
171 /// What was being attempted, as a verb phrase.
172 action: &'static str,
173 /// The path it was attempted on.
174 path: String,
175 /// What the operating system said.
176 message: String,
177 },
178 /// A per-image request named an image the store is not holding.
179 ///
180 /// Carries what it *is* holding, because the likely cause is a tag written
181 /// where the index has a digest reference, and a listing is the way forward.
182 #[error("the sandbox store is not holding `{reference}`; it is holding: {known}")]
183 UnknownImage {
184 /// What was asked for.
185 reference: String,
186 /// The references the index does have, comma-separated.
187 known: String,
188 },
189 /// A box is registered in the store, so something may be using these bytes.
190 ///
191 /// `boxlite` takes an exclusive `flock` on `<store>/.lock` for the lifetime of
192 /// a runtime, which this crate cannot take back: `unsafe_code = "forbid"`
193 /// rules out the `libc::flock` call that acquires it. So the guard is the
194 /// evidence a *lock* would have protected — a registered box — and it is
195 /// checked rather than assumed absent.
196 #[error(
197 "the sandbox store has {boxes} registered box(es); \
198 stop them before clearing, or the bytes a running box is reading go away underneath it"
199 )]
200 LiveBoxes {
201 /// How many boxes are registered.
202 boxes: usize,
203 },
204 /// Something under the store root that this module does not recognise.
205 ///
206 /// ADR-0014 v1.6's limit, enforced rather than trusted: `clear` may drop what
207 /// a pinned digest re-obtains and may drop nothing else, and an unknown entry
208 /// is not known to be re-obtainable.
209 #[error(
210 "the sandbox store holds `{entry}`, which this version of Roteiro does not recognise; \
211 it will not be cleared, and nothing else was cleared either — \
212 report it on issue #433, because an entry a digest does not re-obtain does not belong here"
213 )]
214 UnrecognisedEntry {
215 /// The entry's name under the store root.
216 entry: String,
217 },
218 /// A `base_disk` row points outside the store root.
219 ///
220 /// `base_path` is an absolute path recorded when the base was built, so it is
221 /// data rather than a derivation, and data can name anywhere. This is the
222 /// check that a row cannot aim deletion at a path outside the asset cache.
223 #[error("the sandbox index has a base disk at {path}, which is outside the store root {root}")]
224 BaseOutsideStore {
225 /// Where the row pointed.
226 path: String,
227 /// The root it had to be under.
228 root: String,
229 },
230}
231
232/// Which images a [`clear`] is being asked for.
233///
234/// Two variants rather than an `Option<String>`, because ADR-0014 v1.6 requires
235/// that "clear this image" and "clear everything" be **different arguments** — a
236/// caller asking for one must not be able to receive the other by supplying
237/// nothing.
238#[derive(Debug, Clone, PartialEq, Eq)]
239pub enum Scope {
240 /// Every cached image, and the unattributed bytes alongside them.
241 Everything,
242 /// One image, by the reference the index holds it under. Anything a surviving
243 /// image still references stays.
244 Image(String),
245}
246
247impl Scope {
248 /// The token this serialises as, for a report that names what was asked for.
249 #[must_use]
250 pub fn as_str(&self) -> &str {
251 match self {
252 Self::Everything => "everything",
253 Self::Image(reference) => reference,
254 }
255 }
256}
257
258/// Whether the store's derived artifacts were all claimed by an indexed image.
259///
260/// [`image_digest`] re-derives a key that is private to `boxlite`, so this is the
261/// evidence that the re-derivation still matches. `Complete` means every
262/// `disk-images/*.ext4` in the store was claimed; `Partial` names how many were
263/// not, and those bytes are reported as unattributed rather than deleted.
264#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)]
265#[serde(rename_all = "kebab-case")]
266pub enum Attribution {
267 /// Every derived artifact belongs to an indexed image.
268 Complete,
269 /// Some do not. A per-image `clear` will not touch them.
270 Partial,
271}
272
273/// One image's byte accounting.
274///
275/// `total` is everything the image references; `exclusive` is what dropping this
276/// image *alone* would free. They differ exactly when another cached image shares
277/// a blob, which is the case the set difference exists for — and reporting only
278/// `total` would promise bytes back that a shared layer keeps.
279#[derive(Debug, Clone, Copy, Default, Serialize)]
280pub struct ImageBytes {
281 /// The manifest and config JSON.
282 pub metadata: u64,
283 /// The compressed layer tarballs under `images/layers/`.
284 pub layers: u64,
285 /// The unpacked layer trees under `images/extracted/`.
286 pub extracted: u64,
287 /// The ext4 disk image built from the layer stack, if one has been built.
288 pub disk_image: u64,
289 /// The guest rootfs base built from that disk image, if one has been built.
290 pub base_disk: u64,
291 /// Everything above.
292 pub total: u64,
293 /// What dropping this image alone would actually free — `total` minus every
294 /// byte another cached image also references.
295 pub exclusive: u64,
296}
297
298/// How much of an image's **pulled** content is on disk.
299///
300/// Manifest, config and one entry per unique layer digest — the objects a pull
301/// produces and a run consumes. Deliberately *not* counting the extracted trees,
302/// the disk image or the base rootfs: those are built lazily on first run and are
303/// a cache below this cache, so an image that has only ever been pulled is
304/// complete without them.
305///
306/// The unit matches what issue #433's hand verification counted: 15/15 for
307/// `semgrep` (one manifest, one config, thirteen layers) and 3/3 for `debian`.
308#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize)]
309pub struct Objects {
310 /// How many objects the index says this image has.
311 pub expected: usize,
312 /// How many of them are on disk.
313 pub present: usize,
314}
315
316impl Objects {
317 /// Whether every object the index names is on disk.
318 #[must_use]
319 pub fn complete(self) -> bool {
320 self.expected == self.present
321 }
322}
323
324/// One image in the store.
325#[derive(Debug, Clone, Serialize)]
326pub struct CachedImage {
327 /// The reference the index holds it under — a tag, or a digest-pinned name.
328 pub reference: String,
329 /// The manifest digest, as the index spells it (`sha256:…`).
330 pub manifest_digest: String,
331 /// The config digest, as the index spells it.
332 pub config_digest: String,
333 /// The derived key its disk image and base rootfs are stored under
334 /// ([`image_digest`]).
335 pub image_digest: String,
336 /// When the pull completed, as the index recorded it.
337 pub cached_at: String,
338 /// Whether the index considers the pull finished. A `false` here is a partial
339 /// pull, not a corrupt one, and `security prefetch` completes it.
340 pub pull_complete: bool,
341 /// How many **distinct** layer digests it has. Below the manifest's layer
342 /// count when a layer is named twice.
343 pub layers: usize,
344 /// Its byte accounting.
345 pub bytes: ImageBytes,
346 /// How much of its pulled content is on disk.
347 pub objects: Objects,
348 /// Whether a disk image has been built from it.
349 pub disk_image_built: bool,
350 /// Whether a guest rootfs base has been built from it.
351 pub base_disk_built: bool,
352}
353
354/// Bytes under the store root that no indexed image claims.
355#[derive(Debug, Clone, Serialize)]
356pub struct Unattributed {
357 /// Where it is, relative to the store root.
358 pub path: String,
359 /// Its size in bytes.
360 pub bytes: u64,
361}
362
363/// Something under the store root that [`clear`] deliberately leaves alone, and
364/// why.
365#[derive(Debug, Clone, Serialize)]
366pub struct Preserved {
367 /// Where it is, relative to the store root.
368 pub path: String,
369 /// Why it is not re-obtainable from a pinned digest, in one sentence.
370 pub reason: String,
371}
372
373/// What the sandbox store is holding.
374///
375/// # Why `scope` is a field and not a sentence in a doc comment
376///
377/// The same reason [`crate::tool_security::MachineScope`] carries one: an asset
378/// root is a property of the **machine**, and a caller who selected a project
379/// (ADR-0008) will otherwise read a store size as that project's. There is one
380/// sandbox store per asset root and every repository on the host shares it, so
381/// the document says so in a field that travels with any half of it that gets
382/// quoted.
383#[derive(Debug, Clone, Serialize)]
384pub struct SandboxStatus {
385 /// Stable schema tag ([`SANDBOX_STATUS_SCHEMA`]).
386 pub schema: &'static str,
387 /// Always `"machine"`. The store is shared by every repository on this host.
388 pub scope: &'static str,
389 /// The store root these numbers describe.
390 pub store: String,
391 /// Whether there is a store there at all. `false` means nothing is cached,
392 /// which is a different fact from an empty index.
393 pub present: bool,
394 /// Every image the index is holding, largest first.
395 pub images: Vec<CachedImage>,
396 /// Whether every derived artifact was claimed by one of them.
397 pub attribution: Attribution,
398 /// Bytes no indexed image claims. A per-image `clear` never touches these.
399 pub unattributed: Vec<Unattributed>,
400 /// State a digest does not re-obtain, which `clear` will not remove.
401 pub preserved: Vec<Preserved>,
402 /// How many boxes are registered. Non-zero blocks a `clear`.
403 pub live_boxes: usize,
404 /// Every byte under the store root.
405 pub total_bytes: u64,
406}
407
408/// One image [`clear`] removed, and what removing it freed.
409#[derive(Debug, Clone, Serialize)]
410pub struct RemovedImage {
411 /// The reference it was held under.
412 pub reference: String,
413 /// Bytes freed by removing it — its exclusive bytes, never its total.
414 pub freed_bytes: u64,
415 /// How many objects were removed from the filesystem.
416 pub objects_removed: usize,
417}
418
419/// A surviving image, re-checked against the filesystem after the deletion.
420///
421/// The assertion that matters. A set-difference bug does not present as an error;
422/// it presents as an image whose blobs are gone, discovered on the next run. So
423/// `clear` re-resolves every survivor's manifest, config and layers against the
424/// filesystem *after* deleting, and says so in its report — issue #433's "a
425/// `clear` that cannot demonstrate the surviving images are still complete is one
426/// nobody trusts twice".
427#[derive(Debug, Clone, Serialize)]
428pub struct VerifiedImage {
429 /// The reference it is held under.
430 pub reference: String,
431 /// Its object tally after the deletion.
432 pub objects: Objects,
433 /// Whether every object is still there.
434 pub complete: bool,
435}
436
437/// What a [`clear`] would do, or did.
438#[derive(Debug, Clone, Serialize)]
439pub struct ClearReport {
440 /// Stable schema tag ([`SANDBOX_CLEAR_SCHEMA`]).
441 pub schema: &'static str,
442 /// Always `"machine"`, for the reason [`SandboxStatus::scope`] gives.
443 pub scope: &'static str,
444 /// The store root that was cleared.
445 pub store: String,
446 /// What was asked for — a reference, or `everything`.
447 pub requested: String,
448 /// Whether this is a plan or a completed removal.
449 pub applied: bool,
450 /// The images removed.
451 pub removed: Vec<RemovedImage>,
452 /// The unattributed bytes removed. Only ever populated for
453 /// [`Scope::Everything`].
454 pub removed_unattributed: Vec<Unattributed>,
455 /// Bytes accounted for by the objects removed.
456 pub freed_bytes: u64,
457 /// Every byte under the store root before the deletion.
458 pub store_bytes_before: u64,
459 /// Every byte under the store root after it. Equal to `store_bytes_before` on
460 /// a plan.
461 pub store_bytes_after: u64,
462 /// The survivors, re-checked blob by blob after the deletion.
463 pub retained: Vec<VerifiedImage>,
464 /// State a digest does not re-obtain, left alone and named.
465 pub preserved: Vec<Preserved>,
466}
467
468impl ClearReport {
469 /// The bytes the filesystem actually gave back.
470 ///
471 /// Reported alongside [`ClearReport::freed_bytes`] rather than instead of it:
472 /// the accounted figure is what this module believes it removed, the measured
473 /// one is what the store shrank by, and the two are checkable against each
474 /// other by anyone holding a `du`.
475 ///
476 /// They are not identical, and the gap is one thing: the index moves under a
477 /// `clear` without being part of what was removed. **`SQLite` does not shrink
478 /// a file when rows are deleted** — it frees pages inside it — and a `DELETE`
479 /// under a write transaction can add a page, while a checkpoint can hand back
480 /// a write-ahead log. So the gap runs in **both** directions, it is kilobytes
481 /// against a clear measured in gigabytes, and
482 /// `the_accounted_bytes_and_the_measured_bytes_differ_only_by_the_index` is
483 /// what keeps that claim true. Anything larger is a defect, which is why both
484 /// numbers are reported rather than one.
485 ///
486 /// Measured on the real store: dropping an image that shared every layer with
487 /// a survivor accounted for 0 bytes and measured 32 KiB back, all of it the
488 /// index.
489 #[must_use]
490 pub fn measured_freed_bytes(&self) -> u64 {
491 self.store_bytes_before
492 .saturating_sub(self.store_bytes_after)
493 }
494
495 /// Whether every surviving image is still complete.
496 #[must_use]
497 pub fn survivors_intact(&self) -> bool {
498 self.retained.iter().all(|image| image.complete)
499 }
500}
501
502// ---------------------------------------------------------------------------
503// The index
504// ---------------------------------------------------------------------------
505
506/// One `image_index` row, with its layer list parsed.
507#[derive(Debug, Clone)]
508struct IndexRow {
509 reference: String,
510 manifest_digest: String,
511 config_digest: String,
512 /// In manifest order and **with duplicates**, because [`image_digest`] hashes
513 /// the list as the manifest wrote it.
514 layers: Vec<String>,
515 cached_at: String,
516 complete: bool,
517}
518
519impl IndexRow {
520 /// The distinct layer digests, for the object lists that are keyed by digest.
521 fn unique_layers(&self) -> BTreeSet<String> {
522 self.layers.iter().cloned().collect()
523 }
524}
525
526/// One `base_disk` row.
527#[derive(Debug, Clone)]
528struct BaseRow {
529 name: String,
530 kind: String,
531 path: PathBuf,
532}
533
534/// What the index says, read in one pass.
535#[derive(Debug, Default)]
536struct Index {
537 images: Vec<IndexRow>,
538 /// Base disks that live under the store root. **Only** these; a row naming a
539 /// path elsewhere is in `escaped` and no derivation here can reach it.
540 bases: Vec<BaseRow>,
541 /// Base-disk rows pointing outside the store root.
542 ///
543 /// `base_path` is an absolute path recorded when the base was built, so it is
544 /// data rather than a derivation — and data can name anywhere, including the
545 /// repository store this verb must never reach. Split off at the point it is
546 /// read, so no later code has to remember to check: a relocated store makes
547 /// every row here stale at once, and the guarantee wanted is that nothing
548 /// outside the root is measured, listed **or** deleted, not only the last of
549 /// those.
550 escaped: Vec<PathBuf>,
551 boxes: usize,
552}
553
554/// Read `image_index`, `base_disk` and the box count.
555///
556/// Opened read-only: [`status`] must not be able to write to a store it is only
557/// describing, and [`clear`] does its own writing through a separate connection
558/// once it has decided what to do.
559fn read_index(store: &Path) -> Result<Index, StoreError> {
560 let path = store.join(INDEX_DB);
561 if !path.exists() {
562 return Ok(Index::default());
563 }
564 let fail = |message: String| StoreError::Index {
565 path: path.display().to_string(),
566 message,
567 };
568 let db = rusqlite::Connection::open_with_flags(
569 &path,
570 rusqlite::OpenFlags::SQLITE_OPEN_READ_ONLY | rusqlite::OpenFlags::SQLITE_OPEN_URI,
571 )
572 .map_err(|error| fail(error.to_string()))?;
573
574 let mut images = Vec::new();
575 {
576 let mut statement = db
577 .prepare(
578 "SELECT reference, manifest_digest, config_digest, layers, cached_at, complete \
579 FROM image_index ORDER BY reference",
580 )
581 .map_err(|error| fail(error.to_string()))?;
582 let rows = statement
583 .query_map([], |row| {
584 Ok((
585 row.get::<_, String>(0)?,
586 row.get::<_, String>(1)?,
587 row.get::<_, String>(2)?,
588 row.get::<_, String>(3)?,
589 row.get::<_, String>(4)?,
590 row.get::<_, i64>(5)?,
591 ))
592 })
593 .map_err(|error| fail(error.to_string()))?;
594 for row in rows {
595 let (reference, manifest_digest, config_digest, layers, cached_at, complete) =
596 row.map_err(|error| fail(error.to_string()))?;
597 // A layer list that will not parse is a row this module cannot derive
598 // paths from, and inventing an empty one would make its blobs look
599 // unreferenced — which is how a set difference deletes a live image.
600 // Refusing is the only safe reading.
601 let layers: Vec<String> = serde_json::from_str(&layers).map_err(|error| {
602 fail(format!(
603 "image_index row `{reference}` has an unreadable layer list: {error}"
604 ))
605 })?;
606 images.push(IndexRow {
607 reference,
608 manifest_digest,
609 config_digest,
610 layers,
611 cached_at,
612 complete: complete != 0,
613 });
614 }
615 }
616
617 let mut bases = Vec::new();
618 let mut escaped = Vec::new();
619 {
620 let mut statement = db
621 .prepare("SELECT name, kind, base_path FROM base_disk ORDER BY id")
622 .map_err(|error| fail(error.to_string()))?;
623 let rows = statement
624 .query_map([], |row| {
625 Ok((
626 row.get::<_, Option<String>>(0)?,
627 row.get::<_, String>(1)?,
628 row.get::<_, String>(2)?,
629 ))
630 })
631 .map_err(|error| fail(error.to_string()))?;
632 for row in rows {
633 let (name, kind, path) = row.map_err(|error| fail(error.to_string()))?;
634 let path = PathBuf::from(path);
635 if path.starts_with(store) {
636 bases.push(BaseRow {
637 name: name.unwrap_or_default(),
638 kind,
639 path,
640 });
641 } else {
642 escaped.push(path);
643 }
644 }
645 }
646
647 let boxes = db
648 .query_row("SELECT COUNT(*) FROM box_config", [], |row| {
649 row.get::<_, i64>(0)
650 })
651 .map_err(|error| fail(error.to_string()))?;
652
653 Ok(Index {
654 images,
655 bases,
656 escaped,
657 boxes: usize::try_from(boxes).unwrap_or(usize::MAX),
658 })
659}
660
661// ---------------------------------------------------------------------------
662// Object lists
663// ---------------------------------------------------------------------------
664
665/// The `bases/` prefix a `rootfs` base disk belonging to `digest` is named with.
666///
667/// `boxlite`'s `GuestRootfsManager::version_key` is
668/// `{image_digest[..12]}-{guest_binary_hash[..12]}`, so the image half is a
669/// prefix match and the guest half — which changes when the runtime does — is not
670/// something this crate has to know.
671fn base_name_prefix(image_digest: &str) -> String {
672 let bare = image_digest.strip_prefix("sha256:").unwrap_or(image_digest);
673 format!("{}-", &bare[..12.min(bare.len())])
674}
675
676/// Every path one image references, split into the pulled objects and the
677/// derived ones.
678#[derive(Debug, Default)]
679struct ImageObjects {
680 /// Manifest, config and layer tarballs — what a pull produces. The unit
681 /// [`Objects`] counts.
682 pulled: Vec<PathBuf>,
683 /// Extracted trees, the disk image and the base rootfs — built lazily on
684 /// first run, and a cache below this cache.
685 derived: Vec<PathBuf>,
686 /// How many `rootfs` base disks the index attributes to this image.
687 bases: usize,
688}
689
690impl ImageObjects {
691 fn all(&self) -> impl Iterator<Item = &PathBuf> {
692 self.pulled.iter().chain(self.derived.iter())
693 }
694}
695
696/// Resolve one index row to the paths it references.
697///
698/// The index manifest — the `application/vnd.oci.image.index.v1+json` a tag or a
699/// digest reference resolves through — is *not* here. It is claimed separately by
700/// [`index_manifests_for`], because a tag reference does not record which index it
701/// came from and the association has to be read out of the files themselves.
702fn objects_for(store: &Path, row: &IndexRow, bases: &[BaseRow]) -> ImageObjects {
703 let images = store.join("images");
704 let mut objects = ImageObjects::default();
705
706 objects.pulled.push(
707 images
708 .join("manifests")
709 .join(format!("{}.json", blob_name(&row.manifest_digest))),
710 );
711 objects.pulled.push(
712 images
713 .join("configs")
714 .join(format!("{}.json", blob_name(&row.config_digest))),
715 );
716 for layer in row.unique_layers() {
717 let name = blob_name(&layer);
718 objects
719 .pulled
720 .push(images.join("layers").join(format!("{name}.tar.gz")));
721 objects.derived.push(images.join("extracted").join(name));
722 }
723
724 let digest = image_digest(&row.layers);
725 objects.derived.push(
726 images
727 .join("disk-images")
728 .join(format!("{}.ext4", blob_name(&digest))),
729 );
730 let prefix = base_name_prefix(&digest);
731 for base in bases {
732 if base.kind == "rootfs" && base.name.starts_with(&prefix) {
733 objects.derived.push(base.path.clone());
734 objects.bases += 1;
735 }
736 }
737 objects
738}
739
740/// The index manifests that resolve to any of `retained` platform manifests.
741///
742/// `docker.io/library/debian:bookworm-slim` is held under a **tag**, so its index
743/// digest appears nowhere in `image_index` — only the platform manifest it
744/// resolved to does. Deleting every manifest file the index does not name would
745/// therefore take the index file of a surviving image with it. Reading the files
746/// and keeping any that lists a retained manifest is the association the database
747/// does not carry.
748fn index_manifests_for(store: &Path, retained: &BTreeSet<String>) -> BTreeSet<PathBuf> {
749 let dir = store.join("images").join("manifests");
750 let mut keep = BTreeSet::new();
751 let Ok(entries) = std::fs::read_dir(&dir) else {
752 return keep;
753 };
754 for entry in entries.flatten() {
755 let path = entry.path();
756 let Ok(bytes) = std::fs::read(&path) else {
757 continue;
758 };
759 let Ok(document) = serde_json::from_slice::<serde_json::Value>(&bytes) else {
760 continue;
761 };
762 let Some(children) = document.get("manifests").and_then(|value| value.as_array()) else {
763 continue;
764 };
765 if children
766 .iter()
767 .filter_map(|child| child.get("digest").and_then(|value| value.as_str()))
768 .any(|digest| retained.contains(digest))
769 {
770 keep.insert(path);
771 }
772 }
773 keep
774}
775
776// ---------------------------------------------------------------------------
777// Sizes
778// ---------------------------------------------------------------------------
779
780/// Bytes **allocated** to one file — what removing it gives back.
781///
782/// Not its length. The ext4 disks in this store are sparse, and by a margin
783/// nobody would call rounding: `sha256-0a89fbeb….ext4` in the store measured for
784/// issue #433 is 1.23 GiB long and occupies 1.09 GiB, and the base rootfs beside
785/// it is 256 MiB long and occupies 110 MiB. Reporting the length would tell
786/// somebody a clear had freed 9.1 GiB where `du` says 8.7 GiB, and the number
787/// this verb exists to report would be the one number in it that could not be
788/// checked.
789///
790/// So it is `st_blocks`, which is what `du` counts and what the filesystem
791/// actually returns. Off Unix there is no such field and the length is the best
792/// available answer; nothing in the store is sparse on a filesystem without it.
793fn allocated(metadata: &std::fs::Metadata) -> u64 {
794 #[cfg(unix)]
795 {
796 use std::os::unix::fs::MetadataExt as _;
797 metadata.blocks() * 512
798 }
799 #[cfg(not(unix))]
800 {
801 metadata.len()
802 }
803}
804
805/// Bytes at `path`: one file's allocation, or the sum of every regular file
806/// beneath it.
807///
808/// See [`allocated`] for why this is not the apparent size.
809fn size_of(path: &Path) -> u64 {
810 let Ok(metadata) = std::fs::symlink_metadata(path) else {
811 return 0;
812 };
813 if !metadata.is_dir() {
814 return allocated(&metadata);
815 }
816 let mut total = allocated(&metadata);
817 let mut stack = vec![path.to_path_buf()];
818 while let Some(dir) = stack.pop() {
819 let Ok(entries) = std::fs::read_dir(&dir) else {
820 continue;
821 };
822 for entry in entries.flatten() {
823 let Ok(metadata) = entry.metadata() else {
824 continue;
825 };
826 total += allocated(&metadata);
827 if metadata.is_dir() {
828 stack.push(entry.path());
829 }
830 }
831 }
832 total
833}
834
835/// Measure every path once, so a digest shared by two images is not weighed twice.
836fn measure(paths: impl IntoIterator<Item = PathBuf>) -> BTreeMap<PathBuf, u64> {
837 let mut sizes = BTreeMap::new();
838 for path in paths {
839 sizes.entry(path).or_insert_with_key(|path| size_of(path));
840 }
841 sizes
842}
843
844// ---------------------------------------------------------------------------
845// status
846// ---------------------------------------------------------------------------
847
848/// Where the sandbox store lives under an asset cache root.
849#[must_use]
850pub fn store_root(asset_root: &Path) -> PathBuf {
851 asset_root.join(SANDBOX_STORE_DIR)
852}
853
854/// Report what the sandbox store is holding, per image, with sizes.
855///
856/// Enough to decide what to drop, which is ADR-0014 v1.6's third rule: a
857/// destructive verb with no way to see what it will destroy is invoked blind.
858///
859/// # Errors
860///
861/// [`StoreError::Index`] when the index exists and cannot be read. An **absent**
862/// store is not an error — it is `present: false`, because "nothing is cached" is
863/// an answer rather than a failure.
864pub fn status(asset_root: &Path) -> Result<SandboxStatus, StoreError> {
865 let store = store_root(asset_root);
866 let mut report = SandboxStatus {
867 schema: SANDBOX_STATUS_SCHEMA,
868 scope: "machine",
869 store: store.display().to_string(),
870 present: store.is_dir(),
871 images: Vec::new(),
872 attribution: Attribution::Complete,
873 unattributed: Vec::new(),
874 preserved: Vec::new(),
875 live_boxes: 0,
876 total_bytes: 0,
877 };
878 if !report.present {
879 return Ok(report);
880 }
881
882 let index = read_index(&store)?;
883 report.live_boxes = index.boxes;
884 report.total_bytes = size_of(&store);
885
886 let objects: Vec<(usize, ImageObjects)> = index
887 .images
888 .iter()
889 .enumerate()
890 .map(|(at, row)| (at, objects_for(&store, row, &index.bases)))
891 .collect();
892
893 // Every path any image references, measured once. `references` counts how
894 // many images claim each path, which is what turns a total into an exclusive
895 // figure without measuring anything twice.
896 let sizes = measure(
897 objects
898 .iter()
899 .flat_map(|(_, object)| object.all().cloned())
900 .collect::<Vec<_>>(),
901 );
902 let mut references: BTreeMap<&PathBuf, usize> = BTreeMap::new();
903 for (_, object) in &objects {
904 for path in object.all() {
905 *references.entry(path).or_default() += 1;
906 }
907 }
908
909 for (at, object) in &objects {
910 let row = &index.images[*at];
911 report
912 .images
913 .push(cached_image(&store, row, object, &sizes, &references));
914 }
915 report
916 .images
917 .sort_by_key(|image| std::cmp::Reverse(image.bytes.total));
918
919 let claimed: BTreeSet<PathBuf> = objects
920 .iter()
921 .flat_map(|(_, object)| object.all().cloned())
922 .chain(index_manifests_for(
923 &store,
924 &index
925 .images
926 .iter()
927 .map(|row| row.manifest_digest.clone())
928 .collect(),
929 ))
930 .chain(preserved_paths(&index))
931 .collect();
932 report.unattributed = unattributed(&store, &claimed);
933 if !report.unattributed.is_empty() {
934 report.attribution = Attribution::Partial;
935 }
936 report.preserved = preserved(&index);
937 Ok(report)
938}
939
940/// Build one image's row from its object list and the shared size table.
941fn cached_image(
942 store: &Path,
943 row: &IndexRow,
944 object: &ImageObjects,
945 sizes: &BTreeMap<PathBuf, u64>,
946 references: &BTreeMap<&PathBuf, usize>,
947) -> CachedImage {
948 let images = store.join("images");
949 let mut bytes = ImageBytes::default();
950 for path in object.all() {
951 let size = sizes.get(path).copied().unwrap_or_default();
952 bytes.total += size;
953 if references.get(path).copied().unwrap_or(1) == 1 {
954 bytes.exclusive += size;
955 }
956 if path.starts_with(images.join("layers")) {
957 bytes.layers += size;
958 } else if path.starts_with(images.join("extracted")) {
959 bytes.extracted += size;
960 } else if path.starts_with(images.join("disk-images")) {
961 bytes.disk_image += size;
962 } else if path.starts_with(images.join("manifests"))
963 || path.starts_with(images.join("configs"))
964 {
965 bytes.metadata += size;
966 } else {
967 bytes.base_disk += size;
968 }
969 }
970
971 let digest = image_digest(&row.layers);
972 let disk = images
973 .join("disk-images")
974 .join(format!("{}.ext4", blob_name(&digest)));
975 CachedImage {
976 reference: row.reference.clone(),
977 manifest_digest: row.manifest_digest.clone(),
978 config_digest: row.config_digest.clone(),
979 image_digest: digest,
980 cached_at: row.cached_at.clone(),
981 pull_complete: row.complete,
982 layers: row.unique_layers().len(),
983 bytes,
984 objects: Objects {
985 expected: object.pulled.len(),
986 present: object.pulled.iter().filter(|path| path.exists()).count(),
987 },
988 disk_image_built: disk.exists(),
989 base_disk_built: object.bases > 0,
990 }
991}
992
993/// Digest-keyed objects that no indexed image claims.
994///
995/// One entry per top-level object rather than a single total, because "146 MB of
996/// extracted layer nobody references" is actionable and "146 MB unaccounted" is
997/// not. The live store had exactly one — an extracted tree whose layer tarball
998/// and index row are both gone.
999///
1000/// `claimed` must already carry the [`preserved`] paths as well as the objects
1001/// every indexed image references: a snapshot's base disk is claimed by nobody
1002/// and is not therefore spare.
1003fn unattributed(store: &Path, claimed: &BTreeSet<PathBuf>) -> Vec<Unattributed> {
1004 let images = store.join("images");
1005 let mut scanned: Vec<PathBuf> = IMAGE_DIRS.iter().map(|dir| images.join(dir)).collect();
1006 scanned.push(store.join("bases"));
1007 let mut found = Vec::new();
1008 for dir in scanned {
1009 let Ok(entries) = std::fs::read_dir(&dir) else {
1010 continue;
1011 };
1012 for entry in entries.flatten() {
1013 let path = entry.path();
1014 if claimed.contains(&path) {
1015 continue;
1016 }
1017 found.push(Unattributed {
1018 path: path
1019 .strip_prefix(store)
1020 .unwrap_or(&path)
1021 .display()
1022 .to_string(),
1023 bytes: size_of(&path),
1024 });
1025 }
1026 }
1027 found.sort_by_key(|entry| std::cmp::Reverse(entry.bytes));
1028 found
1029}
1030
1031/// State under the store root that no pinned digest re-obtains.
1032///
1033/// A `rootfs` base is rebuilt from its image; a `snapshot` or a `clone_base` is
1034/// the state of a box somebody ran. ADR-0014 v1.6's permission does not extend to
1035/// it, so it is listed rather than cleared.
1036fn preserved(index: &Index) -> Vec<Preserved> {
1037 index
1038 .bases
1039 .iter()
1040 .filter(|base| base.kind != "rootfs")
1041 .map(|base| Preserved {
1042 path: base.path.display().to_string(),
1043 reason: format!(
1044 "a `{}` base disk is the state of a box that ran, which no digest re-obtains",
1045 base.kind
1046 ),
1047 })
1048 .chain(index.escaped.iter().map(|path| {
1049 Preserved {
1050 path: path.display().to_string(),
1051 reason: "the index names this base disk outside the store root, so nothing \
1052 here measures, lists or removes it"
1053 .to_owned(),
1054 }
1055 }))
1056 .collect()
1057}
1058
1059/// The paths [`preserved`] names, as a set.
1060///
1061/// Folded into `claimed` before [`unattributed`] runs, so that state a digest
1062/// does not re-obtain is never reported as spare and never reaches the doomed
1063/// set. Nobody references a snapshot's base disk, which is exactly why an
1064/// unclaimed-means-spare rule would delete it.
1065fn preserved_paths(index: &Index) -> BTreeSet<PathBuf> {
1066 index
1067 .bases
1068 .iter()
1069 .filter(|base| base.kind != "rootfs")
1070 .map(|base| base.path.clone())
1071 .collect()
1072}
1073
1074// ---------------------------------------------------------------------------
1075// clear
1076// ---------------------------------------------------------------------------
1077
1078/// Split the index into the rows a scope drops and the rows it keeps.
1079///
1080/// Returns them in that order — doomed first — because that is the order the two
1081/// are used in, and a pair whose halves can be swapped by a careless edit is a
1082/// set difference computed the wrong way round.
1083fn select<'a>(
1084 index: &'a Index,
1085 scope: &Scope,
1086) -> Result<(Vec<&'a IndexRow>, Vec<&'a IndexRow>), StoreError> {
1087 match scope {
1088 Scope::Everything => Ok((index.images.iter().collect(), Vec::new())),
1089 Scope::Image(reference) => {
1090 if !index.images.iter().any(|row| &row.reference == reference) {
1091 return Err(StoreError::UnknownImage {
1092 reference: reference.clone(),
1093 known: index
1094 .images
1095 .iter()
1096 .map(|row| row.reference.as_str())
1097 .collect::<Vec<_>>()
1098 .join(", "),
1099 });
1100 }
1101 Ok(index
1102 .images
1103 .iter()
1104 .partition(|row| &row.reference == reference))
1105 }
1106 }
1107}
1108
1109/// Every path a set of index rows references, including the index manifests they
1110/// resolve through.
1111///
1112/// The other half of the set difference, and the reason it is a named function:
1113/// applied to the **surviving** rows it is the retained set, and anything not in
1114/// it is what may go. Computing it from the doomed rows instead is the walk this
1115/// module's documentation says a deletion must never be.
1116fn paths_for(store: &Path, rows: &[&IndexRow], bases: &[BaseRow]) -> BTreeSet<PathBuf> {
1117 rows.iter()
1118 .flat_map(|row| {
1119 objects_for(store, row, bases)
1120 .all()
1121 .cloned()
1122 .collect::<Vec<_>>()
1123 })
1124 .chain(index_manifests_for(
1125 store,
1126 &rows.iter().map(|row| row.manifest_digest.clone()).collect(),
1127 ))
1128 .collect()
1129}
1130
1131/// What a [`clear`] would remove, without removing it.
1132///
1133/// Separate from [`clear`] so the set difference has one implementation and the
1134/// tests can assert on the doomed set directly rather than inferring it from what
1135/// survived a deletion.
1136///
1137/// # Errors
1138///
1139/// [`StoreError::UnknownImage`] when a per-image scope names a reference the index
1140/// does not hold, [`StoreError::LiveBoxes`] when a box is registered,
1141/// [`StoreError::UnrecognisedEntry`] when the store root holds something this
1142/// module cannot classify, [`StoreError::BaseOutsideStore`] when a base-disk row
1143/// points outside the store, and [`StoreError::Index`] when the index cannot be
1144/// read.
1145pub fn plan(asset_root: &Path, scope: &Scope) -> Result<(ClearReport, Vec<PathBuf>), StoreError> {
1146 let store = store_root(asset_root);
1147 let mut report = ClearReport {
1148 schema: SANDBOX_CLEAR_SCHEMA,
1149 scope: "machine",
1150 store: store.display().to_string(),
1151 requested: scope.as_str().to_owned(),
1152 applied: false,
1153 removed: Vec::new(),
1154 removed_unattributed: Vec::new(),
1155 freed_bytes: 0,
1156 store_bytes_before: 0,
1157 store_bytes_after: 0,
1158 retained: Vec::new(),
1159 preserved: Vec::new(),
1160 };
1161 if !store.is_dir() {
1162 return Ok((report, Vec::new()));
1163 }
1164
1165 let index = read_index(&store)?;
1166 if index.boxes > 0 {
1167 return Err(StoreError::LiveBoxes { boxes: index.boxes });
1168 }
1169 guard_entries(&store)?;
1170 guard_bases(&store, &index)?;
1171
1172 let (doomed_rows, surviving_rows) = select(&index, scope)?;
1173 let retained = paths_for(&store, &surviving_rows, &index.bases);
1174
1175 report.store_bytes_before = size_of(&store);
1176 report.preserved = preserved(&index);
1177
1178 let mut doomed: Vec<PathBuf> = Vec::new();
1179 for row in &doomed_rows {
1180 let objects = objects_for(&store, row, &index.bases);
1181 let mine: Vec<PathBuf> = objects
1182 .all()
1183 .filter(|path| !retained.contains(*path))
1184 .filter(|path| path.exists())
1185 .cloned()
1186 .collect();
1187 let freed = mine.iter().map(|path| size_of(path)).sum();
1188 report.removed.push(RemovedImage {
1189 reference: row.reference.clone(),
1190 freed_bytes: freed,
1191 objects_removed: mine.len(),
1192 });
1193 report.freed_bytes += freed;
1194 doomed.extend(mine);
1195 }
1196
1197 // The index manifest a doomed image resolved through goes with it, unless a
1198 // survivor resolves through the same one.
1199 for path in index_manifests_for(
1200 &store,
1201 &doomed_rows
1202 .iter()
1203 .map(|row| row.manifest_digest.clone())
1204 .collect(),
1205 ) {
1206 if !retained.contains(&path) && path.exists() {
1207 report.freed_bytes += size_of(&path);
1208 doomed.push(path);
1209 }
1210 }
1211
1212 // Unattributed bytes are only ever in scope for `everything`. A per-image
1213 // request has no evidence they belong to the image it named, and this module
1214 // does not delete on a guess.
1215 if matches!(scope, Scope::Everything) {
1216 let claimed: BTreeSet<PathBuf> = doomed
1217 .iter()
1218 .cloned()
1219 .chain(preserved_paths(&index))
1220 .collect();
1221 report.removed_unattributed = unattributed(&store, &claimed);
1222 for entry in &report.removed_unattributed {
1223 report.freed_bytes += entry.bytes;
1224 doomed.push(store.join(&entry.path));
1225 }
1226 }
1227
1228 doomed.sort();
1229 doomed.dedup();
1230 report.store_bytes_after = report.store_bytes_before;
1231 Ok((report, doomed))
1232}
1233
1234/// Remove what [`plan`] named, then re-check every surviving image against the
1235/// filesystem.
1236///
1237/// # Errors
1238///
1239/// Everything [`plan`] can return, plus [`StoreError::Io`] if a removal or the
1240/// index update fails.
1241pub fn clear(asset_root: &Path, scope: &Scope) -> Result<ClearReport, StoreError> {
1242 let store = store_root(asset_root);
1243 let (mut report, doomed) = plan(asset_root, scope)?;
1244 if !store.is_dir() {
1245 report.applied = true;
1246 return Ok(report);
1247 }
1248
1249 for path in &doomed {
1250 remove(path)?;
1251 }
1252 // The rows go with the blobs. Leaving them behind makes `status` report images
1253 // whose bytes are gone, which is issue #433's fourth trap and reads as a
1254 // corrupt store rather than a cleared one.
1255 forget(&store, &report.removed)?;
1256
1257 report.applied = true;
1258 report.store_bytes_after = size_of(&store);
1259 report.retained = verify(&store)?;
1260 Ok(report)
1261}
1262
1263/// Delete a file or a directory tree.
1264fn remove(path: &Path) -> Result<(), StoreError> {
1265 let metadata = match std::fs::symlink_metadata(path) {
1266 Ok(metadata) => metadata,
1267 Err(error) if error.kind() == std::io::ErrorKind::NotFound => return Ok(()),
1268 Err(error) => {
1269 return Err(StoreError::Io {
1270 action: "inspecting",
1271 path: path.display().to_string(),
1272 message: error.to_string(),
1273 });
1274 }
1275 };
1276 let outcome = if metadata.is_dir() {
1277 std::fs::remove_dir_all(path)
1278 } else {
1279 std::fs::remove_file(path)
1280 };
1281 outcome.map_err(|error| StoreError::Io {
1282 action: "removing",
1283 path: path.display().to_string(),
1284 message: error.to_string(),
1285 })
1286}
1287
1288/// Drop the index rows for what was removed.
1289///
1290/// Two different rules, because the two tables are keyed differently. An
1291/// `image_index` row is named by the request, so it goes by reference. A
1292/// `base_disk` row is named by nothing the request knows, so it goes **by whether
1293/// its file is still there** — which means the set difference that spared a base
1294/// disk a surviving image shares also spares its row, with no second chance to get
1295/// the difference wrong.
1296fn forget(store: &Path, removed: &[RemovedImage]) -> Result<(), StoreError> {
1297 let path = store.join(INDEX_DB);
1298 if !path.exists() {
1299 return Ok(());
1300 }
1301 let fail = |message: String| StoreError::Index {
1302 path: path.display().to_string(),
1303 message,
1304 };
1305 let db = rusqlite::Connection::open(&path).map_err(|error| fail(error.to_string()))?;
1306 // `IMMEDIATE` takes the write lock up front, so a concurrent `boxlite` pull
1307 // fails to start rather than interleaving with this. It is not the `flock`
1308 // that crate holds — see `StoreError::LiveBoxes` for why this crate cannot
1309 // take that one — and it is the strongest guard available without `unsafe`.
1310 db.execute_batch("BEGIN IMMEDIATE")
1311 .map_err(|error| fail(error.to_string()))?;
1312 for image in removed {
1313 db.execute(
1314 "DELETE FROM image_index WHERE reference = ?1",
1315 [&image.reference],
1316 )
1317 .map_err(|error| fail(error.to_string()))?;
1318 }
1319 let orphaned: Vec<String> = {
1320 let mut statement = db
1321 .prepare("SELECT base_path FROM base_disk WHERE kind = 'rootfs'")
1322 .map_err(|error| fail(error.to_string()))?;
1323 let rows = statement
1324 .query_map([], |row| row.get::<_, String>(0))
1325 .map_err(|error| fail(error.to_string()))?;
1326 rows.filter_map(Result::ok)
1327 .filter(|base_path| !Path::new(base_path).exists())
1328 .collect()
1329 };
1330 for base_path in &orphaned {
1331 db.execute(
1332 "DELETE FROM base_disk WHERE kind = 'rootfs' AND base_path = ?1",
1333 [base_path],
1334 )
1335 .map_err(|error| fail(error.to_string()))?;
1336 }
1337 db.execute_batch("COMMIT")
1338 .map_err(|error| fail(error.to_string()))?;
1339 Ok(())
1340}
1341
1342/// Re-resolve every image the index still holds against the filesystem.
1343fn verify(store: &Path) -> Result<Vec<VerifiedImage>, StoreError> {
1344 let index = read_index(store)?;
1345 Ok(index
1346 .images
1347 .iter()
1348 .map(|row| {
1349 let objects = objects_for(store, row, &index.bases);
1350 let tally = Objects {
1351 expected: objects.pulled.len(),
1352 present: objects.pulled.iter().filter(|path| path.exists()).count(),
1353 };
1354 VerifiedImage {
1355 reference: row.reference.clone(),
1356 objects: tally,
1357 complete: tally.complete(),
1358 }
1359 })
1360 .collect())
1361}
1362
1363/// Refuse if the store root holds something this module cannot classify.
1364fn guard_entries(store: &Path) -> Result<(), StoreError> {
1365 let entries = std::fs::read_dir(store).map_err(|error| StoreError::Io {
1366 action: "reading",
1367 path: store.display().to_string(),
1368 message: error.to_string(),
1369 })?;
1370 for entry in entries.flatten() {
1371 let name = entry.file_name().to_string_lossy().into_owned();
1372 if !KNOWN_ENTRIES.contains(&name.as_str()) {
1373 return Err(StoreError::UnrecognisedEntry { entry: name });
1374 }
1375 }
1376 let images = store.join("images");
1377 if !images.is_dir() {
1378 return Ok(());
1379 }
1380 let entries = std::fs::read_dir(&images).map_err(|error| StoreError::Io {
1381 action: "reading",
1382 path: images.display().to_string(),
1383 message: error.to_string(),
1384 })?;
1385 for entry in entries.flatten() {
1386 let name = entry.file_name().to_string_lossy().into_owned();
1387 if !IMAGE_DIRS.contains(&name.as_str()) {
1388 return Err(StoreError::UnrecognisedEntry {
1389 entry: format!("images/{name}"),
1390 });
1391 }
1392 }
1393 Ok(())
1394}
1395
1396/// Refuse if any base-disk row points outside the store root.
1397///
1398/// [`read_index`] has already made those rows unreachable, so this cannot be what
1399/// prevents a deletion outside the root — it is what makes the situation
1400/// *visible* instead of quietly halving the store's inventory. The usual cause is
1401/// a store that has been moved, where every row is stale at once and the honest
1402/// answer is to say so rather than to clear what is left.
1403fn guard_bases(store: &Path, index: &Index) -> Result<(), StoreError> {
1404 if let Some(path) = index.escaped.first() {
1405 return Err(StoreError::BaseOutsideStore {
1406 path: path.display().to_string(),
1407 root: store.display().to_string(),
1408 });
1409 }
1410 Ok(())
1411}
1412
1413#[cfg(test)]
1414mod tests {
1415 use super::{
1416 Attribution, IMAGE_DIRS, INDEX_DB, SANDBOX_STORE_DIR, Scope, StoreError, blob_name, clear,
1417 image_digest, plan, status,
1418 };
1419 use std::path::PathBuf;
1420
1421 /// The three `boxlite` 0.10.0 tables this module reads, declared as that
1422 /// crate declares them.
1423 ///
1424 /// A restatement, and deliberately a literal one: issue #433's hand-clearing
1425 /// notes describe an `images` table and a `disk-images/` directory, and the
1426 /// live schema is `image_index` / `base_disk` / `base_disk_ref` / `snapshot`.
1427 /// A procedure written against a remembered schema goes stale silently, so
1428 /// what the fixture builds is what was read off the store rather than what
1429 /// anyone recalled about it.
1430 const SCHEMA: &str = "
1431 CREATE TABLE image_index (
1432 reference TEXT PRIMARY KEY NOT NULL,
1433 manifest_digest TEXT NOT NULL,
1434 config_digest TEXT NOT NULL,
1435 layers TEXT NOT NULL,
1436 cached_at TEXT NOT NULL,
1437 complete INTEGER NOT NULL DEFAULT 0
1438 );
1439 CREATE TABLE base_disk (
1440 id TEXT PRIMARY KEY NOT NULL,
1441 source_box_id TEXT NOT NULL,
1442 name TEXT,
1443 kind TEXT NOT NULL CHECK(kind IN ('snapshot', 'clone_base', 'rootfs')),
1444 base_path TEXT NOT NULL,
1445 created_at INTEGER NOT NULL,
1446 json TEXT NOT NULL,
1447 UNIQUE(source_box_id, name)
1448 );
1449 CREATE TABLE box_config (
1450 id TEXT PRIMARY KEY NOT NULL,
1451 name TEXT UNIQUE,
1452 created_at INTEGER NOT NULL,
1453 json TEXT NOT NULL
1454 );
1455 ";
1456
1457 /// A plausible digest for a seed word, so fixtures read like the real store.
1458 fn digest(seed: &str) -> String {
1459 format!("sha256:{}", crate::sha256_hex(seed.as_bytes()))
1460 }
1461
1462 /// An asset root holding a sandbox store, built object by object.
1463 struct Fixture {
1464 root: PathBuf,
1465 }
1466
1467 impl Fixture {
1468 fn new(name: &str) -> Self {
1469 let root = std::env::temp_dir()
1470 .join(format!("rto-exec-sandbox-{name}-{}", std::process::id()));
1471 let _ = std::fs::remove_dir_all(&root);
1472 let store = root.join(SANDBOX_STORE_DIR);
1473 for dir in IMAGE_DIRS {
1474 std::fs::create_dir_all(store.join("images").join(dir)).expect("image dir");
1475 }
1476 std::fs::create_dir_all(store.join("bases")).expect("bases dir");
1477 std::fs::create_dir_all(store.join("db")).expect("db dir");
1478 std::fs::write(store.join(".lock"), []).expect("lock file");
1479 let db = rusqlite::Connection::open(store.join(INDEX_DB)).expect("open index");
1480 db.execute_batch(SCHEMA).expect("index schema");
1481 Self { root }
1482 }
1483
1484 fn store(&self) -> PathBuf {
1485 self.root.join(SANDBOX_STORE_DIR)
1486 }
1487
1488 fn db(&self) -> rusqlite::Connection {
1489 rusqlite::Connection::open(self.store().join(INDEX_DB)).expect("open index")
1490 }
1491
1492 fn write(&self, relative: &str, bytes: usize) -> PathBuf {
1493 let path = self.store().join(relative);
1494 std::fs::create_dir_all(path.parent().expect("a parent")).expect("parent dir");
1495 std::fs::write(&path, vec![b'x'; bytes]).expect("write object");
1496 path
1497 }
1498
1499 /// Add an image: its index row, its pulled objects, its extracted trees
1500 /// and the disk image derived from its layer list.
1501 fn image(&self, reference: &str, layers: &[&str], layer_bytes: usize) {
1502 let manifest = digest(&format!("{reference} manifest"));
1503 let config = digest(&format!("{reference} config"));
1504 let digests: Vec<String> = layers.iter().map(|seed| digest(seed)).collect();
1505 self.db()
1506 .execute(
1507 "INSERT INTO image_index
1508 (reference, manifest_digest, config_digest, layers, cached_at, complete)
1509 VALUES (?1, ?2, ?3, ?4, ?5, 1)",
1510 rusqlite::params![
1511 reference,
1512 manifest,
1513 config,
1514 serde_json::to_string(&digests).expect("layer list"),
1515 "2026-08-19T00:00:00+00:00",
1516 ],
1517 )
1518 .expect("insert image row");
1519 self.write(
1520 &format!("images/manifests/{}.json", blob_name(&manifest)),
1521 64,
1522 );
1523 self.write(&format!("images/configs/{}.json", blob_name(&config)), 32);
1524 for layer in &digests {
1525 self.write(
1526 &format!("images/layers/{}.tar.gz", blob_name(layer)),
1527 layer_bytes,
1528 );
1529 self.write(
1530 &format!("images/extracted/{}/rootfs", blob_name(layer)),
1531 layer_bytes,
1532 );
1533 }
1534 self.write(
1535 &format!(
1536 "images/disk-images/{}.ext4",
1537 blob_name(&image_digest(&digests))
1538 ),
1539 layer_bytes * 4,
1540 );
1541 }
1542
1543 /// The layer tarball a seed word resolves to.
1544 fn layer(&self, seed: &str) -> PathBuf {
1545 self.store()
1546 .join("images/layers")
1547 .join(format!("{}.tar.gz", blob_name(&digest(seed))))
1548 }
1549
1550 /// Add a `base_disk` row and the file it points at.
1551 fn base(&self, id: &str, kind: &str, name: &str) -> PathBuf {
1552 let path = self.write(&format!("bases/{id}.ext4"), 512);
1553 self.db()
1554 .execute(
1555 "INSERT INTO base_disk
1556 (id, source_box_id, name, kind, base_path, created_at, json)
1557 VALUES (?1, '__global__', ?2, ?3, ?4, 0, '{}')",
1558 rusqlite::params![id, name, kind, path.display().to_string()],
1559 )
1560 .expect("insert base row");
1561 path
1562 }
1563 }
1564
1565 /// The disk-image filename is **derived**, and the vector is the live store.
1566 ///
1567 /// `boxlite` keys `images/disk-images/*.ext4` by the SHA-256 of the layer
1568 /// digest strings concatenated in manifest order. Issue #433's hand-clearing
1569 /// notes record that no such association existed and fell back to **mtime**,
1570 /// with the instruction that a shipped `clear` must not. This is the
1571 /// assertion that it does not have to: both vectors are read off the store
1572 /// measured for that issue, and `cimg/rust` is the one that names a digest
1573 /// twice — so a de-duplicating or re-ordering derivation fails here rather
1574 /// than in a wrong deletion.
1575 #[test]
1576 fn the_disk_image_filename_is_derived_from_the_layer_list() {
1577 let debian = [
1578 "sha256:0f5d7465a5bb9d419f60c93d126a161286c73a1ede4a8b2e46bd5e7ad5782cc7".to_owned(),
1579 ];
1580 assert_eq!(
1581 image_digest(&debian),
1582 "sha256:2674b856eab71e6d70f5d8ad573394d1b90f40da02593e3af3a31c17b8de1d97",
1583 "the derivation no longer reproduces the disk image the live store holds for \
1584 docker.io/library/debian:bookworm-slim"
1585 );
1586
1587 let cimg: Vec<String> = [
1588 "c36472b3458398be28ecbfebbaac44143c040eae73411baded48a22060d3055b",
1589 "fee4d731b9208f65a65b57345c4945de0d8eccf9a9f8729e796be8911bd3131c",
1590 "fec6be0b4b4a6668684b8cc97d59c44998ed49004a5e18954caa3f58986549a6",
1591 "943b99e461484cf70776207df97a17783fc424cfeafd5e1bdffab309f42fe84f",
1592 "4dc664574997cda0756a223b9e39e9c4cac313e72dbd59adef0fa723ac8ffc5f",
1593 "cd344ce4edc31b84f799cfd1ff435b61345534535f474fcb8a7f2e9d2ddb209d",
1594 "ab2260fc0eee2ac435fe045b63e2fc28c19cf56cb707101e8eb77601bcf7cdb3",
1595 "4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1",
1596 "5b96641132bf37840e483d28ac60942c3b7b26c2382322c8fa94e62e83b86523",
1597 "4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1",
1598 ]
1599 .iter()
1600 .map(|hex| format!("sha256:{hex}"))
1601 .collect();
1602 assert_eq!(
1603 image_digest(&cimg),
1604 "sha256:b038ce43bcc84d230823ec62558e9a5926f9057206e3eb5ec704da92677fcc0d",
1605 "the derivation no longer reproduces the disk image the live store holds for \
1606 docker.io/cimg/rust, whose layer list names one digest twice"
1607 );
1608 }
1609
1610 /// The index writes `sha256:`; the filesystem writes `sha256-`.
1611 ///
1612 /// One character, and it is why issue #433's first hand-clearing pass matched
1613 /// nothing at all.
1614 #[test]
1615 fn the_on_disk_name_is_not_the_index_digest() {
1616 assert_eq!(blob_name("sha256:abc123"), "sha256-abc123");
1617 }
1618
1619 /// Dropping one image must not take a layer another image still references.
1620 ///
1621 /// The correctness bug this module exists to prevent, and it does not present
1622 /// as an error: it presents as a **surviving image whose blobs are gone**,
1623 /// discovered on some later run. So the assertion is made twice — the shared
1624 /// layer is absent from the doomed set, and the survivor is verified blob by
1625 /// blob after the deletion actually happened.
1626 #[test]
1627 fn a_layer_two_images_share_survives_dropping_one_of_them() {
1628 let fixture = Fixture::new("shared-layer");
1629 fixture.image("registry/a:1", &["common", "only-a"], 4096);
1630 fixture.image("registry/b:1", &["common", "only-b"], 4096);
1631
1632 let scope = Scope::Image("registry/a:1".to_owned());
1633 let (_, doomed) = plan(&fixture.root, &scope).expect("plan");
1634 assert!(
1635 !doomed.contains(&fixture.layer("common")),
1636 "a layer `registry/b:1` still references was put in the doomed set"
1637 );
1638 assert!(
1639 doomed.contains(&fixture.layer("only-a")),
1640 "the layer only the dropped image references was not in the doomed set"
1641 );
1642
1643 let report = clear(&fixture.root, &scope).expect("clear");
1644 assert!(
1645 fixture.layer("common").exists(),
1646 "the shared layer was deleted, so `registry/b:1` is now broken"
1647 );
1648 assert!(!fixture.layer("only-a").exists());
1649 assert_eq!(report.retained.len(), 1);
1650 assert!(
1651 report.survivors_intact(),
1652 "a surviving image is incomplete after the clear: {:?}",
1653 report.retained
1654 );
1655 assert_eq!(report.retained[0].reference, "registry/b:1");
1656 assert_eq!(report.retained[0].objects.expected, 4);
1657 assert_eq!(report.retained[0].objects.present, 4);
1658 }
1659
1660 /// A shared blob is reported as shared, rather than promised back.
1661 ///
1662 /// `total` is what the image references and `exclusive` is what dropping it
1663 /// alone would free. Reporting only the total would offer bytes that a
1664 /// surviving image keeps, which is a `clear` whose number nobody can check.
1665 #[test]
1666 fn the_status_row_separates_what_an_image_references_from_what_it_would_free() {
1667 let fixture = Fixture::new("exclusive-bytes");
1668 fixture.image("registry/a:1", &["common", "only-a"], 4096);
1669 fixture.image("registry/b:1", &["common", "only-b"], 4096);
1670
1671 let report = status(&fixture.root).expect("status");
1672 let image = report
1673 .images
1674 .iter()
1675 .find(|image| image.reference == "registry/a:1")
1676 .expect("the image is listed");
1677 assert!(
1678 image.bytes.exclusive < image.bytes.total,
1679 "a shared layer was counted as reclaimable: {:?}",
1680 image.bytes
1681 );
1682 assert_eq!(image.objects.expected, 4, "manifest, config and two layers");
1683 assert_eq!(image.objects.present, 4);
1684 }
1685
1686 /// `everything` empties the store and the index together, and the two byte
1687 /// figures agree to within the index itself.
1688 ///
1689 /// The rows go with the blobs. Leaving a row behind reports an image whose
1690 /// bytes are gone, which reads as a corrupt store rather than a cleared one —
1691 /// issue #433's fourth trap.
1692 ///
1693 /// It also holds [`super::ClearReport::measured_freed_bytes`]'s claim: the
1694 /// only thing here that can change size without having been removed is the
1695 /// `SQLite` index, which frees pages inside its file rather than shrinking it.
1696 #[test]
1697 fn the_accounted_bytes_and_the_measured_bytes_differ_only_by_the_index() {
1698 let fixture = Fixture::new("everything");
1699 fixture.image("registry/a:1", &["common", "only-a"], 4096);
1700 fixture.image("registry/b:1", &["common", "only-b"], 4096);
1701
1702 let before = status(&fixture.root).expect("status").total_bytes;
1703 let report = clear(&fixture.root, &Scope::Everything).expect("clear");
1704 assert_eq!(report.removed.len(), 2);
1705 assert!(report.retained.is_empty());
1706 assert_eq!(report.store_bytes_before, before);
1707 assert!(
1708 report.store_bytes_after < report.store_bytes_before,
1709 "the store did not shrink"
1710 );
1711 let index = super::size_of(&fixture.store().join("db"));
1712 assert!(
1713 report.freed_bytes.abs_diff(report.measured_freed_bytes()) <= index,
1714 "the accounted bytes ({}) and the bytes the filesystem gave back ({}) differ by \
1715 more than the index ({index}) — in either direction, which is the only thing in \
1716 the store that can change size without having been removed",
1717 report.freed_bytes,
1718 report.measured_freed_bytes()
1719 );
1720
1721 let after = status(&fixture.root).expect("status");
1722 assert!(
1723 after.images.is_empty(),
1724 "the index still reports images whose blobs are gone: {:?}",
1725 after.images
1726 );
1727 }
1728
1729 /// An index manifest a survivor resolves through is kept.
1730 ///
1731 /// A tag reference — `debian:bookworm-slim` in the live store — records only
1732 /// the platform manifest it resolved to, never the index digest above it. A
1733 /// rule of "delete every manifest file no row names" would therefore take a
1734 /// surviving image's index with it, so the association is read out of the
1735 /// files themselves.
1736 #[test]
1737 fn an_index_manifest_a_survivor_resolves_through_is_kept() {
1738 let fixture = Fixture::new("index-manifest");
1739 fixture.image("registry/a:1", &["only-a"], 4096);
1740 fixture.image("registry/b:1", &["only-b"], 4096);
1741
1742 let survivor = digest("registry/b:1 manifest");
1743 let dropped = digest("registry/a:1 manifest");
1744 let keep = fixture.write("images/manifests/sha256-keep.json", 0);
1745 std::fs::write(
1746 &keep,
1747 serde_json::json!({ "manifests": [{ "digest": survivor }] }).to_string(),
1748 )
1749 .expect("write index manifest");
1750 let go = fixture.write("images/manifests/sha256-go.json", 0);
1751 std::fs::write(
1752 &go,
1753 serde_json::json!({ "manifests": [{ "digest": dropped }] }).to_string(),
1754 )
1755 .expect("write index manifest");
1756
1757 // The load-bearing assertion, and it is this one rather than "the file is
1758 // still there after a per-image clear" — nothing in a per-image clear
1759 // could ever reach it, so that assertion would pass with the association
1760 // removed entirely. What the association actually decides is whether these
1761 // files are **claimed**: an index manifest nobody claims is spare, and
1762 // `everything` removes spare bytes.
1763 let before = status(&fixture.root).expect("status");
1764 assert!(
1765 before.unattributed.is_empty(),
1766 "an index manifest a cached image resolves through was reported as \
1767 unattributed, which is one `--everything` away from being deleted while its \
1768 image survives: {:?}",
1769 before.unattributed
1770 );
1771
1772 clear(&fixture.root, &Scope::Image("registry/a:1".to_owned())).expect("clear");
1773 assert!(!go.exists(), "the dropped image's index manifest was kept");
1774 let after = status(&fixture.root).expect("status");
1775 assert!(
1776 after.unattributed.is_empty(),
1777 "the survivor's index manifest stopped being attributed once the other image \
1778 was dropped: {:?}",
1779 after.unattributed
1780 );
1781 assert!(
1782 keep.exists(),
1783 "the index manifest `registry/b:1` resolves through was deleted"
1784 );
1785 }
1786
1787 /// Something under the store root this module cannot classify stops the clear.
1788 ///
1789 /// ADR-0014 v1.6's limit, enforced rather than trusted: the verb may drop what
1790 /// a pinned digest re-obtains and nothing else, and turning up in a cache is
1791 /// not evidence of being re-obtainable. Nothing else is cleared either — a
1792 /// partial clear alongside a refusal is the worst of both.
1793 #[test]
1794 fn an_unrecognised_entry_stops_the_clear_without_removing_anything() {
1795 let fixture = Fixture::new("unrecognised");
1796 fixture.image("registry/a:1", &["only-a"], 4096);
1797 std::fs::create_dir_all(fixture.store().join("provenance")).expect("mystery dir");
1798
1799 let error = clear(&fixture.root, &Scope::Everything)
1800 .expect_err("an entry this module cannot classify must stop the clear");
1801 assert!(
1802 matches!(&error, StoreError::UnrecognisedEntry { entry } if entry == "provenance"),
1803 "expected the unrecognised entry to be named, got: {error}"
1804 );
1805 assert!(
1806 fixture.layer("only-a").exists(),
1807 "the refusal removed objects on its way out"
1808 );
1809 }
1810
1811 /// A `base_disk` row cannot aim a deletion outside the store root.
1812 ///
1813 /// `base_path` is an absolute path recorded when the base was built, so it is
1814 /// data rather than a derivation — and data can name anywhere, including the
1815 /// repository store this verb must never reach.
1816 #[test]
1817 fn a_base_disk_row_pointing_outside_the_store_is_refused() {
1818 let fixture = Fixture::new("escape");
1819 fixture.image("registry/a:1", &["only-a"], 4096);
1820 let outside = fixture.root.join("graph.db");
1821 std::fs::write(&outside, b"not re-obtainable").expect("write");
1822 fixture
1823 .db()
1824 .execute(
1825 "INSERT INTO base_disk
1826 (id, source_box_id, name, kind, base_path, created_at, json)
1827 VALUES ('esc', '__global__', 'escapee', 'rootfs', ?1, 0, '{}')",
1828 rusqlite::params![outside.display().to_string()],
1829 )
1830 .expect("insert base row");
1831
1832 let error = clear(&fixture.root, &Scope::Everything)
1833 .expect_err("a base disk outside the store root must stop the clear");
1834 assert!(
1835 matches!(&error, StoreError::BaseOutsideStore { path, .. } if path.contains("graph.db")),
1836 "expected the escaping path to be named, got: {error}"
1837 );
1838 assert!(outside.exists(), "the clear reached outside the store root");
1839
1840 // And `status` does not reach it either. The containment is at the point
1841 // the row is read, not at the point it is deleted, so a path outside the
1842 // root is never measured or listed — only named as something left alone.
1843 let report = status(&fixture.root).expect("status");
1844 assert!(
1845 report
1846 .preserved
1847 .iter()
1848 .any(|entry| entry.path.contains("graph.db")),
1849 "the escaping row was not named in the status document"
1850 );
1851 let image = &report.images[0];
1852 assert_eq!(
1853 image.bytes.base_disk, 0,
1854 "a file outside the store root was measured into an image's size"
1855 );
1856 assert!(!image.base_disk_built);
1857 }
1858
1859 /// The bytes reported are the bytes `du` reports, because the store is sparse.
1860 ///
1861 /// The ext4 disks are sparse by a margin nobody would call rounding: in the
1862 /// store measured for issue #433, `sha256-0a89fbeb….ext4` is 1.23 GiB long and
1863 /// occupies 1.09 GiB, and the base rootfs is 256 MiB long and occupies 110
1864 /// MiB. A `clear` that reported lengths would claim 9.1 GiB freed where `du`
1865 /// says 8.7 GiB — and the one number this verb exists to produce would be the
1866 /// one number in it that could not be checked.
1867 #[cfg(unix)]
1868 #[test]
1869 fn a_sparse_disk_image_is_counted_by_what_it_occupies() {
1870 use std::io::{Seek as _, Write as _};
1871
1872 let fixture = Fixture::new("sparse");
1873 let path = fixture.write("images/disk-images/sha256-sparse.ext4", 0);
1874 let mut file = std::fs::File::create(&path).expect("create");
1875 file.seek(std::io::SeekFrom::Start(64 << 20)).expect("seek");
1876 file.write_all(b"end").expect("write");
1877 drop(file);
1878
1879 let apparent = std::fs::metadata(&path).expect("metadata").len();
1880 let occupied = super::size_of(&path);
1881 assert!(
1882 apparent > 64 << 20,
1883 "the fixture file is not long enough to be worth measuring"
1884 );
1885 assert!(
1886 occupied < apparent,
1887 "a sparse file was counted by its length ({apparent}) rather than by what it \
1888 occupies ({occupied}), so a clear would over-report what it freed"
1889 );
1890 }
1891
1892 /// A registered box blocks the clear.
1893 ///
1894 /// `boxlite` holds an exclusive `flock` on `<store>/.lock` for a runtime's
1895 /// lifetime, which this crate cannot take back — `unsafe_code = "forbid"`
1896 /// rules out the call that acquires it. So the guard is the evidence such a
1897 /// lock would have protected, and it is checked rather than assumed absent.
1898 #[test]
1899 fn a_registered_box_blocks_the_clear() {
1900 let fixture = Fixture::new("live-box");
1901 fixture.image("registry/a:1", &["only-a"], 4096);
1902 fixture
1903 .db()
1904 .execute(
1905 "INSERT INTO box_config (id, name, created_at, json)
1906 VALUES ('box1', 'running', 0, '{}')",
1907 [],
1908 )
1909 .expect("insert box row");
1910
1911 let error = clear(&fixture.root, &Scope::Everything)
1912 .expect_err("a registered box must stop the clear");
1913 assert!(
1914 matches!(error, StoreError::LiveBoxes { boxes: 1 }),
1915 "expected a live-box refusal, got: {error}"
1916 );
1917 assert!(fixture.layer("only-a").exists());
1918 }
1919
1920 /// A snapshot base disk is preserved, named, and never counted as spare.
1921 ///
1922 /// A `rootfs` base is rebuilt from its image; a `snapshot` is the state of a
1923 /// box somebody ran, and no digest re-obtains it. It is referenced by no
1924 /// image, which is exactly why an unclaimed-means-spare rule would delete it.
1925 #[test]
1926 fn a_snapshot_base_disk_is_preserved_rather_than_treated_as_spare() {
1927 let fixture = Fixture::new("snapshot");
1928 fixture.image("registry/a:1", &["only-a"], 4096);
1929 let snapshot = fixture.base("snap1", "snapshot", "a-snapshot");
1930
1931 let before = status(&fixture.root).expect("status");
1932 assert!(
1933 !before
1934 .unattributed
1935 .iter()
1936 .any(|entry| snapshot.ends_with(&entry.path)),
1937 "a snapshot base disk was reported as unattributed bytes"
1938 );
1939 assert_eq!(before.preserved.len(), 1);
1940
1941 let report = clear(&fixture.root, &Scope::Everything).expect("clear");
1942 assert!(
1943 snapshot.exists(),
1944 "the clear removed a snapshot, which no digest re-obtains"
1945 );
1946 assert_eq!(report.preserved.len(), 1);
1947 }
1948
1949 /// A per-image request that names nothing in the store says what is in it.
1950 ///
1951 /// The likely cause is a tag typed where the index holds a digest reference,
1952 /// so the listing is the way forward rather than a courtesy.
1953 #[test]
1954 fn an_unknown_reference_names_what_the_store_is_holding() {
1955 let fixture = Fixture::new("unknown-image");
1956 fixture.image("registry/a:1", &["only-a"], 4096);
1957
1958 let error = clear(&fixture.root, &Scope::Image("registry/a:2".to_owned()))
1959 .expect_err("a reference the store is not holding must be refused");
1960 assert!(
1961 matches!(&error, StoreError::UnknownImage { known, .. } if known == "registry/a:1"),
1962 "expected the cached references to be named, got: {error}"
1963 );
1964 }
1965
1966 /// Bytes no index row claims survive a per-image clear and go with everything.
1967 ///
1968 /// The live store had one: a 146 MB extracted layer tree whose tarball and
1969 /// index row are both gone. A per-image request has no evidence it belongs to
1970 /// the image it named, so it is reported rather than deleted on a guess.
1971 #[test]
1972 fn unattributed_bytes_survive_a_per_image_clear_and_go_with_everything() {
1973 let fixture = Fixture::new("unattributed");
1974 fixture.image("registry/a:1", &["only-a"], 4096);
1975 fixture.image("registry/b:1", &["only-b"], 4096);
1976 let orphan = fixture.write("images/extracted/sha256-orphan/rootfs", 8192);
1977
1978 let before = status(&fixture.root).expect("status");
1979 assert_eq!(before.attribution, Attribution::Partial);
1980 assert_eq!(before.unattributed.len(), 1);
1981 assert!(before.unattributed[0].path.ends_with("sha256-orphan"));
1982 // At least the tree it holds, and deliberately not an exact figure: this
1983 // entry is a **directory**, and a directory's own allocation is a property
1984 // of the filesystem rather than of the store. APFS reports none for one;
1985 // ext4 reports a 4 KiB block, so an `== 8192` here passed on the machine
1986 // this was written on and failed on CI.
1987 assert!(
1988 before.unattributed[0].bytes >= 8192,
1989 "the unattributed tree was measured as smaller than the file in it: {:?}",
1990 before.unattributed[0]
1991 );
1992
1993 clear(&fixture.root, &Scope::Image("registry/a:1".to_owned())).expect("clear");
1994 assert!(
1995 orphan.exists(),
1996 "a per-image clear deleted bytes it could not attribute to that image"
1997 );
1998
1999 let report = clear(&fixture.root, &Scope::Everything).expect("clear");
2000 assert!(
2001 !orphan.exists(),
2002 "`everything` left unattributed bytes behind"
2003 );
2004 assert_eq!(report.removed_unattributed.len(), 1);
2005 }
2006
2007 /// An absent store is an answer, not a failure.
2008 #[test]
2009 fn an_absent_store_is_reported_rather_than_failing() {
2010 let root =
2011 std::env::temp_dir().join(format!("rto-exec-sandbox-absent-{}", std::process::id()));
2012 let _ = std::fs::remove_dir_all(&root);
2013 let report = status(&root).expect("status");
2014 assert!(!report.present);
2015 assert!(report.images.is_empty());
2016 assert_eq!(report.total_bytes, 0);
2017
2018 let cleared = clear(&root, &Scope::Everything).expect("clear");
2019 assert_eq!(cleared.freed_bytes, 0);
2020 assert!(cleared.applied);
2021 }
2022
2023 /// This module and `boxlite.rs` name the same directory.
2024 ///
2025 /// They cannot share the constant: `boxlite.rs` is behind `exec-boxlite`, and
2026 /// making that feature a condition of *clearing* a store some earlier build
2027 /// filled is the shape of the bootstrap problem that moved provisioning off
2028 /// the backend features in the first place. So the two literals are checked
2029 /// against each other instead, by reading the source — the only way to compare
2030 /// a constant with a `join` argument.
2031 #[test]
2032 fn the_store_directory_is_the_one_boxlite_is_pointed_at() {
2033 let source = include_str!("boxlite.rs");
2034 let marker = "home_dir: assets_root.join(\"";
2035 let named: Vec<&str> = source
2036 .match_indices(marker)
2037 .map(|(at, _)| {
2038 source[at + marker.len()..]
2039 .split_once('"')
2040 .expect("a join argument is closed on the same line")
2041 .0
2042 })
2043 .collect();
2044 assert!(
2045 !named.is_empty(),
2046 "no `home_dir: assets_root.join(..)` was found to check against"
2047 );
2048 for directory in named {
2049 assert_eq!(
2050 directory, SANDBOX_STORE_DIR,
2051 "boxlite.rs points a runtime at `{directory}` and this module clears \
2052 `{SANDBOX_STORE_DIR}`"
2053 );
2054 }
2055 }
2056
2057 /// The document says whose store it is describing.
2058 ///
2059 /// One sandbox store per asset root, shared by every repository on the host —
2060 /// the same hazard `MachineScope` carries a `scope` field for, and the same
2061 /// remedy: a caller who selected a project must not read a machine-global
2062 /// figure as that project's.
2063 #[test]
2064 fn the_status_document_labels_its_scope_as_the_machine() {
2065 let fixture = Fixture::new("scope");
2066 fixture.image("registry/a:1", &["only-a"], 16);
2067 let document =
2068 serde_json::to_value(status(&fixture.root).expect("status")).expect("serialise");
2069 assert_eq!(document["scope"], "machine");
2070 assert_eq!(document["schema"], super::SANDBOX_STATUS_SCHEMA);
2071 assert!(
2072 document["store"]
2073 .as_str()
2074 .expect("a store path")
2075 .ends_with(SANDBOX_STORE_DIR)
2076 );
2077 }
2078}