fs_transaction/change.rs
1//! Transactional writes — the unit every change lands through.
2//!
3//! Changing a set of linked files is rarely a single-file operation. A rename
4//! that keeps backlinks intact has to rewrite every file that pointed at the
5//! moved one; a move between directories has to re-relativize the links inside
6//! the file it moved. What is logically one edit is physically several, and
7//! issued one at a time an I/O failure partway through the burst leaves the
8//! tree torn: updated in the files already written, stale in the ones never
9//! reached.
10//!
11//! A [`ChangeSet`] closes that window. A caller stages its writes into one
12//! instead of issuing them, and [`ChangeSet::apply`] executes the whole set as
13//! a unit: each op records how to undo itself *at the moment it runs*, and the
14//! first failure unwinds every op already applied, in reverse. Either the whole
15//! set lands or the tree is as it was.
16//!
17//! ## What this does and does not buy
18//!
19//! This is **error** atomicity and **crash** atomicity across the whole set. A
20//! failed write, a full disk, a permission error, or a rejected edit cannot
21//! leave the tree half-updated, because unwinding puts back every op already
22//! applied. And no single file can be caught half-written even by a power cut:
23//! every [`FileOp::Write`] lands through [`Storage::write_atomic`], which
24//! stages the new bytes in a temporary sibling, flushes them, and renames it
25//! over the target, so an observer sees the whole old file or the whole new
26//! one, never a splice. A `kill -9` or power cut *between* ops leaves the
27//! journal behind; [`crate::journal::recover`] replays it forward to the
28//! fully-applied state. The distinction is worth keeping sharp: caught errors
29//! abort back to the pre-change state, while crashes recover to the committed
30//! state.
31//!
32//! Both answers are **durable**, not merely consistent. `Ok` means the whole
33//! set — the renames and removals as much as the writes — survives a power
34//! cut from that moment on: every write settles its own flush through
35//! [`Storage::write_atomic`] as it lands, and everything else the set dirtied
36//! (directory entries, execute bits, freshly-minted directory chains) is
37//! flushed before the journal is given up. `Err` from a clean rollback means
38//! the abort is equally a fact: the restored state is flushed and the
39//! journal's deletion made durable, so no later recovery can quietly roll the
40//! aborted set forward. The one answer that promises less is
41//! [`Error::Torn`], which says so.
42//!
43//! Two smaller honesties, both deliberate:
44//!
45//! - **Directories are not unwound.** Applying a set creates any parent
46//! directory its writes need; a rollback leaves an empty one behind. An empty
47//! directory is litter, not a torn tree.
48//! - **Undo is held in memory.** Overwriting or removing a file reads its old
49//! bytes first so the rollback can put them back, which means a removed
50//! payload is briefly held whole. The buffer lives only for the length of the
51//! apply, but it does mean a set is bounded by what fits in memory —
52//! [`FileOp::CopyFrom`] is the escape hatch for a large payload already on
53//! disk.
54//!
55//! ## Staging is also a plan
56//!
57//! Because a set is a value that describes writes without performing them, it
58//! is equally an answer to "what *would* this do?" — the shape a `--dry-run`
59//! needs. [`ChangeSet::ops`] is that view, and it is the same sequence `apply`
60//! will execute rather than a reconstruction of it.
61//!
62//! ## A set can expect
63//!
64//! A set is computed from a reading of the tree, and the tree may have moved
65//! between that reading and the apply. [`ChangeSet::expect`] stages the
66//! reading itself — the bytes a path held when the caller looked, or
67//! [its absence](ChangeSet::expect_absent) — and `apply` checks every
68//! expectation against the tree as it finds it, after the stale-journal
69//! refusal and before the commit point. One that does not hold refuses the
70//! whole set with [`Error::Drifted`] *before* anything is written, journaled,
71//! or unwound: the caller re-reads, restages, and retries, which is
72//! optimistic concurrency in exactly the compare-and-swap sense.
73//!
74//! Expectations speak of the tree **before** the set runs, so a set may
75//! expect a path absent and then write it. And they are never journaled: the
76//! journal is written only once they have held, so recovery completes an
77//! interrupted set unconditionally rather than re-litigating a question the
78//! commit point already answered — a recovered tree would fail its own set's
79//! expectations by construction, having half-applied them.
80//!
81//! ## Single writer
82//!
83//! A set assumes it is the only thing mutating the tree while it applies. There
84//! is no locking here: two processes applying sets against the same root will
85//! race on the journal, and the [`Error::StaleJournal`] check that guards
86//! against a *previous* interrupted change is a check-then-act, not a mutex. A
87//! caller that needs several writers has to serialize them itself.
88//! [Expectations](ChangeSet::expect) narrow this window rather than close
89//! it — the check is itself a check-then-act, and what it detects is a writer
90//! that raced the gap between the caller's *read* and this apply, which is
91//! the far wider gap in practice.
92
93use std::collections::BTreeSet;
94use std::path::{Path, PathBuf};
95
96use crate::error::{Error, Result};
97use crate::fs::Storage;
98use crate::journal::Journal;
99use crate::path::guard_in_root;
100
101/// One staged filesystem operation. Paths are **root-relative** — the root
102/// is joined on at [`apply`](ChangeSet::apply) time, so a set is portable
103/// between trees and prints readably in a dry run.
104#[derive(Debug, Clone, PartialEq, Eq)]
105pub enum FileOp {
106 /// Write `bytes` to `path`, creating it (and any missing parent directory)
107 /// or replacing it wholesale.
108 Write {
109 /// The file to write.
110 path: PathBuf,
111 /// Its full new contents.
112 bytes: Vec<u8>,
113 },
114 /// Move `from` to `to`, creating any missing parent directory of `to`.
115 Rename {
116 /// The current path.
117 from: PathBuf,
118 /// The new path.
119 to: PathBuf,
120 },
121 /// Remove the file at `path`. It must exist.
122 Remove {
123 /// The file to remove.
124 path: PathBuf,
125 },
126 /// Copy the bytes already on disk at `source` to `path`, verbatim.
127 ///
128 /// [`Write`](FileOp::Write) with the payload left where it lies. The journal
129 /// records the *source path* instead of the bytes, so a set that writes a
130 /// large payload costs O(path) of journal rather than a second copy of every
131 /// byte — which is what makes putting a whole captured tree back
132 /// tractable, where `Write` would duplicate every byte of it into the
133 /// journal at the commit point.
134 ///
135 /// **The source must be immutable for the lifetime of the change**, because
136 /// that is the entire correctness argument. A `Write` journals the exact bytes
137 /// it intends, so replay after a crash is deterministic by construction; a
138 /// `CopyFrom` journals a reference, and replay is deterministic only if the
139 /// referent cannot have changed underneath it. A content-addressed blob
140 /// satisfies this by definition — its path *is* the digest of its
141 /// contents, so bytes found there are the bytes intended, or the file is gone
142 /// and replay fails loudly. Do not point this at a mutable file, at a
143 /// path some other op in the same set writes, or at anything outside the
144 /// root.
145 ///
146 /// This bounds *journal* growth, not peak memory: rollback still buffers the
147 /// bytes it overwrites, exactly as `Write` does.
148 CopyFrom {
149 /// The file to write.
150 path: PathBuf,
151 /// The root-relative file to copy from — immutable, and ideally
152 /// content-addressed.
153 source: PathBuf,
154 },
155 /// Make the file at `path` runnable, or not — one bit, not a mode.
156 ///
157 /// Lands through [`Storage::set_executable`], whose default is the honest
158 /// no-op for a backend with no such bit: over such a backend the op
159 /// "applies" as nothing, which is the same nothing the bit's absence
160 /// already means there. Undo is captured through
161 /// [`ReadStorage::executable`](crate::fs::ReadStorage::executable), so a
162 /// bit that was already in the requested state rolls back to itself rather
163 /// than to its opposite.
164 ///
165 /// A path holding a symbolic link is **refused**
166 /// ([`InvalidInput`](std::io::ErrorKind::InvalidInput)), whoever made the
167 /// link: mode writes follow links, so the bit would land on the link's
168 /// referent — wherever it points, the lexical root guard notwithstanding.
169 SetExecutable {
170 /// The file whose execute bit is set or cleared.
171 path: PathBuf,
172 /// Whether the file should be runnable afterwards.
173 executable: bool,
174 },
175 /// Place a symbolic link at `path` pointing at `target`, replacing
176 /// whatever is there.
177 ///
178 /// The target is **recorded, never resolved**: it may point outside the
179 /// root, at nothing, or at another link, and staging it writes nothing
180 /// through it — the same terms as [`Storage::set_link`]. What is *not*
181 /// permitted is another op in the same set addressing a path that
182 /// traverses this link: the root guard is lexical, and a set that writes
183 /// through its own fresh link is writing wherever the link points.
184 ///
185 /// Over a backend that models no links the op is refused
186 /// ([`Unsupported`](std::io::ErrorKind::Unsupported)) and the set unwinds:
187 /// unlike an execute bit, a link has no honest substitute.
188 SetLink {
189 /// Where the link itself lives.
190 path: PathBuf,
191 /// What it points at — recorded as given.
192 target: PathBuf,
193 },
194}
195
196impl FileOp {
197 /// The path this op ultimately affects — the destination for a write or a
198 /// rename, the victim for a remove. What a dry run lists.
199 pub fn path(&self) -> &Path {
200 match self {
201 FileOp::Write { path, .. }
202 | FileOp::CopyFrom { path, .. }
203 | FileOp::SetExecutable { path, .. }
204 | FileOp::SetLink { path, .. } => path,
205 FileOp::Rename { to, .. } => to,
206 FileOp::Remove { path } => path,
207 }
208 }
209}
210
211/// What one [expectation](ChangeSet::expect) says the tree holds at a path,
212/// checked against the tree as it stands when [`apply`](ChangeSet::apply)
213/// begins — before the set writes anything.
214#[derive(Debug, Clone, PartialEq, Eq)]
215pub enum Expected {
216 /// The path holds exactly these bytes, read as the backend reads — which
217 /// is *through* a symbolic link, if one stands there.
218 Bytes(Vec<u8>),
219 /// Nothing is at the path: no file, no directory, and no link — a
220 /// dangling link is an entry, and counts as occupied.
221 Absent,
222}
223
224/// A set of writes staged as one unit, applied all-or-nothing by
225/// [`apply`](ChangeSet::apply).
226///
227/// Built by the mutation ops as they compute their edits, and applied once at
228/// the end. Ops execute in the order they were staged: a set is a *sequence*,
229/// not a bag, because `rename`-then-write and remove-then-rewrite-the-parent
230/// depend on it.
231#[derive(Debug, Clone, Default, PartialEq, Eq)]
232pub struct ChangeSet {
233 ops: Vec<FileOp>,
234 expected: Vec<(PathBuf, Expected)>,
235}
236
237impl ChangeSet {
238 /// An empty set.
239 pub fn new() -> Self {
240 Self::default()
241 }
242
243 /// Stage a write of `contents` to `path` (root-relative).
244 pub fn write(&mut self, path: impl Into<PathBuf>, contents: impl Into<Vec<u8>>) -> &mut Self {
245 self.ops.push(FileOp::Write {
246 path: path.into(),
247 bytes: contents.into(),
248 });
249 self
250 }
251
252 /// Stage a move from `from` to `to` (both root-relative).
253 pub fn rename(&mut self, from: impl Into<PathBuf>, to: impl Into<PathBuf>) -> &mut Self {
254 self.ops.push(FileOp::Rename {
255 from: from.into(),
256 to: to.into(),
257 });
258 self
259 }
260
261 /// Stage the removal of `path` (root-relative).
262 pub fn remove(&mut self, path: impl Into<PathBuf>) -> &mut Self {
263 self.ops.push(FileOp::Remove { path: path.into() });
264 self
265 }
266
267 /// Stage a copy of the file at `source` to `path` (both root-relative),
268 /// instead of carrying its bytes through the set.
269 ///
270 /// See [`FileOp::CopyFrom`] for the immutability the source has to satisfy —
271 /// it is what keeps crash recovery deterministic.
272 pub fn copy_from(&mut self, path: impl Into<PathBuf>, source: impl Into<PathBuf>) -> &mut Self {
273 self.ops.push(FileOp::CopyFrom {
274 path: path.into(),
275 source: source.into(),
276 });
277 self
278 }
279
280 /// Stage making the file at `path` (root-relative) runnable, or not.
281 pub fn set_executable(&mut self, path: impl Into<PathBuf>, executable: bool) -> &mut Self {
282 self.ops.push(FileOp::SetExecutable {
283 path: path.into(),
284 executable,
285 });
286 self
287 }
288
289 /// Stage a symbolic link at `path` (root-relative) pointing at `target`,
290 /// replacing whatever is there.
291 ///
292 /// See [`FileOp::SetLink`] for what the target is and is not: recorded,
293 /// never resolved, and not a door for other ops in the set to write
294 /// through.
295 pub fn set_link(&mut self, path: impl Into<PathBuf>, target: impl Into<PathBuf>) -> &mut Self {
296 self.ops.push(FileOp::SetLink {
297 path: path.into(),
298 target: target.into(),
299 });
300 self
301 }
302
303 /// Expect `path` (root-relative) to hold exactly `contents` when the set
304 /// applies — the bytes the caller read when it computed this set.
305 ///
306 /// Checked before the commit point; a mismatch (including the file being
307 /// gone) refuses the whole set with [`Error::Drifted`] and nothing is
308 /// written. See the [module docs](self#a-set-can-expect) for when
309 /// expectations are checked and what they do and do not guard against.
310 pub fn expect(&mut self, path: impl Into<PathBuf>, contents: impl Into<Vec<u8>>) -> &mut Self {
311 self.expected
312 .push((path.into(), Expected::Bytes(contents.into())));
313 self
314 }
315
316 /// Expect nothing to be at `path` (root-relative) when the set applies —
317 /// the guard for a create that must not overwrite what a racing writer
318 /// put there first.
319 ///
320 /// An expectation speaks of the tree *before* the set runs, so expecting
321 /// a path absent and then writing that same path is the ordinary use, not
322 /// a contradiction.
323 pub fn expect_absent(&mut self, path: impl Into<PathBuf>) -> &mut Self {
324 self.expected.push((path.into(), Expected::Absent));
325 self
326 }
327
328 /// The staged ops, in execution order. The dry-run view.
329 pub fn ops(&self) -> &[FileOp] {
330 &self.ops
331 }
332
333 /// The staged expectations, in the order staged — the dry-run view's
334 /// other half: what the set demands of the tree, next to what it
335 /// [does](Self::ops) to it.
336 pub fn expected(&self) -> &[(PathBuf, Expected)] {
337 &self.expected
338 }
339
340 /// The bytes this set will leave at `path`, if it writes it — the *last*
341 /// write staged, since a later one supersedes an earlier.
342 ///
343 /// This is what makes a set safe to read back mid-build. A document can be
344 /// touched twice by one op (`reparent` repoints a child that is somehow its
345 /// own old parent, and must then edit the text it just staged rather than the
346 /// stale copy on disk), and before staging existed the second edit read the
347 /// first one's *write* off the filesystem. Nothing hits the filesystem now
348 /// until commit, so the set has to answer instead.
349 ///
350 /// `None` if the set does not write `path` — including when it renames or
351 /// removes it, and including a [`FileOp::CopyFrom`], whose bytes are on disk
352 /// at the source rather than held in the set. This is deliberately a lookup,
353 /// not a filesystem overlay: it resolves the one hazard staging introduces and
354 /// nothing more. A caller that must read back a path it staged a copy to has
355 /// to read the source itself.
356 pub fn staged(&self, path: &Path) -> Option<&[u8]> {
357 self.ops.iter().rev().find_map(|op| match op {
358 FileOp::Write { path: p, bytes } if p == path => Some(bytes.as_slice()),
359 _ => None,
360 })
361 }
362
363 /// Where this set moves `path` to, if it moves it — following a chain of
364 /// renames to the final destination. `None` if the set leaves it where it is.
365 ///
366 /// The companion to [`staged`](Self::staged) for anything holding a path this
367 /// set might move out from under it. The registry is exactly that: it knows
368 /// which document it persists into, and a set that renames that document has
369 /// to be followed, or its write lands at a path the set just emptied.
370 pub fn renamed_to(&self, path: &Path) -> Option<PathBuf> {
371 let mut current = path.to_path_buf();
372 let mut moved = false;
373 for op in &self.ops {
374 if let FileOp::Rename { from, to } = op
375 && *from == current
376 {
377 current = to.clone();
378 moved = true;
379 }
380 }
381 moved.then_some(current)
382 }
383
384 /// Whether nothing is staged — no ops and no
385 /// [expectations](Self::expect) — so [`apply`](ChangeSet::apply) would be
386 /// a no-op.
387 pub fn is_empty(&self) -> bool {
388 self.ops.is_empty() && self.expected.is_empty()
389 }
390
391 /// The number of staged ops.
392 pub fn len(&self) -> usize {
393 self.ops.len()
394 }
395
396 /// Append `other`'s ops after this set's, consuming it. Its expectations
397 /// come along too — they still speak of the pre-apply tree, exactly as
398 /// they did in the set that staged them.
399 pub fn extend(&mut self, other: ChangeSet) -> &mut Self {
400 self.ops.extend(other.ops);
401 self.expected.extend(other.expected);
402 self
403 }
404
405 /// Execute every staged op against `fs`, rooted at `root`, as one unit —
406 /// crash-atomically, behind the [default](Journal::DEFAULT_NAME)
407 /// write-ahead journal.
408 ///
409 /// Shorthand for [`Journal::default().apply(..)`](Journal::apply); reach
410 /// for a named [`Journal`] when the default file name would collide with
411 /// something the tree already means. Whichever is used here, the same one
412 /// has to be used to [`recover`](Journal::recover) an interruption of it.
413 pub async fn apply<FS: Storage>(&self, fs: &FS, root: &Path) -> Result<()> {
414 Journal::default().apply(self, fs, root).await
415 }
416}
417
418impl Journal {
419 /// Execute every op `changes` staged against `fs`, rooted at `root`, as one
420 /// unit — crash-atomically, behind this write-ahead journal.
421 ///
422 /// The set's intent is journaled and flushed *before* any document is
423 /// touched (see [`crate::journal`]); that flush is the commit point. From
424 /// there the ops run in order, each recording how to undo itself:
425 ///
426 /// - **On success**, everything the set dirtied is flushed durable —
427 /// barriers capped by one drain — and only then is the journal removed:
428 /// `Ok` means the change survives a power cut, not merely that it
429 /// happened.
430 /// - **On an error** (a full disk, a permission fault), every op already
431 /// applied is unwound in reverse, the restored state is flushed, and the
432 /// journal is durably cleared — the mutation aborts as if it never
433 /// began, and a power cut cannot contradict the abort by resurrecting
434 /// the journal for recovery to roll forward.
435 /// - **On a crash** (a `kill -9`, a power cut) there is no error to catch and
436 /// no chance to unwind, so the journal simply survives; the next
437 /// [`crate::journal::recover`] rolls the set forward to its fully-applied
438 /// state. An interrupted change set is therefore always resolved to a
439 /// consistent tree — fully before it on a caught error, fully after it
440 /// on a crash.
441 ///
442 /// A set of **one** op skips the journal entirely: a single op is already
443 /// indivisible on a backend claiming `atomic_replace`, so there is no
444 /// multi-file window for a journal to close, and a crash leaves the op either
445 /// wholly done or wholly not — the same two states a recovered set lands on.
446 /// It is the ordinary shape of a save, and it costs one file operation rather
447 /// than four. What it does not skip is durability: a lone rename or remove
448 /// still flushes the entries it edited before `Ok`, on the same promise a
449 /// journaled set keeps.
450 ///
451 /// The rare exception is a rollback that *itself* fails ([`Error::Torn`]):
452 /// the pre-change state could not be restored, so — rather than leave an
453 /// unknown one — the journal is kept, and recovery will later roll the set
454 /// forward to the consistent applied state. Either way the tree lands on
455 /// a state this crate can name.
456 ///
457 /// Takes `fs`/`root` rather than a higher-level object so a bootstrap
458 /// that must write two files before the tree exists can still land them
459 /// together.
460 ///
461 /// Whatever recovers an interruption of this call must name the same
462 /// journal — see [`Journal`] for why the two operations live together.
463 pub async fn apply<FS: Storage>(
464 &self,
465 changes: &ChangeSet,
466 fs: &FS,
467 root: &Path,
468 ) -> Result<()> {
469 if changes.is_empty() {
470 return Ok(());
471 }
472 // Clamp every staged path to the root *before* anything is
473 // written or journaled. A set is built from root-relative,
474 // already-normalized paths when a caller builds them that way — but
475 // `apply` also lands sets assembled from data it did not author,
476 // and a link target that resolves to `../../../etc/passwd` must be refused
477 // rather than let an apply write outside the tree it was pointed at.
478 // Expectation paths are clamped on the same terms: they are read, and
479 // a set must no more be able to probe `../../../etc/passwd` than to
480 // write it.
481 guard_ops(&changes.ops)?;
482 for (path, _) in &changes.expected {
483 guard_in_root(path)?;
484 }
485 // Refuse to clobber a journal left by a *previous* interrupted change. Its
486 // presence means an earlier mutation crashed mid-apply and has not been
487 // recovered; overwriting it with this set's intent would strand the old
488 // change half-applied with no record of how to finish it. Recovery
489 // ([`Journal::recover`]) must complete it first. A journal this same apply is about to write does not exist yet,
490 // so this only ever fires on a genuinely stale one.
491 let journal = self.path_in(root);
492 if fs.try_exists(&journal).await? {
493 return Err(Error::StaleJournal(journal));
494 }
495 // Expectations are checked here and nowhere else: after the
496 // stale-journal refusal, because a tree with an unrecovered change in
497 // it is mid-flight and not yet in any state worth comparing against;
498 // and before the fast path and the commit point alike, so a set of
499 // one is guarded exactly as a set of many. A failure is a refusal,
500 // not an abort — nothing has been written, so there is nothing to
501 // unwind and no flush to owe.
502 check_expected(fs, root, &changes.expected).await?;
503 if changes.ops.is_empty() {
504 return Ok(());
505 }
506 // A set of one needs no journal. The journal exists to make *several*
507 // file operations land as one unit; a lone op is already indivisible on a
508 // backend claiming `atomic_replace` — a `write_atomic` is all-or-nothing
509 // by construction, and a lone `rename` or `unlink` is atomic by the
510 // filesystem's own guarantee. Journaling it would write, flush, and then
511 // delete a second file in order to restate a promise the op already
512 // carries, roughly tripling what the commonest mutation there is — saving
513 // one document — costs in writes and flushes alike.
514 //
515 // What this gives up is *liveness*, not safety. With a journal, a crash
516 // mid-apply is rolled forward to the applied state by the next
517 // [`crate::journal::recover`]; without one, a crash simply means the op
518 // did not happen. For a set of one those are the only two states there
519 // are — no caller can observe a half-applied set of one — so all that
520 // changes is which side of the atomic instant a crash lands on, never
521 // whether it lands on one at all.
522 //
523 // The stale-journal refusal above still applies: this path writes no
524 // journal, but it must not slip a write past an *earlier* interrupted
525 // change that recovery has yet to roll forward, or recovery would later
526 // overwrite what was just written.
527 //
528 // A lone `SetLink` is excluded: the port's contract for `set_link` is
529 // "replaces whatever is there", not "in one indivisible step", so on a
530 // backend that replaces by remove-then-remake a crash inside the call
531 // can leave the path holding neither the old file nor the link. That
532 // is a half-applied set of one — the very thing the fast path's
533 // argument says cannot exist — so the op takes the journal, whose
534 // recovery re-runs `set_link` to the applied state.
535 if changes.ops.len() == 1
536 && fs.capabilities().atomic_replace
537 && !matches!(changes.ops[0], FileOp::SetLink { .. })
538 {
539 // No undo to record, either. Nothing preceded this op that could need
540 // unwinding, and every failure mode leaves the target untouched — so
541 // the reflexive read of the very file about to be overwritten, whose
542 // only purpose is to hold the old bytes for a rollback that cannot
543 // happen here, goes with it.
544 //
545 // The flush debt is still owed: a lone rename or remove edits
546 // directory entries nothing else will flush, and `Ok` from this
547 // crate means the op outlives a power cut — for a set of one
548 // exactly as for a set of many. The one honest caveat, the same
549 // one `write_atomic` has always had: if the certifying flush
550 // itself fails, the op has landed but `Ok` is withheld, and with
551 // no journal and no undo there is nothing to roll back — the
552 // error is the flush's, and the caller knows the op is at most
553 // applied-but-uncertified.
554 let mut touched = BTreeSet::new();
555 exec(fs, root, &changes.ops[0], None, &mut touched).await?;
556 return Ok(crate::fs::flush_all_durable(fs, touched, root).await?);
557 }
558 // The commit point: durably record the whole intent before touching a
559 // single document. `write_atomic` flushes it, so a crash finds the
560 // journal whole or not at all — never half-written. A journal kept
561 // outside the root may be pointed at a directory nothing has made yet
562 // (a cache directory on a fresh machine), so its home is made here —
563 // and every directory the making mints is flushed durable before the
564 // intent is trusted to live there, because a commit point inside a
565 // chain of unflushed names is one a power cut can take back whole:
566 // the journal file durable, the directory naming it gone, and a
567 // half-applied set with no record to roll forward. `write_atomic`
568 // flushes the home itself; the chain above it is owed here. The root
569 // needs no such courtesy, since a tree being applied to exists.
570 if let Some(home) = self.home() {
571 for made in crate::fs::create_dir_all_traced(fs, home).await? {
572 fs.sync(&made, crate::fs::Durability::Durable).await?;
573 }
574 }
575 fs.write_atomic(&journal, &crate::journal::encode(&changes.ops)?)
576 .await?;
577
578 let mut undo: Vec<Undo> = Vec::new();
579 let mut touched = BTreeSet::new();
580 let mut cause: Option<Error> = None;
581 for op in &changes.ops {
582 if let Err(e) = exec(fs, root, op, Some(&mut undo), &mut touched).await {
583 cause = Some(e);
584 break;
585 }
586 }
587 // Applied cleanly — now make that mean something across a power cut.
588 // `exec` barriered every name-unstable debt as it ran; what remains
589 // is the stable ones (edited directories, fresh chains), flushed as
590 // barriers capped by one drain of the root, so the whole set survives
591 // before the journal that certifies it is given up. A certification
592 // that *fails* is treated exactly as a failed op — the set rolls
593 // back — because "applied, but perhaps not durable" is neither of
594 // the two endpoints `Ok` and `Err` name.
595 if cause.is_none()
596 && let Err(e) = crate::fs::flush_all_durable(fs, touched, root).await
597 {
598 cause = Some(e.into());
599 }
600 if let Some(cause) = cause {
601 return Err(match unwind_durable(fs, undo, root, &journal).await {
602 // Reverted cleanly and durably: the abort is now a fact a
603 // power cut cannot contradict, so the cause alone is the
604 // answer.
605 Ok(()) => cause,
606 // Could not revert, or could not certify the reversion: the
607 // journal is kept where possible, so recovery rolls the set
608 // forward to the consistent applied state.
609 Err(rollback) => Error::Torn {
610 cause: cause.to_string(),
611 rollback: rollback.to_string(),
612 },
613 });
614 }
615 // The deletion itself is deliberately *not* flushed: if a crash
616 // resurrects the journal, the ops it names are already durable and
617 // replay is idempotent, so the next recovery no-ops through it and
618 // clears it — the designed-for state, at the price of at most one
619 // StaleJournal prompt.
620 match fs.remove_file(&journal).await {
621 Ok(()) => Ok(()),
622 // The set is applied and certified durable; only the journal's
623 // retirement failed. Rolling a *certified* change back over a
624 // delete error would be strictly worse, and a plain `Err` would
625 // claim an abort that did not happen — so this is `Torn`, whose
626 // contract fits exactly: the tree is at a nameable state, and
627 // the surviving journal makes the next recovery an idempotent
628 // no-op replay that clears it.
629 Err(e) => Err(Error::Torn {
630 cause: format!(
631 "the set applied and was certified durable, but its journal \
632 could not be retired: {e}"
633 ),
634 rollback: "the surviving journal will be replayed idempotently and \
635 cleared by the next recovery"
636 .to_string(),
637 }),
638 }
639 }
640}
641
642/// Reverse every recorded op, certify the reversion durable, then durably
643/// retire `journal` — the abort-side counterpart of the flush `apply` runs on
644/// success, with the unwind folded in.
645///
646/// Folded in because the barriers must interleave: a step that writes bytes
647/// barriers them *immediately*, while the name it wrote still resolves — a
648/// later step may move it (the reversed order of a rename-then-edit set does
649/// exactly that), and a barrier deferred to the end would be addressed to
650/// nothing and quietly no-op. Directory debts are stable names and batch.
651/// Best-effort like the unwind it absorbs: a step that fails does not abandon
652/// the rest — the more that is put back the better — and the first failure is
653/// what gets reported, with the journal left standing so recovery can roll
654/// the set forward to the nameable applied state.
655///
656/// The certification order is the argument. Barriers; then, for a
657/// [homed](crate::Journal::kept_in) journal, one drain of the root — the home
658/// may live on another device, whose drain proves nothing about the tree's —
659/// making the restored state a fact; then the journal's deletion; then one
660/// drain of the journal's own directory, making the retirement a fact too.
661/// When the journal lives in the root the two caps collapse into one, placed
662/// after the deletion so it certifies both. Only after all of it is `Err` a
663/// promise: durably before the set, with no journal for a later recovery to
664/// contradict the abort with. A failure *after* the deletion leaves the abort
665/// certified but its retirement not — still [`Error::Torn`]'s territory, and
666/// still nameable: if the deletion survives, recovery finds nothing; if a
667/// power cut takes it back, recovery rolls the set forward.
668async fn unwind_durable<FS: Storage>(
669 fs: &FS,
670 undo: Vec<Undo>,
671 root: &Path,
672 journal: &Path,
673) -> Result<()> {
674 let mut first_error: Option<std::io::Error> = None;
675 let mut dirs: BTreeSet<PathBuf> = BTreeSet::new();
676 for step in undo.into_iter().rev() {
677 let result = match step {
678 Undo::Restore { path, bytes } => {
679 // The parent joins the debt even when a Restore born of an
680 // overwrite left it unchanged: the same variant reverses a
681 // `Remove`, whose reversal *re-creates* the entry, and an
682 // extra barrier on an unchanged directory costs less than
683 // telling the two origins apart.
684 if let Some(dir) = crate::fs::parent_dir(&path) {
685 dirs.insert(dir.to_path_buf());
686 }
687 match fs.write(&path, &bytes).await {
688 Ok(()) => fs.sync(&path, crate::fs::Durability::Ordered).await,
689 e => e,
690 }
691 }
692 // Already absent is already undone — see `Undo::Delete`. Reporting it
693 // would raise `Error::Torn` over the single most ordinary rollback
694 // there is: a write to a new file that failed before creating it.
695 Undo::Delete { path } => {
696 if let Some(dir) = crate::fs::parent_dir(&path) {
697 dirs.insert(dir.to_path_buf());
698 }
699 match fs.remove_file(&path).await {
700 Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()),
701 other => other,
702 }
703 }
704 Undo::Rename { from, to } => {
705 for side in [&from, &to] {
706 if let Some(dir) = crate::fs::parent_dir(side) {
707 dirs.insert(dir.to_path_buf());
708 }
709 }
710 fs.rename(&from, &to).await
711 }
712 Undo::SetExecutable { path, executable } => {
713 match fs.set_executable(&path, executable).await {
714 // The inode, barriered while the name still resolves.
715 Ok(()) => fs.sync(&path, crate::fs::Durability::Ordered).await,
716 e => e,
717 }
718 }
719 Undo::Relink { path, target } => {
720 if let Some(dir) = crate::fs::parent_dir(&path) {
721 dirs.insert(dir.to_path_buf());
722 }
723 fs.set_link(&path, &target).await
724 }
725 // The link first, tolerantly (the `set_link` being reversed may
726 // have failed before creating it), and only then the bytes — a
727 // plain write while the link stands would land them in its target,
728 // which is also why a remove that fails for a real reason must
729 // stop the write rather than precede it.
730 Undo::RestoreOverLink { path, bytes } => {
731 if let Some(dir) = crate::fs::parent_dir(&path) {
732 dirs.insert(dir.to_path_buf());
733 }
734 let removed = match fs.remove_file(&path).await {
735 Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()),
736 other => other,
737 };
738 match removed {
739 Ok(()) => match fs.write(&path, &bytes).await {
740 Ok(()) => fs.sync(&path, crate::fs::Durability::Ordered).await,
741 e => e,
742 },
743 Err(e) => Err(e),
744 }
745 }
746 };
747 if let Err(e) = result
748 && first_error.is_none()
749 {
750 first_error = Some(e);
751 }
752 }
753 if let Some(e) = first_error {
754 return Err(e.into());
755 }
756 for dir in dirs {
757 fs.sync(&dir, crate::fs::Durability::Ordered).await?;
758 }
759 let jparent = crate::fs::parent_dir(journal);
760 if jparent != Some(root) {
761 fs.sync(root, crate::fs::Durability::Durable).await?;
762 }
763 fs.remove_file(journal).await?;
764 match jparent {
765 Some(dir) => Ok(fs.sync(dir, crate::fs::Durability::Durable).await?),
766 None => Ok(()),
767 }
768}
769
770/// How to reverse one applied op, recorded against the state that op found.
771///
772/// Recorded *per op at execution time*, not for the whole set up front, because
773/// ops in a set are not independent: `rename` moves `a.md` to `sub/a.md` and
774/// then rewrites `sub/a.md`'s re-relativized links, so the write's undo has to
775/// restore the bytes the rename put there — a snapshot taken before the set ran
776/// would say "`sub/a.md` did not exist; delete it", and the rename's undo would
777/// then have nothing to move back. Paths here are already root-joined.
778enum Undo {
779 /// Put these bytes back (the file existed and was overwritten or removed).
780 Restore { path: PathBuf, bytes: Vec<u8> },
781 /// Delete the file (it did not exist before the write created it).
782 ///
783 /// Recorded *before* the write it reverses, because a write that fails
784 /// partway still leaves a file behind — so this has to tolerate finding
785 /// nothing there, which is the case where the write failed before creating
786 /// anything at all. Undoing nothing is success, not a torn tree.
787 Delete { path: PathBuf },
788 /// Move `from` back to `to`.
789 Rename { from: PathBuf, to: PathBuf },
790 /// Put the execute bit back the way it was.
791 SetExecutable { path: PathBuf, executable: bool },
792 /// Point the link back at its old target (the path held a link before a
793 /// [`FileOp::SetLink`] repointed it).
794 Relink { path: PathBuf, target: PathBuf },
795 /// Put a regular file's bytes back where a link now stands.
796 ///
797 /// Not a [`Restore`](Undo::Restore): a plain `write` to a path holding a
798 /// link writes *through* it, landing the old bytes in whatever the link
799 /// points at instead of back at the path. The link has to be removed
800 /// first — and tolerantly, since the `set_link` being reversed may have
801 /// failed before creating it.
802 RestoreOverLink { path: PathBuf, bytes: Vec<u8> },
803}
804
805/// Apply one op, optionally recording how to reverse it, and always recording
806/// what it dirtied.
807///
808/// `undo` is `None` only for a set of one, which has no rollback to feed: see
809/// the fast path in [`ChangeSet::apply`]. Recording is not merely unused there,
810/// it is worth skipping — for a write it costs a full read of the file about to
811/// be replaced.
812///
813/// `touched` collects everything this op changed that *nothing has flushed
814/// yet* — the flush debt `apply` settles once, at the end, before the journal
815/// is dropped. A write's rename-published entry (its bytes are barriered by
816/// [`Storage::replace`] itself); a rename's or remove's edited entries; an
817/// execute-bit flip's inode; a fresh directory chain. Deferring the lot to
818/// one barrier-capped flush is what makes ten writes into a directory cost
819/// one drain, not ten.
820async fn exec<FS: Storage>(
821 fs: &FS,
822 root: &Path,
823 op: &FileOp,
824 mut undo: Option<&mut Vec<Undo>>,
825 touched: &mut BTreeSet<PathBuf>,
826) -> Result<()> {
827 match op {
828 FileOp::Write { path, bytes } => {
829 let full = root.join(path);
830 // Record the undo *before* writing: a write that fails partway
831 // (a full disk) leaves a truncated file, and restoring the old
832 // bytes over it is exactly the repair.
833 if let Some(undo) = undo {
834 capture_replaced(fs, &full, undo).await?;
835 }
836 ensure_parent(fs, &full, touched).await?;
837 // Land the document through the atomic-replace protocol, so even a
838 // crash mid-write cannot expose a half-written file. `replace`,
839 // not `write_atomic`: the atomicity is per-file, but durability is
840 // the *set's* — the parent entry (and, on a backend that cannot
841 // replace atomically, the plainly-written bytes) joins the flush
842 // debt the apply settles once, so ten writes into one directory
843 // cost one drain rather than ten.
844 fs.replace(&full, bytes).await?;
845 settle_write_debt(fs, &full, touched).await?;
846 }
847 FileOp::Rename { from, to } => {
848 let (from_full, to_full) = (root.join(from), root.join(to));
849 if let Some(undo) = undo.as_deref_mut() {
850 // The destination may be occupied, and the rename replaces
851 // the occupant — the port contract's load-bearing half. What
852 // it replaces is therefore part of "the tree as it was", and
853 // the rollback owes it back: captured here, *before* the
854 // Rename undo below, so the reversed unwind first moves the
855 // mover home and then restores the displaced occupant into
856 // the vacated name. An empty destination records a tolerant
857 // delete, which the rename-back has already satisfied.
858 capture_replaced(fs, &to_full, undo).await?;
859 }
860 ensure_parent(fs, &to_full, touched).await?;
861 fs.rename(&from_full, &to_full).await?;
862 // Two directory entries changed — the name removed from one
863 // parent, added to the other — and nothing has flushed either.
864 // The rename's *atomicity* across a crash is the metadata
865 // journal's own gift on every filesystem this crate targets; what
866 // the flush buys is that it is not taken back wholesale.
867 for side in [&from_full, &to_full] {
868 if let Some(dir) = crate::fs::parent_dir(side) {
869 touched.insert(dir.to_path_buf());
870 }
871 }
872 if let Some(undo) = undo {
873 undo.push(Undo::Rename {
874 from: to_full,
875 to: from_full,
876 });
877 }
878 }
879 FileOp::Remove { path } => {
880 let full = root.join(path);
881 // The entry leaves its parent, and nothing else flushes that.
882 if let Some(dir) = crate::fs::parent_dir(&full) {
883 touched.insert(dir.to_path_buf());
884 }
885 match undo {
886 // What was removed is the undo, so it has to be read out
887 // before it goes — the *link* where the path holds one
888 // (`remove_file` removes the link, never its target, so a
889 // rollback that rewrote it as a file holding the target's
890 // bytes would remove a link and give back a copy), the bytes
891 // everywhere else. A dangling link is removable on the same
892 // terms; reading through it to capture bytes would refuse an
893 // op the filesystem itself permits.
894 Some(undo) => match fs.read_link(&full).await {
895 Ok(Some(target)) => {
896 fs.remove_file(&full).await?;
897 undo.push(Undo::Relink { path: full, target });
898 }
899 _ => {
900 let old = fs.read(&full).await?;
901 fs.remove_file(&full).await?;
902 undo.push(Undo::Restore {
903 path: full,
904 bytes: old,
905 });
906 }
907 },
908 None => fs.remove_file(&full).await?,
909 }
910 }
911 // A `Write` whose bytes were left at the source. The read happens here, at
912 // execution time, rather than when the op was staged — that is the whole
913 // saving, and it is why the source has to be immutable.
914 FileOp::CopyFrom { path, source } => {
915 let (full, source_full) = (root.join(path), root.join(source));
916 let bytes = fs.read(&source_full).await?;
917 if let Some(undo) = undo {
918 capture_replaced(fs, &full, undo).await?;
919 }
920 ensure_parent(fs, &full, touched).await?;
921 fs.replace(&full, &bytes).await?;
922 settle_write_debt(fs, &full, touched).await?;
923 }
924 FileOp::SetExecutable { path, executable } => {
925 let full = root.join(path);
926 guard_not_link(fs, &full).await?;
927 if let Some(undo) = undo {
928 // Captured through the read half, so the rollback restores
929 // what *was* — not the blind opposite of what was asked,
930 // which is wrong whenever the bit was already in the
931 // requested state. A backend that declines the question
932 // (`None`) has no bit to restore and the op below will no-op
933 // on it too, so nothing is recorded.
934 if let Some(was) = fs.executable(&full).await? {
935 undo.push(Undo::SetExecutable {
936 path: full.clone(),
937 executable: was,
938 });
939 }
940 }
941 fs.set_executable(&full, *executable).await?;
942 // A mode is inode metadata, and the inode is barriered *now*,
943 // while the name still resolves to it — a later op in this very
944 // set may rename or remove the name, and a flush deferred to the
945 // final pass would then be addressed to nothing and quietly
946 // no-op. The parent joins the batched debt instead: a stable
947 // name, and what keeps the final drain owed at all.
948 fs.sync(&full, crate::fs::Durability::Ordered).await?;
949 if let Some(dir) = crate::fs::parent_dir(&full) {
950 touched.insert(dir.to_path_buf());
951 }
952 }
953 FileOp::SetLink { path, target } => {
954 let full = root.join(path);
955 if let Some(undo) = undo {
956 match fs.read_link(&full).await {
957 // The path held a link: point it back afterwards.
958 Ok(Some(old_target)) => undo.push(Undo::Relink {
959 path: full.clone(),
960 target: old_target,
961 }),
962 // The backend models no links at all; `set_link` below
963 // will refuse, so there is nothing to record — the op
964 // never applies.
965 Ok(None) => {}
966 // Nothing there: the undo is removal, and `Delete`
967 // removes a link as readily as a file.
968 Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
969 undo.push(Undo::Delete { path: full.clone() });
970 }
971 // Not a link — a regular file about to give way to one.
972 // Its bytes are the undo, restored *after* the link is
973 // removed (see [`Undo::RestoreOverLink`]); any error
974 // reading them aborts the op before it touches anything.
975 Err(_) => {
976 let old = fs.read(&full).await?;
977 undo.push(Undo::RestoreOverLink {
978 path: full.clone(),
979 bytes: old,
980 });
981 }
982 }
983 }
984 ensure_parent(fs, &full, touched).await?;
985 fs.set_link(&full, target).await?;
986 // The link is an entry (and its inode rides on the entry's
987 // flush): the parent is the debt.
988 if let Some(dir) = crate::fs::parent_dir(&full) {
989 touched.insert(dir.to_path_buf());
990 }
991 }
992 }
993 Ok(())
994}
995
996/// Settle what one [`Storage::replace`] leaves behind: the parent entry the
997/// rename published joins the batched debt, and on a backend that cannot
998/// replace atomically the plainly-written bytes are barriered *here* — while
999/// the name still resolves to them, since a later op in the same set may
1000/// rename or remove it, and a flush deferred to the final pass would then be
1001/// addressed to nothing. The final drain makes the barrier durable.
1002pub(crate) async fn settle_write_debt<FS: Storage>(
1003 fs: &FS,
1004 full: &Path,
1005 touched: &mut BTreeSet<PathBuf>,
1006) -> Result<()> {
1007 if !fs.capabilities().atomic_replace {
1008 fs.sync(full, crate::fs::Durability::Ordered).await?;
1009 }
1010 if let Some(dir) = crate::fs::parent_dir(full) {
1011 touched.insert(dir.to_path_buf());
1012 }
1013 Ok(())
1014}
1015
1016/// Refuse a [`FileOp::SetExecutable`] whose path holds a symbolic link.
1017///
1018/// Mode reads and writes follow links — `metadata` and `set_permissions`
1019/// both do — so setting the bit "at" a link sets it on the link's *referent*,
1020/// wherever that is. The root guard cannot see this: it is lexical, the link
1021/// is not, and a set that stages `set_link("l", "/outside/victim")` then
1022/// `set_executable("l", true)` would chmod a file the tree does not own. The
1023/// same applies to a link the set never made. So the op is refused on any
1024/// link, loudly, before the undo capture reads a bit that is not the path's
1025/// own. Shared by the apply and the journal replay, which faces the same
1026/// combination from bytes it did not author.
1027pub(crate) async fn guard_not_link<FS: Storage>(fs: &FS, full: &Path) -> Result<()> {
1028 if let Ok(Some(_)) = fs.read_link(full).await {
1029 return Err(Error::Io(std::io::Error::new(
1030 std::io::ErrorKind::InvalidInput,
1031 format!(
1032 "refusing to set the execute bit through the symbolic link at {}",
1033 full.display()
1034 ),
1035 )));
1036 }
1037 Ok(())
1038}
1039
1040/// Record how to put back whatever a replacing write (`Write`, `CopyFrom`) is
1041/// about to displace at `full`.
1042///
1043/// The link question comes first, because a plain `read` follows one: capture
1044/// by bytes alone and a path holding a link rolls back to a *regular file*
1045/// holding a copy of its target — the link gone, shared content duplicated,
1046/// and "the tree is as it was" quietly false. So: a link is put back as a link
1047/// ([`Undo::Relink`] — `write_atomic` will have replaced the entry itself,
1048/// and `set_link` restores it the same way); nothing is put back by deletion;
1049/// and only a path holding an actual file is captured as bytes.
1050async fn capture_replaced<FS: Storage>(fs: &FS, full: &Path, undo: &mut Vec<Undo>) -> Result<()> {
1051 match fs.read_link(full).await {
1052 Ok(Some(target)) => undo.push(Undo::Relink {
1053 path: full.to_path_buf(),
1054 target,
1055 }),
1056 Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
1057 undo.push(Undo::Delete {
1058 path: full.to_path_buf(),
1059 });
1060 }
1061 // `Ok(None)` — a backend with no links, where nothing can be one —
1062 // and any other error — the path holds something that is not a link —
1063 // both fall through to capturing bytes.
1064 _ => match fs.read(full).await {
1065 Ok(old) => undo.push(Undo::Restore {
1066 path: full.to_path_buf(),
1067 bytes: old,
1068 }),
1069 Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
1070 undo.push(Undo::Delete {
1071 path: full.to_path_buf(),
1072 });
1073 }
1074 Err(e) => return Err(e.into()),
1075 },
1076 }
1077 Ok(())
1078}
1079
1080/// Clamp every path a sequence of ops names to the root it will run against.
1081///
1082/// Shared by [`ChangeSet::apply`], which guards sets assembled from data it
1083/// did not author, and by [`Journal::recover`](crate::Journal::recover), whose
1084/// input is *always* that: a journal is bytes found on disk, and — homed in a
1085/// synced folder, or planted — possibly bytes some other machine wrote. The
1086/// checksum authenticates nothing (anyone can recompute FNV-1a), so replay
1087/// must refuse an escaping path exactly as the apply that would have written
1088/// the journal honestly would have.
1089///
1090/// The one path deliberately *not* clamped is a [`FileOp::SetLink`] target:
1091/// nothing is written through it — it is recorded, not resolved — and a link
1092/// is allowed to point wherever links point, outside the root included.
1093pub(crate) fn guard_ops(ops: &[FileOp]) -> Result<()> {
1094 for op in ops {
1095 match op {
1096 FileOp::Write { path, .. }
1097 | FileOp::Remove { path }
1098 | FileOp::SetExecutable { path, .. }
1099 | FileOp::SetLink { path, .. } => {
1100 guard_in_root(path)?;
1101 }
1102 FileOp::Rename { from, to } => {
1103 guard_in_root(from)?;
1104 guard_in_root(to)?;
1105 }
1106 // The source is clamped too: it is read, and a set assembled by a
1107 // caller must not be able to pull `../../../etc/passwd` into the
1108 // tree any more than it may write out of one.
1109 FileOp::CopyFrom { path, source } => {
1110 guard_in_root(path)?;
1111 guard_in_root(source)?;
1112 }
1113 }
1114 }
1115 Ok(())
1116}
1117
1118/// Check every staged expectation against the tree as it stands, before the
1119/// set touches anything.
1120///
1121/// Bytes are read as the backend reads — through a standing link — while
1122/// absence is judged on the *entry*: a dangling link names no bytes but still
1123/// occupies the path, and a create that expected absence must not land on top
1124/// of it. `read_link` settles the link question (its error is the normal
1125/// answer for "no link here", so only a real fault propagates), `try_exists`
1126/// settles the rest.
1127async fn check_expected<FS: Storage>(
1128 fs: &FS,
1129 root: &Path,
1130 expected: &[(PathBuf, Expected)],
1131) -> Result<()> {
1132 for (rel, want) in expected {
1133 let full = root.join(rel);
1134 match want {
1135 Expected::Bytes(bytes) => match fs.read(&full).await {
1136 Ok(found) if found == *bytes => {}
1137 Ok(_) => return Err(Error::Drifted(rel.clone())),
1138 Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
1139 return Err(Error::Drifted(rel.clone()));
1140 }
1141 Err(e) => return Err(e.into()),
1142 },
1143 Expected::Absent => {
1144 let link = match fs.read_link(&full).await {
1145 Ok(link) => link,
1146 Err(e)
1147 if matches!(
1148 e.kind(),
1149 std::io::ErrorKind::NotFound | std::io::ErrorKind::InvalidInput
1150 ) =>
1151 {
1152 None
1153 }
1154 Err(e) => return Err(e.into()),
1155 };
1156 if link.is_some() || fs.try_exists(&full).await? {
1157 return Err(Error::Drifted(rel.clone()));
1158 }
1159 }
1160 }
1161 }
1162 Ok(())
1163}
1164
1165/// Create `full`'s parent directory if it is missing. Unconditional (rather than
1166/// staged as its own op) because a directory is not part of the document graph:
1167/// it is an artifact of *where* a write lands, so it belongs to the write.
1168///
1169/// Every directory the making mints joins `touched`: each is an entry of its
1170/// own, persisting separately from the file that prompted it, and a
1171/// durably-flushed file inside a chain of unflushed names is a file a power
1172/// cut can orphan.
1173async fn ensure_parent<FS: Storage>(
1174 fs: &FS,
1175 full: &Path,
1176 touched: &mut BTreeSet<PathBuf>,
1177) -> Result<()> {
1178 if let Some(dir) = crate::fs::parent_dir(full) {
1179 for made in crate::fs::create_dir_all_traced(fs, dir).await? {
1180 touched.insert(made);
1181 }
1182 }
1183 Ok(())
1184}
1185
1186#[cfg(test)]
1187mod tests {
1188 use super::*;
1189 use crate::exec::block_on;
1190 use crate::fs::{ReadStorage, StdFs};
1191 use crate::fs_faults::{FailAtWrite, FsEvent, RecordingFs};
1192 use crate::journal::Journal;
1193
1194 fn tmp(name: &str) -> PathBuf {
1195 let dir = std::env::temp_dir().join(format!("fstx-change-{name}"));
1196 let _ = std::fs::remove_dir_all(&dir);
1197 std::fs::create_dir_all(&dir).unwrap();
1198 dir
1199 }
1200
1201 fn read(root: &Path, rel: &str) -> Option<String> {
1202 std::fs::read_to_string(root.join(rel)).ok()
1203 }
1204
1205 #[test]
1206 fn applies_every_op_in_order() {
1207 let root = tmp("apply");
1208 std::fs::write(root.join("parent.md"), "old parent").unwrap();
1209 let mut cs = ChangeSet::new();
1210 cs.write("child.md", "child");
1211 cs.write("parent.md", "new parent");
1212 block_on(cs.apply(&StdFs, &root)).unwrap();
1213 assert_eq!(read(&root, "child.md").as_deref(), Some("child"));
1214 assert_eq!(read(&root, "parent.md").as_deref(), Some("new parent"));
1215 }
1216
1217 #[test]
1218 fn creates_missing_parent_directories() {
1219 let root = tmp("mkdir");
1220 let mut cs = ChangeSet::new();
1221 cs.write("deep/nested/child.md", "hi");
1222 block_on(cs.apply(&StdFs, &root)).unwrap();
1223 assert_eq!(read(&root, "deep/nested/child.md").as_deref(), Some("hi"));
1224 }
1225
1226 #[test]
1227 fn a_copy_lands_the_source_bytes_and_leaves_the_source_alone() {
1228 let root = tmp("copy");
1229 std::fs::create_dir_all(root.join("history/blobs/9f")).unwrap();
1230 std::fs::write(root.join("history/blobs/9f/86d081"), "captured").unwrap();
1231 std::fs::write(root.join("notes.md"), "damaged").unwrap();
1232
1233 let mut cs = ChangeSet::new();
1234 cs.copy_from("notes.md", "history/blobs/9f/86d081");
1235 // A path that does not exist yet gets its parent made, as a write does.
1236 cs.copy_from("deep/fresh.md", "history/blobs/9f/86d081");
1237 block_on(cs.apply(&StdFs, &root)).unwrap();
1238
1239 assert_eq!(read(&root, "notes.md").as_deref(), Some("captured"));
1240 assert_eq!(read(&root, "deep/fresh.md").as_deref(), Some("captured"));
1241 // The blob is shared by every event naming it: read, never consumed.
1242 assert_eq!(
1243 read(&root, "history/blobs/9f/86d081").as_deref(),
1244 Some("captured")
1245 );
1246 }
1247
1248 #[test]
1249 fn a_failed_copy_rolls_back_exactly_as_a_failed_write_does() {
1250 // A copy is a write whose payload was fetched late, so it must record the
1251 // same undo: restore what it overwrote, delete what it created.
1252 let root = tmp("rollback-copy");
1253 std::fs::create_dir_all(root.join("history/blobs/9f")).unwrap();
1254 std::fs::write(root.join("history/blobs/9f/86d081"), "captured").unwrap();
1255 std::fs::write(root.join("notes.md"), "damaged").unwrap();
1256
1257 let mut cs = ChangeSet::new();
1258 cs.copy_from("notes.md", "history/blobs/9f/86d081");
1259 cs.copy_from("fresh.md", "history/blobs/9f/86d081");
1260 cs.write("doomed.md", "never lands");
1261 let err = block_on(cs.apply(&FailAtWrite::nth(2), &root)).unwrap_err();
1262 assert!(err.to_string().contains("disk full"), "{err}");
1263
1264 assert_eq!(read(&root, "notes.md").as_deref(), Some("damaged"));
1265 assert_eq!(read(&root, "fresh.md"), None);
1266 }
1267
1268 #[test]
1269 fn a_copy_from_a_missing_source_fails_before_the_target_is_touched() {
1270 // The half-synced event: the manifest names a blob the transport has not
1271 // delivered. Better to fail the set than to write a hole into the tree.
1272 let root = tmp("copy-missing-source");
1273 std::fs::write(root.join("notes.md"), "damaged").unwrap();
1274 let mut cs = ChangeSet::new();
1275 cs.copy_from("notes.md", "history/blobs/9f/86d081");
1276 assert!(block_on(cs.apply(&StdFs, &root)).is_err());
1277 assert_eq!(read(&root, "notes.md").as_deref(), Some("damaged"));
1278 }
1279
1280 #[test]
1281 fn a_copy_cannot_read_from_outside_the_root() {
1282 // The source is read, so it is clamped like every written path: a set a
1283 // caller assembled must not be able to pull the host's files into the tree.
1284 let root = tmp("copy-escape");
1285 let mut cs = ChangeSet::new();
1286 cs.copy_from("stolen.md", "../../../etc/passwd");
1287 let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
1288 assert!(matches!(err, Error::Escape(_)), "{err:?}");
1289 assert_eq!(read(&root, "stolen.md"), None);
1290 }
1291
1292 #[test]
1293 fn a_failed_write_restores_the_files_already_written() {
1294 let root = tmp("rollback-write");
1295 std::fs::write(root.join("parent.md"), "old parent").unwrap();
1296 std::fs::write(root.join("child.md"), "old child").unwrap();
1297
1298 // Three writes staged; the third fails.
1299 let mut cs = ChangeSet::new();
1300 cs.write("child.md", "new child");
1301 cs.write("parent.md", "new parent");
1302 cs.write("third.md", "third");
1303 let err = block_on(cs.apply(&FailAtWrite::nth(2), &root)).unwrap_err();
1304 assert!(err.to_string().contains("disk full"), "{err}");
1305
1306 // Everything is as it was found — no half-linked tree.
1307 assert_eq!(read(&root, "child.md").as_deref(), Some("old child"));
1308 assert_eq!(read(&root, "parent.md").as_deref(), Some("old parent"));
1309 }
1310
1311 #[test]
1312 fn a_failed_write_deletes_files_the_set_had_created() {
1313 let root = tmp("rollback-create");
1314 let mut cs = ChangeSet::new();
1315 cs.write("fresh.md", "fresh");
1316 cs.write("doomed.md", "doomed");
1317 let err = block_on(cs.apply(&FailAtWrite::nth(1), &root)).unwrap_err();
1318 assert!(err.to_string().contains("disk full"), "{err}");
1319 // The file the set created before failing is gone, not orphaned.
1320 assert_eq!(read(&root, "fresh.md"), None);
1321 }
1322
1323 #[cfg(unix)]
1324 fn is_executable(root: &Path, rel: &str) -> bool {
1325 use std::os::unix::fs::PermissionsExt as _;
1326 std::fs::metadata(root.join(rel))
1327 .unwrap()
1328 .permissions()
1329 .mode()
1330 & 0o111
1331 != 0
1332 }
1333
1334 #[cfg(unix)]
1335 #[test]
1336 fn sets_and_clears_the_execute_bit() {
1337 let root = tmp("exec-bit");
1338 std::fs::write(root.join("run.sh"), "#!/bin/sh").unwrap();
1339 std::fs::write(root.join("plain.md"), "notes").unwrap();
1340
1341 let mut cs = ChangeSet::new();
1342 cs.set_executable("run.sh", true);
1343 cs.set_executable("plain.md", false);
1344 block_on(cs.apply(&StdFs, &root)).unwrap();
1345
1346 assert!(is_executable(&root, "run.sh"));
1347 assert!(!is_executable(&root, "plain.md"));
1348 }
1349
1350 #[cfg(unix)]
1351 #[test]
1352 fn a_failed_set_restores_the_execute_bit_it_flipped() {
1353 let root = tmp("rollback-exec");
1354 std::fs::write(root.join("run.sh"), "#!/bin/sh").unwrap();
1355 assert!(!is_executable(&root, "run.sh"));
1356
1357 let mut cs = ChangeSet::new();
1358 cs.set_executable("run.sh", true);
1359 cs.write("doomed.md", "never lands");
1360 let err = block_on(cs.apply(&FailAtWrite::nth(0), &root)).unwrap_err();
1361 assert!(err.to_string().contains("disk full"), "{err}");
1362
1363 assert!(
1364 !is_executable(&root, "run.sh"),
1365 "the bit must roll back to what was, not stay flipped"
1366 );
1367 }
1368
1369 #[cfg(unix)]
1370 #[test]
1371 fn a_bit_already_in_the_requested_state_rolls_back_to_itself() {
1372 // The undo is captured through the read half, not derived as "the
1373 // opposite of what was asked" — which is wrong exactly here.
1374 use std::os::unix::fs::PermissionsExt as _;
1375 let root = tmp("rollback-exec-noop");
1376 std::fs::write(root.join("run.sh"), "#!/bin/sh").unwrap();
1377 let mut perms = std::fs::metadata(root.join("run.sh"))
1378 .unwrap()
1379 .permissions();
1380 perms.set_mode(perms.mode() | 0o100);
1381 std::fs::set_permissions(root.join("run.sh"), perms).unwrap();
1382
1383 let mut cs = ChangeSet::new();
1384 cs.set_executable("run.sh", true); // already true
1385 cs.write("doomed.md", "never lands");
1386 block_on(cs.apply(&FailAtWrite::nth(0), &root)).unwrap_err();
1387
1388 assert!(
1389 is_executable(&root, "run.sh"),
1390 "rolling back a no-op flip must not clear a bit the set never set"
1391 );
1392 }
1393
1394 #[cfg(unix)]
1395 #[test]
1396 fn lands_a_link_and_reads_nothing_through_it() {
1397 let root = tmp("link");
1398 let mut cs = ChangeSet::new();
1399 // A dangling target, and one pointing outside the root: both are
1400 // honest links — recorded, never resolved.
1401 cs.set_link("here.md", "nowhere/yet.md");
1402 cs.set_link("out.md", "../elsewhere.md");
1403 block_on(cs.apply(&StdFs, &root)).unwrap();
1404
1405 assert_eq!(
1406 std::fs::read_link(root.join("here.md")).unwrap(),
1407 PathBuf::from("nowhere/yet.md")
1408 );
1409 assert_eq!(
1410 std::fs::read_link(root.join("out.md")).unwrap(),
1411 PathBuf::from("../elsewhere.md")
1412 );
1413 }
1414
1415 #[cfg(unix)]
1416 #[test]
1417 fn a_failed_set_restores_the_file_a_link_replaced() {
1418 // The sharp edge `Undo::RestoreOverLink` exists for: a plain write
1419 // while the link stands would land the old bytes in the link's
1420 // *target*. The rollback must leave a regular file holding them, and
1421 // the target untouched.
1422 let root = tmp("rollback-link-over-file");
1423 std::fs::write(root.join("victim.md"), "the original").unwrap();
1424 std::fs::write(root.join("target.md"), "someone else's file").unwrap();
1425
1426 let mut cs = ChangeSet::new();
1427 cs.set_link("victim.md", "target.md");
1428 cs.write("doomed.md", "never lands");
1429 block_on(cs.apply(&FailAtWrite::nth(0), &root)).unwrap_err();
1430
1431 let md = std::fs::symlink_metadata(root.join("victim.md")).unwrap();
1432 assert!(md.file_type().is_file(), "the link must be gone");
1433 assert_eq!(read(&root, "victim.md").as_deref(), Some("the original"));
1434 assert_eq!(
1435 read(&root, "target.md").as_deref(),
1436 Some("someone else's file"),
1437 "nothing may be written through the link during rollback"
1438 );
1439 }
1440
1441 #[cfg(unix)]
1442 #[test]
1443 fn a_failed_set_repoints_a_link_it_had_repointed() {
1444 let root = tmp("rollback-relink");
1445 std::os::unix::fs::symlink("old-target.md", root.join("link.md")).unwrap();
1446
1447 let mut cs = ChangeSet::new();
1448 cs.set_link("link.md", "new-target.md");
1449 cs.write("doomed.md", "never lands");
1450 block_on(cs.apply(&FailAtWrite::nth(0), &root)).unwrap_err();
1451
1452 assert_eq!(
1453 std::fs::read_link(root.join("link.md")).unwrap(),
1454 PathBuf::from("old-target.md")
1455 );
1456 }
1457
1458 #[cfg(unix)]
1459 #[test]
1460 fn a_failed_set_removes_a_link_it_had_created() {
1461 let root = tmp("rollback-link-fresh");
1462 let mut cs = ChangeSet::new();
1463 cs.set_link("fresh.md", "anywhere.md");
1464 cs.write("doomed.md", "never lands");
1465 block_on(cs.apply(&FailAtWrite::nth(0), &root)).unwrap_err();
1466
1467 assert!(
1468 std::fs::symlink_metadata(root.join("fresh.md")).is_err(),
1469 "the created link must be gone"
1470 );
1471 }
1472
1473 #[cfg(unix)]
1474 #[test]
1475 fn a_lone_link_takes_the_journal_rather_than_the_fast_path() {
1476 // `set_link`'s contract is "replaces", not "replaces indivisibly", so
1477 // a set of one link still needs the journal a crash can roll forward —
1478 // the fast path's no-half-applied-state argument does not cover it.
1479 let root = tmp("lone-link");
1480 let fs = RecordingFs::local();
1481 let mut cs = ChangeSet::new();
1482 cs.set_link("link.md", "target.md");
1483 block_on(cs.apply(&fs, &root)).unwrap();
1484
1485 let journaled = fs
1486 .events()
1487 .iter()
1488 .any(|e| matches!(e, FsEvent::Write(p) if Journal::default().owns_path(p)));
1489 assert!(journaled, "events: {:?}", fs.events());
1490 }
1491
1492 #[test]
1493 fn an_execute_flip_over_a_backend_with_no_bit_applies_as_nothing() {
1494 // The op means "make this runnable", and on a backend where nothing
1495 // is runnable there is nothing to do — the honest no-op, not an error.
1496 let fs = crate::fs::InMemoryFs::new();
1497 block_on(fs.write(Path::new("root/doc.md"), b"hi")).unwrap();
1498 let mut cs = ChangeSet::new();
1499 cs.set_executable("doc.md", true);
1500 cs.write("other.md", "lands");
1501 block_on(cs.apply(&fs, Path::new("root"))).unwrap();
1502 assert_eq!(
1503 block_on(fs.read_to_string(Path::new("root/other.md"))).unwrap(),
1504 "lands"
1505 );
1506 }
1507
1508 #[test]
1509 fn a_link_over_a_backend_without_links_unwinds_the_set() {
1510 // Unlike an execute bit, a link has no honest substitute: the backend
1511 // refuses, and the refusal aborts the whole set.
1512 struct NoLinks(crate::fs::InMemoryFs);
1513 impl crate::fs::ReadStorage for NoLinks {
1514 async fn read(&self, path: &Path) -> std::io::Result<Vec<u8>> {
1515 self.0.read(path).await
1516 }
1517 async fn read_to_string(&self, path: &Path) -> std::io::Result<String> {
1518 self.0.read_to_string(path).await
1519 }
1520 async fn read_dir(&self, path: &Path) -> std::io::Result<Vec<crate::fs::DirEntry>> {
1521 self.0.read_dir(path).await
1522 }
1523 async fn metadata(&self, path: &Path) -> std::io::Result<crate::fs::Metadata> {
1524 self.0.metadata(path).await
1525 }
1526 // `executable` and `read_link` stay at the defaults: the declines.
1527 }
1528 impl Storage for NoLinks {
1529 async fn write(&self, path: &Path, contents: &[u8]) -> std::io::Result<()> {
1530 self.0.write(path, contents).await
1531 }
1532 async fn create_dir_all(&self, path: &Path) -> std::io::Result<()> {
1533 self.0.create_dir_all(path).await
1534 }
1535 async fn remove_file(&self, path: &Path) -> std::io::Result<()> {
1536 self.0.remove_file(path).await
1537 }
1538 async fn remove_dir_all(&self, path: &Path) -> std::io::Result<()> {
1539 self.0.remove_dir_all(path).await
1540 }
1541 async fn rename(&self, from: &Path, to: &Path) -> std::io::Result<()> {
1542 self.0.rename(from, to).await
1543 }
1544 fn capabilities(&self) -> crate::fs::Capabilities {
1545 self.0.capabilities()
1546 }
1547 // `write_atomic` deliberately stays at the default too: its
1548 // temp-then-rename runs fine over the wrapped backend now that
1549 // `InMemoryFs::rename` replaces an occupied file, which this
1550 // test then exercises for free.
1551 // `set_link` stays at the default: the refusal.
1552 }
1553
1554 let fs = NoLinks(crate::fs::InMemoryFs::new());
1555 block_on(fs.0.write(Path::new("root/before.md"), b"old")).unwrap();
1556 let mut cs = ChangeSet::new();
1557 cs.write("before.md", "new");
1558 cs.set_link("link.md", "target.md");
1559 let err = block_on(cs.apply(&fs, Path::new("root"))).unwrap_err();
1560 assert!(err.to_string().contains("symbolic links"), "{err}");
1561 assert_eq!(
1562 block_on(fs.0.read_to_string(Path::new("root/before.md"))).unwrap(),
1563 "old",
1564 "the write that preceded the refused link must unwind"
1565 );
1566 }
1567
1568 #[cfg(unix)]
1569 #[test]
1570 fn a_failed_set_restores_a_link_a_write_replaced_as_a_link() {
1571 // Capture-by-bytes would follow the link and roll back to a regular
1572 // file holding a copy of the target — link gone, content duplicated,
1573 // "the tree is as it was" quietly false. The link must come back as
1574 // a link, pointing where it pointed.
1575 let root = tmp("rollback-write-over-link");
1576 std::fs::write(root.join("target.md"), "the target").unwrap();
1577 std::os::unix::fs::symlink("target.md", root.join("link.md")).unwrap();
1578
1579 let mut cs = ChangeSet::new();
1580 cs.write("link.md", "replaces the link");
1581 cs.write("doomed.md", "never lands");
1582 block_on(cs.apply(&FailAtWrite::nth(1), &root)).unwrap_err();
1583
1584 let md = std::fs::symlink_metadata(root.join("link.md")).unwrap();
1585 assert!(
1586 md.file_type().is_symlink(),
1587 "the link must come back as a link"
1588 );
1589 assert_eq!(
1590 std::fs::read_link(root.join("link.md")).unwrap(),
1591 PathBuf::from("target.md")
1592 );
1593 assert_eq!(
1594 read(&root, "target.md").as_deref(),
1595 Some("the target"),
1596 "the rollback must not have written through the link"
1597 );
1598 }
1599
1600 #[cfg(unix)]
1601 #[test]
1602 fn a_failed_set_restores_a_link_it_removed_as_a_link() {
1603 // Also the dangling case: `remove_file` removes a link the read-based
1604 // capture could never have read through, so removal must not require
1605 // the target to exist.
1606 let root = tmp("rollback-remove-link");
1607 std::os::unix::fs::symlink("nowhere.md", root.join("dangling.md")).unwrap();
1608
1609 let mut cs = ChangeSet::new();
1610 cs.remove("dangling.md");
1611 cs.write("doomed.md", "never lands");
1612 block_on(cs.apply(&FailAtWrite::nth(0), &root)).unwrap_err();
1613
1614 assert_eq!(
1615 std::fs::read_link(root.join("dangling.md")).unwrap(),
1616 PathBuf::from("nowhere.md"),
1617 "the removed link must come back as the link it was"
1618 );
1619 }
1620
1621 #[cfg(unix)]
1622 #[test]
1623 fn the_execute_bit_is_refused_through_a_link_and_the_set_unwinds() {
1624 // The escape this closes: `set_link` at a path inside the root, then
1625 // `set_executable` at that same path — mode writes follow links, so
1626 // without the refusal the bit lands on the referent, wherever it
1627 // points. The root guard is lexical and cannot see it; the op must.
1628 use std::os::unix::fs::PermissionsExt as _;
1629 let root = tmp("exec-through-link");
1630 let outside = tmp("exec-through-link-outside");
1631 std::fs::write(outside.join("victim.sh"), "#!/bin/sh").unwrap();
1632 let victim = outside.join("victim.sh");
1633 let mode_before = std::fs::metadata(&victim).unwrap().permissions().mode();
1634
1635 let mut cs = ChangeSet::new();
1636 cs.set_link("l", victim.to_str().unwrap());
1637 cs.set_executable("l", true);
1638 let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
1639
1640 assert!(err.to_string().contains("symbolic link"), "{err}");
1641 assert_eq!(
1642 std::fs::metadata(&victim).unwrap().permissions().mode(),
1643 mode_before,
1644 "the referent's mode must be untouched"
1645 );
1646 assert!(
1647 std::fs::symlink_metadata(root.join("l")).is_err(),
1648 "the refused set must unwind the link it made"
1649 );
1650 }
1651
1652 // ---- the durability of an answer ----
1653
1654 #[test]
1655 fn a_set_of_renames_and_removes_is_flushed_before_the_journal_is_dropped() {
1656 // `Ok` means the change survives a power cut. Renames and removes
1657 // edit directory entries no per-op call flushes, so the apply must
1658 // settle that debt — barriers capped by one durable sync — before it
1659 // gives up the journal that certifies the set.
1660 let root = tmp("flush-before-drop");
1661 std::fs::write(root.join("a.md"), "a").unwrap();
1662 std::fs::write(root.join("c.md"), "c").unwrap();
1663 let fs = RecordingFs::local();
1664 let mut cs = ChangeSet::new();
1665 cs.rename("a.md", "sub/b.md");
1666 cs.remove("c.md");
1667 block_on(cs.apply(&fs, &root)).unwrap();
1668
1669 let journal = Journal::default().path_in(&root);
1670 let jtmp = crate::fs::temp_sibling(&journal);
1671 assert_eq!(
1672 fs.events(),
1673 vec![
1674 // The commit point: the journal, via write_atomic.
1675 FsEvent::Write(jtmp.clone()),
1676 FsEvent::Sync(jtmp.clone(), crate::fs::Durability::Ordered),
1677 FsEvent::Rename(jtmp, journal.clone()),
1678 FsEvent::Sync(root.clone(), crate::fs::Durability::Durable),
1679 // The ops.
1680 FsEvent::Rename(root.join("a.md"), root.join("sub/b.md")),
1681 FsEvent::Remove(root.join("c.md")),
1682 // The debt: both touched directories, barriers capped by one
1683 // drain of the root — the anchor, which always exists, where
1684 // whichever debt happened to sort last might not — and only
1685 // then the journal.
1686 FsEvent::Sync(root.join("sub"), crate::fs::Durability::Ordered),
1687 FsEvent::Sync(root.clone(), crate::fs::Durability::Durable),
1688 FsEvent::Remove(journal),
1689 ]
1690 );
1691 }
1692
1693 #[test]
1694 fn a_lone_rename_flushes_the_entries_it_edited() {
1695 // The fast path skips the journal, never the promise.
1696 let root = tmp("lone-rename-flush");
1697 std::fs::write(root.join("a.md"), "a").unwrap();
1698 let fs = RecordingFs::local();
1699 let mut cs = ChangeSet::new();
1700 cs.rename("a.md", "b.md");
1701 block_on(cs.apply(&fs, &root)).unwrap();
1702
1703 assert_eq!(
1704 fs.events(),
1705 vec![
1706 FsEvent::Rename(root.join("a.md"), root.join("b.md")),
1707 FsEvent::Sync(root.clone(), crate::fs::Durability::Durable),
1708 ]
1709 );
1710 }
1711
1712 #[test]
1713 fn a_lone_write_pays_the_staging_flush_and_one_drain() {
1714 // The commonest mutation there is must not get slower: a lone write
1715 // is `replace`'s three steps plus the one durable flush of its
1716 // parent — the same four events `write_atomic` always cost it.
1717 let root = tmp("lone-write-flush");
1718 let fs = RecordingFs::local();
1719 let mut cs = ChangeSet::new();
1720 cs.write("a.md", "a");
1721 block_on(cs.apply(&fs, &root)).unwrap();
1722
1723 let tmp_name = crate::fs::temp_sibling(&root.join("a.md"));
1724 assert_eq!(
1725 fs.events(),
1726 vec![
1727 FsEvent::Write(tmp_name.clone()),
1728 FsEvent::Sync(tmp_name.clone(), crate::fs::Durability::Ordered),
1729 FsEvent::Rename(tmp_name, root.join("a.md")),
1730 FsEvent::Sync(root.clone(), crate::fs::Durability::Durable),
1731 ]
1732 );
1733 }
1734
1735 #[cfg(unix)]
1736 #[test]
1737 fn a_lone_exec_flip_flushes_the_inode_it_edited() {
1738 // A mode is inode metadata: the inode is barriered while its name
1739 // still resolves, and the anchored drain makes the barrier durable.
1740 let root = tmp("lone-exec-flush");
1741 std::fs::write(root.join("run.sh"), "#!/bin/sh").unwrap();
1742 let fs = RecordingFs::local();
1743 let mut cs = ChangeSet::new();
1744 cs.set_executable("run.sh", true);
1745 block_on(cs.apply(&fs, &root)).unwrap();
1746
1747 assert_eq!(
1748 fs.events(),
1749 vec![
1750 FsEvent::SetExecutable(root.join("run.sh"), true),
1751 FsEvent::Sync(root.join("run.sh"), crate::fs::Durability::Ordered),
1752 FsEvent::Sync(root.clone(), crate::fs::Durability::Durable),
1753 ]
1754 );
1755 }
1756
1757 /// A recording double whose `n`th non-journal write fails — the lever for
1758 /// exercising the rollback paths while still reading the event stream.
1759 struct FailingRecorder {
1760 inner: RecordingFs,
1761 writes: std::cell::Cell<usize>,
1762 fail_at: usize,
1763 }
1764 impl crate::fs::ReadStorage for FailingRecorder {
1765 async fn read(&self, path: &Path) -> std::io::Result<Vec<u8>> {
1766 self.inner.read(path).await
1767 }
1768 async fn read_to_string(&self, path: &Path) -> std::io::Result<String> {
1769 self.inner.read_to_string(path).await
1770 }
1771 async fn read_dir(&self, path: &Path) -> std::io::Result<Vec<crate::fs::DirEntry>> {
1772 self.inner.read_dir(path).await
1773 }
1774 async fn metadata(&self, path: &Path) -> std::io::Result<crate::fs::Metadata> {
1775 self.inner.metadata(path).await
1776 }
1777 async fn read_link(&self, path: &Path) -> std::io::Result<Option<PathBuf>> {
1778 self.inner.read_link(path).await
1779 }
1780 }
1781 impl Storage for FailingRecorder {
1782 async fn write(&self, path: &Path, contents: &[u8]) -> std::io::Result<()> {
1783 if !Journal::default().owns_path(path) {
1784 let n = self.writes.get();
1785 self.writes.set(n + 1);
1786 if n == self.fail_at {
1787 return Err(std::io::Error::other("disk full (test)"));
1788 }
1789 }
1790 self.inner.write(path, contents).await
1791 }
1792 async fn create_dir_all(&self, path: &Path) -> std::io::Result<()> {
1793 self.inner.create_dir_all(path).await
1794 }
1795 async fn remove_file(&self, path: &Path) -> std::io::Result<()> {
1796 self.inner.remove_file(path).await
1797 }
1798 async fn remove_dir_all(&self, path: &Path) -> std::io::Result<()> {
1799 self.inner.remove_dir_all(path).await
1800 }
1801 async fn rename(&self, from: &Path, to: &Path) -> std::io::Result<()> {
1802 self.inner.rename(from, to).await
1803 }
1804 fn capabilities(&self) -> crate::fs::Capabilities {
1805 self.inner.capabilities()
1806 }
1807 async fn sync(&self, path: &Path, need: crate::fs::Durability) -> std::io::Result<()> {
1808 self.inner.sync(path, need).await
1809 }
1810 }
1811
1812 #[test]
1813 fn an_abort_flushes_the_restored_state_and_durably_retires_the_journal() {
1814 // `Err` from a clean rollback is a promise too: the restored state is
1815 // flushed, the journal's deletion made durable — so a power cut right
1816 // after the abort cannot resurrect the journal for the next recovery
1817 // to roll the aborted set forward.
1818 let root = tmp("abort-durable");
1819 std::fs::write(root.join("existing.md"), "before").unwrap();
1820 let fs = FailingRecorder {
1821 inner: RecordingFs::local(),
1822 writes: std::cell::Cell::new(0),
1823 fail_at: 1,
1824 };
1825 let mut cs = ChangeSet::new();
1826 cs.write("existing.md", "after");
1827 cs.write("doomed.md", "never lands");
1828 let err = block_on(cs.apply(&fs, &root)).unwrap_err();
1829 assert!(matches!(err, Error::Io(_)), "{err:?}");
1830 assert_eq!(read(&root, "existing.md").as_deref(), Some("before"));
1831
1832 // The tail of the event stream is the abort's certification: restored
1833 // bytes barriered the moment they land (while their name still
1834 // resolves), the edited directories barriered after, the journal
1835 // removed, and its directory drained.
1836 let journal = Journal::default().path_in(&root);
1837 let events = fs.inner.events();
1838 let tail = &events[events.len() - 4..];
1839 assert_eq!(
1840 tail,
1841 &[
1842 FsEvent::Sync(root.join("existing.md"), crate::fs::Durability::Ordered),
1843 FsEvent::Sync(root.clone(), crate::fs::Durability::Ordered),
1844 FsEvent::Remove(journal),
1845 FsEvent::Sync(root.clone(), crate::fs::Durability::Durable),
1846 ],
1847 "events: {events:?}"
1848 );
1849 }
1850
1851 #[test]
1852 fn many_writes_into_one_directory_cost_one_drain() {
1853 // The engineered-away redundancy: per-file durable parent flushes
1854 // would make an N-write set drain the device N+1 times. Through
1855 // `replace` and the batched final pass, exactly two drains remain —
1856 // the journal's commit point, and the cap that certifies the set.
1857 let root = tmp("write-economy");
1858 let fs = RecordingFs::local();
1859 let mut cs = ChangeSet::new();
1860 cs.write("a.md", "a");
1861 cs.write("b.md", "b");
1862 cs.write("c.md", "c");
1863 cs.write("d.md", "d");
1864 block_on(cs.apply(&fs, &root)).unwrap();
1865
1866 let drains = fs
1867 .events()
1868 .iter()
1869 .filter(|e| matches!(e, FsEvent::Sync(_, crate::fs::Durability::Durable)))
1870 .count();
1871 assert_eq!(drains, 2, "events: {:?}", fs.events());
1872 }
1873
1874 #[cfg(unix)]
1875 #[test]
1876 fn a_flipped_bit_survives_its_name_being_renamed_away() {
1877 // The trap the anchored flush exists for: the flip's debt is a name,
1878 // and a later op in the same set moves it. The inode must be
1879 // barriered while the name still resolves, and the one drain must
1880 // land on a path that still exists — the root — never on whichever
1881 // stale name happened to sort last, which `sync` would answer with a
1882 // silent no-op.
1883 let root = tmp("flip-then-rename");
1884 std::fs::write(root.join("z.sh"), "#!/bin/sh").unwrap();
1885 let fs = RecordingFs::local();
1886 let mut cs = ChangeSet::new();
1887 cs.set_executable("z.sh", true);
1888 cs.rename("z.sh", "a.sh");
1889 block_on(cs.apply(&fs, &root)).unwrap();
1890
1891 assert!(is_executable(&root, "a.sh"));
1892 let journal = Journal::default().path_in(&root);
1893 let jtmp = crate::fs::temp_sibling(&journal);
1894 assert_eq!(
1895 fs.events(),
1896 vec![
1897 FsEvent::Write(jtmp.clone()),
1898 FsEvent::Sync(jtmp.clone(), crate::fs::Durability::Ordered),
1899 FsEvent::Rename(jtmp, journal.clone()),
1900 FsEvent::Sync(root.clone(), crate::fs::Durability::Durable),
1901 FsEvent::SetExecutable(root.join("z.sh"), true),
1902 // The inode, while `z.sh` still names it.
1903 FsEvent::Sync(root.join("z.sh"), crate::fs::Durability::Ordered),
1904 FsEvent::Rename(root.join("z.sh"), root.join("a.sh")),
1905 // The cap on the root — which exists — not on the stale name.
1906 FsEvent::Sync(root.clone(), crate::fs::Durability::Durable),
1907 FsEvent::Remove(journal),
1908 ]
1909 );
1910 }
1911
1912 /// A recording double whose `sync` fails once, on the `n`th `Durable`
1913 /// drain of `anchor` — the smallest lever that reaches the certification
1914 /// paths without touching anything else.
1915 struct FailNthDrain {
1916 inner: RecordingFs,
1917 anchor: PathBuf,
1918 drains: std::cell::Cell<usize>,
1919 fail_at: usize,
1920 }
1921 impl crate::fs::ReadStorage for FailNthDrain {
1922 async fn read(&self, path: &Path) -> std::io::Result<Vec<u8>> {
1923 self.inner.read(path).await
1924 }
1925 async fn read_to_string(&self, path: &Path) -> std::io::Result<String> {
1926 self.inner.read_to_string(path).await
1927 }
1928 async fn read_dir(&self, path: &Path) -> std::io::Result<Vec<crate::fs::DirEntry>> {
1929 self.inner.read_dir(path).await
1930 }
1931 async fn metadata(&self, path: &Path) -> std::io::Result<crate::fs::Metadata> {
1932 self.inner.metadata(path).await
1933 }
1934 async fn read_link(&self, path: &Path) -> std::io::Result<Option<PathBuf>> {
1935 self.inner.read_link(path).await
1936 }
1937 }
1938 impl Storage for FailNthDrain {
1939 async fn write(&self, path: &Path, contents: &[u8]) -> std::io::Result<()> {
1940 self.inner.write(path, contents).await
1941 }
1942 async fn create_dir_all(&self, path: &Path) -> std::io::Result<()> {
1943 self.inner.create_dir_all(path).await
1944 }
1945 async fn remove_file(&self, path: &Path) -> std::io::Result<()> {
1946 self.inner.remove_file(path).await
1947 }
1948 async fn remove_dir_all(&self, path: &Path) -> std::io::Result<()> {
1949 self.inner.remove_dir_all(path).await
1950 }
1951 async fn rename(&self, from: &Path, to: &Path) -> std::io::Result<()> {
1952 self.inner.rename(from, to).await
1953 }
1954 fn capabilities(&self) -> crate::fs::Capabilities {
1955 self.inner.capabilities()
1956 }
1957 async fn sync(&self, path: &Path, need: crate::fs::Durability) -> std::io::Result<()> {
1958 if need == crate::fs::Durability::Durable && path == self.anchor.as_path() {
1959 let n = self.drains.get();
1960 self.drains.set(n + 1);
1961 if n == self.fail_at {
1962 return Err(std::io::Error::other("cannot drain (test)"));
1963 }
1964 }
1965 self.inner.sync(path, need).await
1966 }
1967 }
1968
1969 #[test]
1970 fn an_abort_flushes_the_entry_a_restored_removal_recreates() {
1971 // Rolling back a Remove re-creates a directory entry; an abort that
1972 // certified only the bytes would let a power cut keep the removal the
1973 // caller was told never happened.
1974 let root = tmp("abort-remove-entry");
1975 std::fs::create_dir_all(root.join("sub")).unwrap();
1976 std::fs::write(root.join("sub/gone.md"), "kept after all").unwrap();
1977
1978 let fs = FailingRecorder {
1979 inner: RecordingFs::local(),
1980 writes: std::cell::Cell::new(0),
1981 fail_at: 0,
1982 };
1983 let mut cs = ChangeSet::new();
1984 cs.remove("sub/gone.md");
1985 cs.write("doomed.md", "never lands");
1986 block_on(cs.apply(&fs, &root)).unwrap_err();
1987
1988 assert_eq!(
1989 read(&root, "sub/gone.md").as_deref(),
1990 Some("kept after all")
1991 );
1992 let events = fs.inner.events();
1993 let entry_flushed = events
1994 .iter()
1995 .position(|e| matches!(e, FsEvent::Sync(p, _) if *p == root.join("sub")));
1996 let journal_retired = events
1997 .iter()
1998 .rposition(|e| matches!(e, FsEvent::Remove(p) if Journal::default().owns_path(p)));
1999 match (entry_flushed, journal_retired) {
2000 (Some(flush), Some(retire)) => assert!(
2001 flush < retire,
2002 "the recreated entry must be flushed before the journal goes; events: {events:?}"
2003 ),
2004 _ => panic!("expected a sub flush and a journal retirement; events: {events:?}"),
2005 }
2006 }
2007
2008 #[test]
2009 fn a_failed_certification_rolls_the_set_back() {
2010 // "Applied, but perhaps not durable" is neither of the two endpoints
2011 // Ok and Err name — so a flush that fails is treated exactly as a
2012 // failed op, and the caller's Err still means durably-before.
2013 let root = tmp("failed-certification");
2014 std::fs::write(root.join("existing.md"), "before").unwrap();
2015 // Drain #0 is the journal commit's parent flush; #1 is the set's
2016 // certifying cap — the one that fails. The abort's own drain (#2)
2017 // succeeds, so the rollback certifies cleanly.
2018 let fs = FailNthDrain {
2019 inner: RecordingFs::local(),
2020 anchor: root.clone(),
2021 drains: std::cell::Cell::new(0),
2022 fail_at: 1,
2023 };
2024 let mut cs = ChangeSet::new();
2025 cs.write("existing.md", "after");
2026 cs.write("fresh.md", "fresh");
2027 let err = block_on(cs.apply(&fs, &root)).unwrap_err();
2028
2029 assert!(matches!(err, Error::Io(_)), "a clean rollback: {err:?}");
2030 assert_eq!(read(&root, "existing.md").as_deref(), Some("before"));
2031 assert_eq!(read(&root, "fresh.md"), None);
2032 assert!(
2033 !Journal::default().path_in(&root).exists(),
2034 "the abort durably retired the journal"
2035 );
2036 }
2037
2038 #[test]
2039 fn a_failed_rename_set_restores_the_file_the_rename_displaced() {
2040 // The destination's occupant is part of "the tree as it was": the
2041 // rename replaces it (the port contract's load-bearing half), so the
2042 // rollback owes it back — first the mover home, then the victim into
2043 // the vacated name.
2044 let root = tmp("rollback-rename-victim");
2045 std::fs::write(root.join("a.md"), "the mover").unwrap();
2046 std::fs::write(root.join("b.md"), "the victim").unwrap();
2047
2048 let mut cs = ChangeSet::new();
2049 cs.rename("a.md", "b.md");
2050 cs.write("doomed.md", "never lands");
2051 block_on(cs.apply(&FailAtWrite::nth(0), &root)).unwrap_err();
2052
2053 assert_eq!(read(&root, "a.md").as_deref(), Some("the mover"));
2054 assert_eq!(read(&root, "b.md").as_deref(), Some("the victim"));
2055 }
2056
2057 #[test]
2058 fn a_deep_chain_written_through_a_set_is_flushed_link_by_link() {
2059 // The same debt OrderedBatch and the journal's home already pay:
2060 // directories the set freshly mints are entries a power cut can take
2061 // back out from under a durably-flushed file.
2062 let root = tmp("set-chain-flush");
2063 let fs = RecordingFs::local();
2064 let mut cs = ChangeSet::new();
2065 cs.write("deep/nested/a.md", "a");
2066 cs.write("b.md", "b");
2067 block_on(cs.apply(&fs, &root)).unwrap();
2068
2069 for dir in [root.clone(), root.join("deep"), root.join("deep/nested")] {
2070 assert!(
2071 fs.events()
2072 .iter()
2073 .any(|e| matches!(e, FsEvent::Sync(p, _) if *p == dir)),
2074 "{} never flushed; events: {:?}",
2075 dir.display(),
2076 fs.events()
2077 );
2078 }
2079 }
2080
2081 #[test]
2082 fn a_clean_rollback_reports_the_cause_not_a_tear() {
2083 // `Torn` means "this crate cannot say what is on disk" — it must be reserved
2084 // for a rollback that genuinely failed. The commonest rollback of all is a
2085 // write to a *new* file that failed before creating it, whose undo then
2086 // finds nothing to delete; calling that a tear would cry wolf on every
2087 // ordinary full disk. Asserted on the variant, because `Torn`'s message
2088 // embeds the cause and so still matches a "disk full" substring check.
2089 let root = tmp("clean-rollback");
2090 std::fs::write(root.join("existing.md"), "before").unwrap();
2091 let mut cs = ChangeSet::new();
2092 cs.write("existing.md", "after");
2093 cs.write("brand-new.md", "never lands");
2094 let err = block_on(cs.apply(&FailAtWrite::nth(1), &root)).unwrap_err();
2095
2096 assert!(
2097 matches!(err, Error::Io(_)),
2098 "a clean rollback should surface the cause itself, got: {err:?}"
2099 );
2100 assert_eq!(read(&root, "existing.md").as_deref(), Some("before"));
2101 assert_eq!(read(&root, "brand-new.md"), None);
2102 }
2103
2104 #[test]
2105 fn a_failed_write_after_a_rename_moves_the_file_back() {
2106 // The ordering `mutate::rename` actually uses: move the file, then
2107 // rewrite it with its re-relativized links. The write's undo must
2108 // restore the *renamed* bytes so the rename's undo has something to
2109 // move back — the reason undo is recorded per-op, not up front.
2110 let root = tmp("rollback-rename");
2111 std::fs::write(root.join("a.md"), "original").unwrap();
2112 let mut cs = ChangeSet::new();
2113 cs.rename("a.md", "sub/a.md");
2114 cs.write("sub/a.md", "rewritten");
2115 cs.write("parent.md", "never gets here");
2116 let err = block_on(cs.apply(&FailAtWrite::nth(1), &root)).unwrap_err();
2117 assert!(err.to_string().contains("disk full"), "{err}");
2118
2119 assert_eq!(read(&root, "a.md").as_deref(), Some("original"));
2120 assert_eq!(read(&root, "sub/a.md"), None);
2121 }
2122
2123 #[test]
2124 fn a_failed_write_restores_a_removed_file() {
2125 let root = tmp("rollback-remove");
2126 std::fs::write(root.join("gone.md"), "precious").unwrap();
2127 let mut cs = ChangeSet::new();
2128 cs.remove("gone.md");
2129 cs.write("parent.md", "boom");
2130 let err = block_on(cs.apply(&FailAtWrite::nth(0), &root)).unwrap_err();
2131 assert!(err.to_string().contains("disk full"), "{err}");
2132 assert_eq!(read(&root, "gone.md").as_deref(), Some("precious"));
2133 }
2134
2135 #[test]
2136 fn every_document_write_lands_atomically_and_leaves_no_temp_files() {
2137 // The payoff of routing `FileOp::Write` through `write_atomic`: applying a
2138 // set stages each document through a sibling and renames it into place, so
2139 // no reader ever catches one half-written, and a clean apply leaves not one
2140 // staging file behind.
2141 let root = tmp("apply-atomic");
2142 std::fs::write(root.join("parent.md"), "old parent").unwrap();
2143 let fs = RecordingFs::local();
2144 let mut cs = ChangeSet::new();
2145 cs.write("child.md", "child");
2146 cs.write("parent.md", "new parent");
2147 block_on(cs.apply(&fs, &root)).unwrap();
2148
2149 assert_eq!(read(&root, "child.md").as_deref(), Some("child"));
2150 assert_eq!(read(&root, "parent.md").as_deref(), Some("new parent"));
2151
2152 // Every write in the log is either a staging sibling or a rename target —
2153 // never a plain write straight to a document path.
2154 for event in fs.events() {
2155 if let FsEvent::Write(p) = event {
2156 let name = p.file_name().unwrap().to_string_lossy();
2157 assert!(
2158 name.contains("fstx-tmp"),
2159 "wrote a document non-atomically: {name}"
2160 );
2161 }
2162 }
2163 // And nothing staging survives.
2164 let leftovers: Vec<_> = std::fs::read_dir(&root)
2165 .unwrap()
2166 .map(|e| e.unwrap().file_name().to_string_lossy().into_owned())
2167 .filter(|n| n.contains("fstx-tmp"))
2168 .collect();
2169 assert!(
2170 leftovers.is_empty(),
2171 "staging files survived apply: {leftovers:?}"
2172 );
2173 }
2174
2175 #[test]
2176 fn apply_journals_before_touching_documents_and_clears_it_after() {
2177 // The commit-point protocol: the journal is written and renamed into
2178 // place *before* the first document write, and removed *after* the last —
2179 // so a crash is always found with the journal either whole (roll forward)
2180 // or absent (nothing began).
2181 let root = tmp("journal-order");
2182 std::fs::write(root.join("parent.md"), "old parent").unwrap();
2183 let fs = RecordingFs::local();
2184 let mut cs = ChangeSet::new();
2185 cs.write("child.md", "child");
2186 cs.write("parent.md", "new parent");
2187 block_on(cs.apply(&fs, &root)).unwrap();
2188
2189 let events = fs.events();
2190 let journal = Journal::default().path_in(&root);
2191
2192 // The journal is renamed into place before any document write happens.
2193 let journal_committed = events
2194 .iter()
2195 .position(|e| matches!(e, FsEvent::Rename(_, to) if *to == journal))
2196 .expect("journal must be committed");
2197 let first_doc_write = events
2198 .iter()
2199 .position(|e| matches!(e, FsEvent::Write(p) if !Journal::default().owns_path(p)))
2200 .expect("a document must be written");
2201 assert!(
2202 journal_committed < first_doc_write,
2203 "the journal must be durable before any document is touched"
2204 );
2205
2206 // And it is removed at the very end — nothing survives a clean apply.
2207 assert_eq!(events.last(), Some(&FsEvent::Remove(journal.clone())));
2208 assert!(!journal.exists());
2209 }
2210
2211 #[test]
2212 fn a_set_of_one_lands_without_a_journal_at_all() {
2213 // The counterpart to the test above, and the reason it stages two ops: a
2214 // lone op is already indivisible, so the journal that makes *several* land
2215 // together has nothing left to guarantee and is skipped. The assertion is
2216 // the exact event list, because what is being claimed is an absence —
2217 // "contains no journal write" would still pass if the set quietly grew a
2218 // second file operation somewhere else.
2219 let root = tmp("journal-single");
2220 std::fs::write(root.join("doc.md"), "old").unwrap();
2221 let fs = RecordingFs::local();
2222 let mut cs = ChangeSet::new();
2223 cs.write("doc.md", "new");
2224 block_on(cs.apply(&fs, &root)).unwrap();
2225
2226 let (target, temp) = (root.join("doc.md"), root.join(".doc.md.fstx-tmp"));
2227 assert_eq!(std::fs::read_to_string(&target).unwrap(), "new");
2228 assert_eq!(
2229 fs.events(),
2230 vec![
2231 FsEvent::Write(temp.clone()),
2232 FsEvent::Sync(temp.clone(), crate::fs::Durability::Ordered),
2233 FsEvent::Rename(temp, target),
2234 FsEvent::Sync(root.clone(), crate::fs::Durability::Durable),
2235 ],
2236 "a set of one must cost exactly one atomic write and nothing else"
2237 );
2238 assert!(!Journal::default().path_in(&root).exists());
2239 }
2240
2241 #[test]
2242 fn a_set_of_one_still_refuses_to_run_over_a_stale_journal() {
2243 // Skipping the journal must not also skip the *check* for one. An earlier
2244 // change crashed mid-apply and recovery has yet to roll it forward; a save
2245 // that slipped past would be silently overwritten when it finally does.
2246 let root = tmp("journal-single-stale");
2247 std::fs::write(root.join("doc.md"), "old").unwrap();
2248 std::fs::write(
2249 Journal::default().path_in(&root),
2250 "a previous change's intent",
2251 )
2252 .unwrap();
2253
2254 let mut cs = ChangeSet::new();
2255 cs.write("doc.md", "new");
2256 let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
2257
2258 assert!(matches!(err, Error::StaleJournal(_)), "got {err:?}");
2259 assert_eq!(
2260 std::fs::read_to_string(root.join("doc.md")).unwrap(),
2261 "old",
2262 "the refused write must not have happened"
2263 );
2264 }
2265
2266 #[test]
2267 fn a_set_of_one_does_not_read_the_file_it_is_about_to_replace() {
2268 // The undo bookkeeping is what made a save read its own target back, and a
2269 // set of one has no rollback to feed it to. Proven the only way an absent
2270 // read can be: a target that cannot be read at all still writes fine.
2271 let root = tmp("journal-single-unreadable");
2272 let target = root.join("doc.md");
2273 std::fs::write(&target, "old").unwrap();
2274 let mut perms = std::fs::metadata(&target).unwrap().permissions();
2275 #[cfg(unix)]
2276 {
2277 use std::os::unix::fs::PermissionsExt;
2278 perms.set_mode(0o200); // write-only: any read of it fails
2279 }
2280 std::fs::set_permissions(&target, perms).unwrap();
2281
2282 let mut cs = ChangeSet::new();
2283 cs.write("doc.md", "new");
2284 block_on(cs.apply(&StdFs, &root)).expect("a write-only target is still replaceable");
2285
2286 // Reading the result back needs the readability restored first: an
2287 // atomic write preserves the target's mode, so a write-only document is
2288 // still write-only afterwards. That is the point of the mode being
2289 // carried across, and it is why this cannot simply read the file.
2290 #[cfg(unix)]
2291 {
2292 use std::os::unix::fs::PermissionsExt;
2293 assert_eq!(
2294 std::fs::metadata(&target).unwrap().permissions().mode() & 0o777,
2295 0o200,
2296 "replacing the contents must not have changed who may read it"
2297 );
2298 std::fs::set_permissions(&target, std::fs::Permissions::from_mode(0o600)).unwrap();
2299 }
2300 assert_eq!(std::fs::read_to_string(&target).unwrap(), "new");
2301 }
2302
2303 #[test]
2304 fn a_caught_error_reverts_and_leaves_no_journal_behind() {
2305 // An error mid-apply unwinds to the pre-change state *and* clears the
2306 // journal — so a later recovery cannot roll the aborted set forward.
2307 let root = tmp("journal-abort");
2308 std::fs::write(root.join("existing.md"), "before").unwrap();
2309 let mut cs = ChangeSet::new();
2310 cs.write("existing.md", "after");
2311 cs.write("brand-new.md", "never lands");
2312 let err = block_on(cs.apply(&FailAtWrite::nth(1), &root)).unwrap_err();
2313
2314 assert!(err.to_string().contains("disk full"), "{err}");
2315 assert_eq!(read(&root, "existing.md").as_deref(), Some("before"));
2316 assert_eq!(read(&root, "brand-new.md"), None);
2317 assert!(
2318 !Journal::default().path_in(&root).exists(),
2319 "a cleanly-reverted change must not leave a journal to roll forward"
2320 );
2321 }
2322
2323 #[test]
2324 fn a_crash_mid_apply_is_recovered_forward_from_the_journal() {
2325 // The end-to-end crash story: apply writes the journal, a crash strikes
2326 // before the set finishes (modeled by leaving the journal and only the
2327 // first write on disk), and `recover` rolls the rest forward.
2328 let root = tmp("journal-crash");
2329 std::fs::write(root.join("parent.md"), "old parent").unwrap();
2330 let mut cs = ChangeSet::new();
2331 cs.write("child.md", "child");
2332 cs.write("parent.md", "new parent");
2333
2334 // The journal the real apply would have committed at its commit point.
2335 std::fs::write(
2336 Journal::default().path_in(&root),
2337 crate::journal::encode(cs.ops()).unwrap(),
2338 )
2339 .unwrap();
2340 // A crash after the first document landed but before the second.
2341 std::fs::write(root.join("child.md"), "child").unwrap();
2342
2343 let outcome = block_on(crate::journal::recover(&StdFs, &root)).unwrap();
2344 assert_eq!(outcome, crate::journal::Recovered::Applied(2));
2345 assert_eq!(read(&root, "child.md").as_deref(), Some("child"));
2346 assert_eq!(read(&root, "parent.md").as_deref(), Some("new parent"));
2347 assert!(!Journal::default().path_in(&root).exists());
2348 }
2349
2350 #[test]
2351 fn apply_refuses_a_path_that_escapes_the_root() {
2352 // A staged op whose path climbs above the root (a hostile link target that
2353 // resolved to `../escape.md`) is refused before anything — journal or
2354 // document — is written.
2355 let root = tmp("escape-write");
2356 let mut cs = ChangeSet::new();
2357 cs.write("../escape.md", "should never land");
2358 let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
2359 assert!(
2360 matches!(err, Error::Escape(_)),
2361 "expected Escape, got {err:?}"
2362 );
2363 // Nothing was written, in or out of the root, and no journal remains.
2364 assert!(!root.parent().unwrap().join("escape.md").exists());
2365 assert!(!Journal::default().path_in(&root).exists());
2366 }
2367
2368 #[test]
2369 fn apply_refuses_an_absolute_path() {
2370 // An absolute path would ignore the root under `root.join`; it escapes too.
2371 let root = tmp("escape-abs");
2372 let mut cs = ChangeSet::new();
2373 cs.write("/tmp/fstx-abs-escape-should-not-exist.md", "nope");
2374 let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
2375 assert!(
2376 matches!(err, Error::Escape(_)),
2377 "expected Escape, got {err:?}"
2378 );
2379 }
2380
2381 #[test]
2382 fn apply_refuses_to_clobber_a_stale_journal() {
2383 // A journal from a *previous* interrupted change is on disk. Applying a new
2384 // set must refuse rather than overwrite it — the old change would otherwise
2385 // be stranded with no record to recover from.
2386 let root = tmp("stale-journal");
2387 std::fs::write(root.join("doc.md"), "before").unwrap();
2388 // Pretend a prior change crashed mid-apply, leaving a valid journal.
2389 let prior = vec![FileOp::Write {
2390 path: "other.md".into(),
2391 bytes: b"prior".to_vec(),
2392 }];
2393 std::fs::write(
2394 Journal::default().path_in(&root),
2395 crate::journal::encode(&prior).unwrap(),
2396 )
2397 .unwrap();
2398
2399 let mut cs = ChangeSet::new();
2400 cs.write("doc.md", "after");
2401 let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
2402 assert!(
2403 matches!(err, Error::StaleJournal(_)),
2404 "expected StaleJournal, got {err:?}"
2405 );
2406 // The new set did not land, and the old journal is untouched — recovery can
2407 // still complete the interrupted change.
2408 assert_eq!(read(&root, "doc.md").as_deref(), Some("before"));
2409 assert!(Journal::default().path_in(&root).exists());
2410 }
2411
2412 #[test]
2413 fn apply_proceeds_once_the_stale_journal_is_recovered() {
2414 // After recovery clears the journal, the same set applies cleanly — the
2415 // refusal is about an *unrecovered* interruption, not a permanent lock.
2416 let root = tmp("stale-journal-cleared");
2417 std::fs::write(root.join("doc.md"), "before").unwrap();
2418 let prior = vec![FileOp::Write {
2419 path: "other.md".into(),
2420 bytes: b"prior".to_vec(),
2421 }];
2422 std::fs::write(
2423 Journal::default().path_in(&root),
2424 crate::journal::encode(&prior).unwrap(),
2425 )
2426 .unwrap();
2427 block_on(crate::journal::recover(&StdFs, &root)).unwrap();
2428
2429 let mut cs = ChangeSet::new();
2430 cs.write("doc.md", "after");
2431 block_on(cs.apply(&StdFs, &root)).unwrap();
2432 assert_eq!(read(&root, "doc.md").as_deref(), Some("after"));
2433 assert_eq!(read(&root, "other.md").as_deref(), Some("prior"));
2434 }
2435
2436 #[test]
2437 fn an_expectation_that_holds_lets_the_set_apply() {
2438 let root = tmp("expect-holds");
2439 std::fs::write(root.join("doc.md"), "as read").unwrap();
2440 let mut cs = ChangeSet::new();
2441 cs.expect("doc.md", "as read");
2442 cs.write("doc.md", "rewritten");
2443 cs.write("index.md", "points at doc");
2444 block_on(cs.apply(&StdFs, &root)).unwrap();
2445 assert_eq!(read(&root, "doc.md").as_deref(), Some("rewritten"));
2446 assert_eq!(read(&root, "index.md").as_deref(), Some("points at doc"));
2447 }
2448
2449 #[test]
2450 fn a_drifted_expectation_refuses_the_set_before_anything_is_written() {
2451 // The tree moved between the caller's read and the apply. The whole
2452 // set — including ops on paths that did NOT drift — is refused, and
2453 // the refusal precedes the commit point: no document touched, no
2454 // journal written.
2455 let root = tmp("expect-drift");
2456 std::fs::write(root.join("doc.md"), "someone else's edit").unwrap();
2457 let mut cs = ChangeSet::new();
2458 cs.expect("doc.md", "as read");
2459 cs.write("doc.md", "rewritten");
2460 cs.write("index.md", "points at doc");
2461 let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
2462 assert!(
2463 matches!(&err, Error::Drifted(p) if p == Path::new("doc.md")),
2464 "expected Drifted(doc.md), got {err:?}"
2465 );
2466 assert_eq!(
2467 read(&root, "doc.md").as_deref(),
2468 Some("someone else's edit")
2469 );
2470 assert_eq!(read(&root, "index.md"), None);
2471 assert!(!Journal::default().path_in(&root).exists());
2472 }
2473
2474 #[test]
2475 fn an_expectation_of_a_missing_file_is_drift_not_io() {
2476 // "I read these bytes, and now there is no file at all" is the same
2477 // story as "now there are different bytes": something else moved the
2478 // tree. The caller gets the retryable answer, not a bare NotFound.
2479 let root = tmp("expect-gone");
2480 let mut cs = ChangeSet::new();
2481 cs.expect("doc.md", "as read");
2482 cs.write("doc.md", "rewritten");
2483 let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
2484 assert!(
2485 matches!(&err, Error::Drifted(p) if p == Path::new("doc.md")),
2486 "expected Drifted, got {err:?}"
2487 );
2488 }
2489
2490 #[test]
2491 fn an_expected_absence_is_drift_when_the_path_is_occupied() {
2492 let root = tmp("expect-occupied");
2493 std::fs::write(root.join("new.md"), "raced you to it").unwrap();
2494 let mut cs = ChangeSet::new();
2495 cs.expect_absent("new.md");
2496 cs.write("new.md", "mine");
2497 let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
2498 assert!(
2499 matches!(&err, Error::Drifted(p) if p == Path::new("new.md")),
2500 "expected Drifted, got {err:?}"
2501 );
2502 assert_eq!(read(&root, "new.md").as_deref(), Some("raced you to it"));
2503 }
2504
2505 #[test]
2506 #[cfg(unix)]
2507 fn a_dangling_link_counts_as_occupied_for_absence() {
2508 // `try_exists` follows links, so a dangling one answers "no" — but the
2509 // entry is there, and a create that expected absence must see it.
2510 let root = tmp("expect-dangling");
2511 std::os::unix::fs::symlink("points-at-nothing.md", root.join("new.md")).unwrap();
2512 let mut cs = ChangeSet::new();
2513 cs.expect_absent("new.md");
2514 cs.write("new.md", "mine");
2515 let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
2516 assert!(
2517 matches!(&err, Error::Drifted(p) if p == Path::new("new.md")),
2518 "expected Drifted, got {err:?}"
2519 );
2520 }
2521
2522 #[test]
2523 fn expectations_speak_of_the_tree_before_the_set_runs() {
2524 // Expecting a path absent and then writing that same path is the
2525 // ordinary create guard, not a contradiction — and the second apply
2526 // of the very same set drifts, because the first one occupied it.
2527 let root = tmp("expect-preimage");
2528 let mut cs = ChangeSet::new();
2529 cs.expect_absent("new.md");
2530 cs.write("new.md", "mine");
2531 cs.write("index.md", "names it");
2532 block_on(cs.apply(&StdFs, &root)).unwrap();
2533 assert_eq!(read(&root, "new.md").as_deref(), Some("mine"));
2534
2535 let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
2536 assert!(
2537 matches!(&err, Error::Drifted(p) if p == Path::new("new.md")),
2538 "expected Drifted on the second apply, got {err:?}"
2539 );
2540 }
2541
2542 #[test]
2543 fn a_lone_op_is_guarded_exactly_as_a_set_of_many() {
2544 // The fast path skips the journal, never the expectations.
2545 let root = tmp("expect-fast-path");
2546 std::fs::write(root.join("doc.md"), "someone else's edit").unwrap();
2547 let mut cs = ChangeSet::new();
2548 cs.expect("doc.md", "as read");
2549 cs.write("doc.md", "rewritten");
2550 assert_eq!(cs.len(), 1, "this test is about the fast path");
2551 let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
2552 assert!(
2553 matches!(err, Error::Drifted(_)),
2554 "expected Drifted, got {err:?}"
2555 );
2556 assert_eq!(
2557 read(&root, "doc.md").as_deref(),
2558 Some("someone else's edit")
2559 );
2560 }
2561
2562 #[test]
2563 fn an_expectation_only_set_checks_without_writing() {
2564 // A set of zero ops and one expectation is an assertion about the
2565 // tree: Ok when it holds, Drifted when it does not, and no journal
2566 // either way.
2567 let root = tmp("expect-only");
2568 std::fs::write(root.join("doc.md"), "as read").unwrap();
2569 let mut cs = ChangeSet::new();
2570 cs.expect("doc.md", "as read");
2571 assert!(!cs.is_empty(), "an expectation is staged state");
2572 block_on(cs.apply(&StdFs, &root)).unwrap();
2573
2574 std::fs::write(root.join("doc.md"), "moved").unwrap();
2575 let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
2576 assert!(
2577 matches!(err, Error::Drifted(_)),
2578 "expected Drifted, got {err:?}"
2579 );
2580 assert!(!Journal::default().path_in(&root).exists());
2581 }
2582
2583 #[test]
2584 fn a_stale_journal_wins_over_a_drifted_expectation() {
2585 // An unrecovered change means the tree is mid-flight — not yet in any
2586 // state worth comparing against — so the stale refusal comes first.
2587 let root = tmp("expect-stale-first");
2588 std::fs::write(root.join("doc.md"), "drifted").unwrap();
2589 let prior = vec![FileOp::Write {
2590 path: "other.md".into(),
2591 bytes: b"prior".to_vec(),
2592 }];
2593 std::fs::write(
2594 Journal::default().path_in(&root),
2595 crate::journal::encode(&prior).unwrap(),
2596 )
2597 .unwrap();
2598 let mut cs = ChangeSet::new();
2599 cs.expect("doc.md", "as read");
2600 cs.write("doc.md", "rewritten");
2601 let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
2602 assert!(
2603 matches!(err, Error::StaleJournal(_)),
2604 "expected StaleJournal, got {err:?}"
2605 );
2606 }
2607
2608 #[test]
2609 fn recovery_never_rechecks_a_committed_sets_expectations() {
2610 // Expectations are not journaled: the commit point asserts they held.
2611 // A recovered tree would fail its own set's expectations by
2612 // construction — here, the crash landed the very write whose absence
2613 // the set expected — and recovery must complete it regardless.
2614 let root = tmp("expect-recovery");
2615 let mut cs = ChangeSet::new();
2616 cs.expect_absent("new.md");
2617 cs.write("new.md", "mine");
2618 cs.write("index.md", "names it");
2619
2620 // The journal the real apply committed (its encoding carries no
2621 // expectations to recheck), and a crash after the first write.
2622 std::fs::write(
2623 Journal::default().path_in(&root),
2624 crate::journal::encode(cs.ops()).unwrap(),
2625 )
2626 .unwrap();
2627 std::fs::write(root.join("new.md"), "mine").unwrap();
2628
2629 let outcome = block_on(crate::journal::recover(&StdFs, &root)).unwrap();
2630 assert_eq!(outcome, crate::journal::Recovered::Applied(2));
2631 assert_eq!(read(&root, "index.md").as_deref(), Some("names it"));
2632 }
2633
2634 #[test]
2635 fn an_escaping_expectation_path_is_refused() {
2636 // An expectation reads; a set must no more probe outside the root
2637 // than write there.
2638 let root = tmp("expect-escape");
2639 let mut cs = ChangeSet::new();
2640 cs.expect("../secret.md", "sniffed");
2641 cs.write("doc.md", "cover");
2642 let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
2643 assert!(
2644 matches!(err, Error::Escape(_)),
2645 "expected Escape, got {err:?}"
2646 );
2647 assert_eq!(read(&root, "doc.md"), None);
2648 }
2649
2650 #[test]
2651 fn extend_carries_expectations_along() {
2652 let root = tmp("expect-extend");
2653 std::fs::write(root.join("doc.md"), "moved").unwrap();
2654 let mut guarded = ChangeSet::new();
2655 guarded.expect("doc.md", "as read");
2656 guarded.write("doc.md", "rewritten");
2657 let mut cs = ChangeSet::new();
2658 cs.write("index.md", "names it");
2659 cs.extend(guarded);
2660 assert_eq!(cs.expected().len(), 1);
2661 let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
2662 assert!(
2663 matches!(err, Error::Drifted(_)),
2664 "expected Drifted, got {err:?}"
2665 );
2666 assert_eq!(read(&root, "index.md"), None);
2667 }
2668
2669 #[test]
2670 fn staged_ops_are_readable_without_applying() {
2671 // The dry-run view: a set describes writes without performing them.
2672 let root = tmp("dry-run");
2673 let mut cs = ChangeSet::new();
2674 cs.write("child.md", "child");
2675 cs.remove("old.md");
2676 assert_eq!(cs.len(), 2);
2677 assert_eq!(
2678 cs.ops().iter().map(FileOp::path).collect::<Vec<_>>(),
2679 [Path::new("child.md"), Path::new("old.md")]
2680 );
2681 assert_eq!(read(&root, "child.md"), None);
2682 }
2683}