Skip to main content

fs_transaction/
change.rs

1//! Transactional writes — the unit every change lands through.
2//!
3//! Changing a set of linked files is rarely a single-file operation. A rename
4//! that keeps backlinks intact has to rewrite every file that pointed at the
5//! moved one; a move between directories has to re-relativize the links inside
6//! the file it moved. What is logically one edit is physically several, and
7//! issued one at a time an I/O failure partway through the burst leaves the
8//! tree torn: updated in the files already written, stale in the ones never
9//! reached.
10//!
11//! A [`ChangeSet`] closes that window. A caller stages its writes into one
12//! instead of issuing them, and [`ChangeSet::apply`] executes the whole set as
13//! a unit: each op records how to undo itself *at the moment it runs*, and the
14//! first failure unwinds every op already applied, in reverse. Either the whole
15//! set lands or the tree is as it was.
16//!
17//! ## What this does and does not buy
18//!
19//! This is **error** atomicity and **crash** atomicity across the whole set. A
20//! failed write, a full disk, a permission error, or a rejected edit cannot
21//! leave the tree half-updated, because unwinding puts back every op already
22//! applied. And no single file can be caught half-written even by a power cut:
23//! every [`FileOp::Write`] lands through [`Storage::write_atomic`], which
24//! stages the new bytes in a temporary sibling, flushes them, and renames it
25//! over the target, so an observer sees the whole old file or the whole new
26//! one, never a splice. A `kill -9` or power cut *between* ops leaves the
27//! journal behind; [`crate::journal::recover`] replays it forward to the
28//! fully-applied state. The distinction is worth keeping sharp: caught errors
29//! abort back to the pre-change state, while crashes recover to the committed
30//! state.
31//!
32//! Two smaller honesties, both deliberate:
33//!
34//! - **Directories are not unwound.** Applying a set creates any parent
35//!   directory its writes need; a rollback leaves an empty one behind. An empty
36//!   directory is litter, not a torn tree.
37//! - **Undo is held in memory.** Overwriting or removing a file reads its old
38//!   bytes first so the rollback can put them back, which means a removed
39//!   payload is briefly held whole. The buffer lives only for the length of the
40//!   apply, but it does mean a set is bounded by what fits in memory —
41//!   [`FileOp::CopyFrom`] is the escape hatch for a large payload already on
42//!   disk.
43//!
44//! ## Staging is also a plan
45//!
46//! Because a set is a value that describes writes without performing them, it
47//! is equally an answer to "what *would* this do?" — the shape a `--dry-run`
48//! needs. [`ChangeSet::ops`] is that view, and it is the same sequence `apply`
49//! will execute rather than a reconstruction of it.
50//!
51//! ## Single writer
52//!
53//! A set assumes it is the only thing mutating the tree while it applies. There
54//! is no locking here: two processes applying sets against the same root will
55//! race on the journal, and the [`Error::StaleJournal`] check that guards
56//! against a *previous* interrupted change is a check-then-act, not a mutex. A
57//! caller that needs several writers has to serialize them itself.
58
59use std::path::{Path, PathBuf};
60
61use crate::error::{Error, Result};
62use crate::fs::Storage;
63use crate::journal::Journal;
64
65/// One staged filesystem operation. Paths are **root-relative** — the root
66/// is joined on at [`apply`](ChangeSet::apply) time, so a set is portable
67/// between trees and prints readably in a dry run.
68#[derive(Debug, Clone, PartialEq, Eq)]
69pub enum FileOp {
70    /// Write `bytes` to `path`, creating it (and any missing parent directory)
71    /// or replacing it wholesale.
72    Write {
73        /// The file to write.
74        path: PathBuf,
75        /// Its full new contents.
76        bytes: Vec<u8>,
77    },
78    /// Move `from` to `to`, creating any missing parent directory of `to`.
79    Rename {
80        /// The current path.
81        from: PathBuf,
82        /// The new path.
83        to: PathBuf,
84    },
85    /// Remove the file at `path`. It must exist.
86    Remove {
87        /// The file to remove.
88        path: PathBuf,
89    },
90    /// Copy the bytes already on disk at `source` to `path`, verbatim.
91    ///
92    /// [`Write`](FileOp::Write) with the payload left where it lies. The journal
93    /// records the *source path* instead of the bytes, so a set that writes a
94    /// large payload costs O(path) of journal rather than a second copy of every
95    /// byte — which is what makes putting a whole captured tree back
96    /// tractable, where `Write` would duplicate every byte of it into the
97    /// journal at the commit point.
98    ///
99    /// **The source must be immutable for the lifetime of the change**, because
100    /// that is the entire correctness argument. A `Write` journals the exact bytes
101    /// it intends, so replay after a crash is deterministic by construction; a
102    /// `CopyFrom` journals a reference, and replay is deterministic only if the
103    /// referent cannot have changed underneath it. A content-addressed blob
104    /// satisfies this by definition — its path *is* the digest of its
105    /// contents, so bytes found there are the bytes intended, or the file is gone
106    /// and replay fails loudly. Do not point this at a mutable file, at a
107    /// path some other op in the same set writes, or at anything outside the
108    /// root.
109    ///
110    /// This bounds *journal* growth, not peak memory: rollback still buffers the
111    /// bytes it overwrites, exactly as `Write` does.
112    CopyFrom {
113        /// The file to write.
114        path: PathBuf,
115        /// The root-relative file to copy from — immutable, and ideally
116        /// content-addressed.
117        source: PathBuf,
118    },
119}
120
121impl FileOp {
122    /// The path this op ultimately affects — the destination for a write or a
123    /// rename, the victim for a remove. What a dry run lists.
124    pub fn path(&self) -> &Path {
125        match self {
126            FileOp::Write { path, .. } | FileOp::CopyFrom { path, .. } => path,
127            FileOp::Rename { to, .. } => to,
128            FileOp::Remove { path } => path,
129        }
130    }
131}
132
133/// A set of writes staged as one unit, applied all-or-nothing by
134/// [`apply`](ChangeSet::apply).
135///
136/// Built by the mutation ops as they compute their edits, and applied once at
137/// the end. Ops execute in the order they were staged: a set is a *sequence*,
138/// not a bag, because `rename`-then-write and remove-then-rewrite-the-parent
139/// depend on it.
140#[derive(Debug, Clone, Default, PartialEq, Eq)]
141pub struct ChangeSet {
142    ops: Vec<FileOp>,
143}
144
145impl ChangeSet {
146    /// An empty set.
147    pub fn new() -> Self {
148        Self::default()
149    }
150
151    /// Stage a write of `contents` to `path` (root-relative).
152    pub fn write(&mut self, path: impl Into<PathBuf>, contents: impl Into<Vec<u8>>) -> &mut Self {
153        self.ops.push(FileOp::Write {
154            path: path.into(),
155            bytes: contents.into(),
156        });
157        self
158    }
159
160    /// Stage a move from `from` to `to` (both root-relative).
161    pub fn rename(&mut self, from: impl Into<PathBuf>, to: impl Into<PathBuf>) -> &mut Self {
162        self.ops.push(FileOp::Rename {
163            from: from.into(),
164            to: to.into(),
165        });
166        self
167    }
168
169    /// Stage the removal of `path` (root-relative).
170    pub fn remove(&mut self, path: impl Into<PathBuf>) -> &mut Self {
171        self.ops.push(FileOp::Remove { path: path.into() });
172        self
173    }
174
175    /// Stage a copy of the file at `source` to `path` (both root-relative),
176    /// instead of carrying its bytes through the set.
177    ///
178    /// See [`FileOp::CopyFrom`] for the immutability the source has to satisfy —
179    /// it is what keeps crash recovery deterministic.
180    pub fn copy_from(&mut self, path: impl Into<PathBuf>, source: impl Into<PathBuf>) -> &mut Self {
181        self.ops.push(FileOp::CopyFrom {
182            path: path.into(),
183            source: source.into(),
184        });
185        self
186    }
187
188    /// The staged ops, in execution order. The dry-run view.
189    pub fn ops(&self) -> &[FileOp] {
190        &self.ops
191    }
192
193    /// The bytes this set will leave at `path`, if it writes it — the *last*
194    /// write staged, since a later one supersedes an earlier.
195    ///
196    /// This is what makes a set safe to read back mid-build. A document can be
197    /// touched twice by one op (`reparent` repoints a child that is somehow its
198    /// own old parent, and must then edit the text it just staged rather than the
199    /// stale copy on disk), and before staging existed the second edit read the
200    /// first one's *write* off the filesystem. Nothing hits the filesystem now
201    /// until commit, so the set has to answer instead.
202    ///
203    /// `None` if the set does not write `path` — including when it renames or
204    /// removes it, and including a [`FileOp::CopyFrom`], whose bytes are on disk
205    /// at the source rather than held in the set. This is deliberately a lookup,
206    /// not a filesystem overlay: it resolves the one hazard staging introduces and
207    /// nothing more. A caller that must read back a path it staged a copy to has
208    /// to read the source itself.
209    pub fn staged(&self, path: &Path) -> Option<&[u8]> {
210        self.ops.iter().rev().find_map(|op| match op {
211            FileOp::Write { path: p, bytes } if p == path => Some(bytes.as_slice()),
212            _ => None,
213        })
214    }
215
216    /// Where this set moves `path` to, if it moves it — following a chain of
217    /// renames to the final destination. `None` if the set leaves it where it is.
218    ///
219    /// The companion to [`staged`](Self::staged) for anything holding a path this
220    /// set might move out from under it. The registry is exactly that: it knows
221    /// which document it persists into, and a set that renames that document has
222    /// to be followed, or its write lands at a path the set just emptied.
223    pub fn renamed_to(&self, path: &Path) -> Option<PathBuf> {
224        let mut current = path.to_path_buf();
225        let mut moved = false;
226        for op in &self.ops {
227            if let FileOp::Rename { from, to } = op
228                && *from == current
229            {
230                current = to.clone();
231                moved = true;
232            }
233        }
234        moved.then_some(current)
235    }
236
237    /// Whether nothing is staged — [`apply`](ChangeSet::apply) would be a no-op.
238    pub fn is_empty(&self) -> bool {
239        self.ops.is_empty()
240    }
241
242    /// The number of staged ops.
243    pub fn len(&self) -> usize {
244        self.ops.len()
245    }
246
247    /// Append `other`'s ops after this set's, consuming it.
248    pub fn extend(&mut self, other: ChangeSet) -> &mut Self {
249        self.ops.extend(other.ops);
250        self
251    }
252
253    /// Execute every staged op against `fs`, rooted at `root`, as one unit —
254    /// crash-atomically, behind the [default](Journal::DEFAULT_NAME)
255    /// write-ahead journal.
256    ///
257    /// Shorthand for [`Journal::default().apply(..)`](Journal::apply); reach
258    /// for a named [`Journal`] when the default file name would collide with
259    /// something the tree already means. Whichever is used here, the same one
260    /// has to be used to [`recover`](Journal::recover) an interruption of it.
261    pub async fn apply<FS: Storage>(&self, fs: &FS, root: &Path) -> Result<()> {
262        Journal::default().apply(self, fs, root).await
263    }
264}
265
266impl Journal {
267    /// Execute every op `changes` staged against `fs`, rooted at `root`, as one
268    /// unit — crash-atomically, behind this write-ahead journal.
269    ///
270    /// The set's intent is journaled and flushed *before* any document is
271    /// touched (see [`crate::journal`]); that flush is the commit point. From
272    /// there the ops run in order, each recording how to undo itself:
273    ///
274    /// - **On success**, the journal is removed and the change is done.
275    /// - **On an error** (a full disk, a permission fault), every op already
276    ///   applied is unwound in reverse, the tree is restored to what it was,
277    ///   and the journal is cleared — the mutation aborts as if it never began.
278    /// - **On a crash** (a `kill -9`, a power cut) there is no error to catch and
279    ///   no chance to unwind, so the journal simply survives; the next
280    ///   [`crate::journal::recover`] rolls the set forward to its fully-applied
281    ///   state. An interrupted change set is therefore always resolved to a
282    ///   consistent tree — fully before it on a caught error, fully after it
283    ///   on a crash.
284    ///
285    /// A set of **one** op skips the journal entirely: a single op is already
286    /// indivisible on a backend claiming `atomic_replace`, so there is no
287    /// multi-file window for a journal to close, and a crash leaves the op either
288    /// wholly done or wholly not — the same two states a recovered set lands on.
289    /// It is the ordinary shape of a save, and it costs one file operation rather
290    /// than four.
291    ///
292    /// The rare exception is a rollback that *itself* fails ([`Error::Torn`]):
293    /// the pre-change state could not be restored, so — rather than leave an
294    /// unknown one — the journal is kept, and recovery will later roll the set
295    /// forward to the consistent applied state. Either way the tree lands on
296    /// a state this crate can name.
297    ///
298    /// Takes `fs`/`root` rather than a higher-level object so a bootstrap
299    /// that must write two files before the tree exists can still land them
300    /// together.
301    ///
302    /// Whatever recovers an interruption of this call must name the same
303    /// journal — see [`Journal`] for why the two operations live together.
304    pub async fn apply<FS: Storage>(
305        &self,
306        changes: &ChangeSet,
307        fs: &FS,
308        root: &Path,
309    ) -> Result<()> {
310        if changes.ops.is_empty() {
311            return Ok(());
312        }
313        // Clamp every staged path to the root *before* anything is
314        // written or journaled. A set is built from root-relative,
315        // already-normalized paths when a caller builds them that way — but
316        // `apply` also lands sets assembled from data it did not author,
317        // and a link target that resolves to `../../../etc/passwd` must be refused
318        // rather than let an apply write outside the tree it was pointed at.
319        for op in &changes.ops {
320            match op {
321                FileOp::Write { path, .. } | FileOp::Remove { path } => {
322                    guard_in_root(path)?;
323                }
324                FileOp::Rename { from, to } => {
325                    guard_in_root(from)?;
326                    guard_in_root(to)?;
327                }
328                // The source is clamped too: it is read, and a set assembled by a
329                // caller must not be able to pull `../../../etc/passwd` into the
330                // tree any more than it may write out of one.
331                FileOp::CopyFrom { path, source } => {
332                    guard_in_root(path)?;
333                    guard_in_root(source)?;
334                }
335            }
336        }
337        // Refuse to clobber a journal left by a *previous* interrupted change. Its
338        // presence means an earlier mutation crashed mid-apply and has not been
339        // recovered; overwriting it with this set's intent would strand the old
340        // change half-applied with no record of how to finish it. Recovery
341        // ([`Journal::recover`]) must complete it first. A journal this same apply is about to write does not exist yet,
342        // so this only ever fires on a genuinely stale one.
343        let journal = self.path_in(root);
344        if fs.try_exists(&journal).await? {
345            return Err(Error::StaleJournal(journal));
346        }
347        // A set of one needs no journal. The journal exists to make *several*
348        // file operations land as one unit; a lone op is already indivisible on a
349        // backend claiming `atomic_replace` — a `write_atomic` is all-or-nothing
350        // by construction, and a lone `rename` or `unlink` is atomic by the
351        // filesystem's own guarantee. Journaling it would write, flush, and then
352        // delete a second file in order to restate a promise the op already
353        // carries, roughly tripling what the commonest mutation there is — saving
354        // one document — costs in writes and flushes alike.
355        //
356        // What this gives up is *liveness*, not safety. With a journal, a crash
357        // mid-apply is rolled forward to the applied state by the next
358        // [`crate::journal::recover`]; without one, a crash simply means the op
359        // did not happen. For a set of one those are the only two states there
360        // are — no caller can observe a half-applied set of one — so all that
361        // changes is which side of the atomic instant a crash lands on, never
362        // whether it lands on one at all.
363        //
364        // The stale-journal refusal above still applies: this path writes no
365        // journal, but it must not slip a write past an *earlier* interrupted
366        // change that recovery has yet to roll forward, or recovery would later
367        // overwrite what was just written.
368        if changes.ops.len() == 1 && fs.capabilities().atomic_replace {
369            // No undo to record, either. Nothing preceded this op that could need
370            // unwinding, and every failure mode leaves the target untouched — so
371            // the reflexive read of the very file about to be overwritten, whose
372            // only purpose is to hold the old bytes for a rollback that cannot
373            // happen here, goes with it.
374            return exec(fs, root, &changes.ops[0], None).await;
375        }
376        // The commit point: durably record the whole intent before touching a
377        // single document. `write_atomic` flushes it, so a crash finds the
378        // journal whole or not at all — never half-written.
379        fs.write_atomic(&journal, &crate::journal::encode(&changes.ops)?)
380            .await?;
381
382        let mut undo: Vec<Undo> = Vec::new();
383        for op in &changes.ops {
384            let Err(cause) = exec(fs, root, op, Some(&mut undo)).await else {
385                continue;
386            };
387            return Err(match unwind(fs, undo).await {
388                // Reverted cleanly: the change aborted, so the journal must go —
389                // otherwise recovery would later roll this very set *forward*,
390                // undoing the abort. If even the delete fails, fall through to
391                // `Torn` and let recovery complete the set instead.
392                Ok(()) => match fs.remove_file(&journal).await {
393                    Ok(()) => cause,
394                    Err(cleanup) => Error::Torn {
395                        cause: cause.to_string(),
396                        rollback: cleanup.to_string(),
397                    },
398                },
399                // Could not revert: keep the journal so recovery rolls the set
400                // forward to the consistent applied state.
401                Err(rollback) => Error::Torn {
402                    cause: cause.to_string(),
403                    rollback: rollback.to_string(),
404                },
405            });
406        }
407        // Applied cleanly. Drop the journal; if this delete fails, a later
408        // recovery re-applies the set idempotently and clears it — harmless.
409        fs.remove_file(&journal).await?;
410        Ok(())
411    }
412}
413
414/// How to reverse one applied op, recorded against the state that op found.
415///
416/// Recorded *per op at execution time*, not for the whole set up front, because
417/// ops in a set are not independent: `rename` moves `a.md` to `sub/a.md` and
418/// then rewrites `sub/a.md`'s re-relativized links, so the write's undo has to
419/// restore the bytes the rename put there — a snapshot taken before the set ran
420/// would say "`sub/a.md` did not exist; delete it", and the rename's undo would
421/// then have nothing to move back. Paths here are already root-joined.
422enum Undo {
423    /// Put these bytes back (the file existed and was overwritten or removed).
424    Restore { path: PathBuf, bytes: Vec<u8> },
425    /// Delete the file (it did not exist before the write created it).
426    ///
427    /// Recorded *before* the write it reverses, because a write that fails
428    /// partway still leaves a file behind — so this has to tolerate finding
429    /// nothing there, which is the case where the write failed before creating
430    /// anything at all. Undoing nothing is success, not a torn tree.
431    Delete { path: PathBuf },
432    /// Move `from` back to `to`.
433    Rename { from: PathBuf, to: PathBuf },
434}
435
436/// Apply one op, optionally recording how to reverse it.
437///
438/// `undo` is `None` only for a set of one, which has no rollback to feed: see
439/// the fast path in [`ChangeSet::apply`]. Recording is not merely unused there,
440/// it is worth skipping — for a write it costs a full read of the file about to
441/// be replaced.
442async fn exec<FS: Storage>(
443    fs: &FS,
444    root: &Path,
445    op: &FileOp,
446    undo: Option<&mut Vec<Undo>>,
447) -> Result<()> {
448    match op {
449        FileOp::Write { path, bytes } => {
450            let full = root.join(path);
451            // Record the undo *before* writing: a write that fails partway
452            // (a full disk) leaves a truncated file, and restoring the old
453            // bytes over it is exactly the repair.
454            if let Some(undo) = undo {
455                match fs.read(&full).await {
456                    Ok(old) => undo.push(Undo::Restore {
457                        path: full.clone(),
458                        bytes: old,
459                    }),
460                    Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
461                        undo.push(Undo::Delete { path: full.clone() });
462                    }
463                    Err(e) => return Err(e.into()),
464                }
465            }
466            ensure_parent(fs, &full).await?;
467            // Land the document through the atomic-replace protocol, so even a
468            // crash mid-write cannot expose a half-written file: the write goes to
469            // a staging sibling and is renamed into place. On a backend without
470            // atomic rename this degrades to a plain durable write (see
471            // [`Storage::write_atomic`]) — the per-file guarantee follows the
472            // backend's declared capabilities.
473            fs.write_atomic(&full, bytes).await?;
474        }
475        FileOp::Rename { from, to } => {
476            let (from_full, to_full) = (root.join(from), root.join(to));
477            ensure_parent(fs, &to_full).await?;
478            fs.rename(&from_full, &to_full).await?;
479            if let Some(undo) = undo {
480                undo.push(Undo::Rename {
481                    from: to_full,
482                    to: from_full,
483                });
484            }
485        }
486        FileOp::Remove { path } => {
487            let full = root.join(path);
488            match undo {
489                // The removed bytes are the undo, so they have to be read out
490                // before the file goes.
491                Some(undo) => {
492                    let old = fs.read(&full).await?;
493                    fs.remove_file(&full).await?;
494                    undo.push(Undo::Restore {
495                        path: full,
496                        bytes: old,
497                    });
498                }
499                None => fs.remove_file(&full).await?,
500            }
501        }
502        // A `Write` whose bytes were left at the source. The read happens here, at
503        // execution time, rather than when the op was staged — that is the whole
504        // saving, and it is why the source has to be immutable.
505        FileOp::CopyFrom { path, source } => {
506            let (full, source_full) = (root.join(path), root.join(source));
507            let bytes = fs.read(&source_full).await?;
508            if let Some(undo) = undo {
509                match fs.read(&full).await {
510                    Ok(old) => undo.push(Undo::Restore {
511                        path: full.clone(),
512                        bytes: old,
513                    }),
514                    Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
515                        undo.push(Undo::Delete { path: full.clone() });
516                    }
517                    Err(e) => return Err(e.into()),
518                }
519            }
520            ensure_parent(fs, &full).await?;
521            fs.write_atomic(&full, &bytes).await?;
522        }
523    }
524    Ok(())
525}
526
527/// Reverse every recorded op, last-applied first. Best-effort: a step that fails
528/// does not abandon the rest — the more that is put back the better — and the
529/// first failure is what gets reported.
530async fn unwind<FS: Storage>(fs: &FS, undo: Vec<Undo>) -> Result<()> {
531    let mut first_error = None;
532    for step in undo.into_iter().rev() {
533        let result = match step {
534            Undo::Restore { path, bytes } => fs.write(&path, &bytes).await,
535            // Already absent is already undone — see `Undo::Delete`. Reporting it
536            // would raise `Error::Torn` over the single most ordinary rollback
537            // there is: a write to a new file that failed before creating it.
538            Undo::Delete { path } => match fs.remove_file(&path).await {
539                Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(()),
540                other => other,
541            },
542            Undo::Rename { from, to } => fs.rename(&from, &to).await,
543        };
544        if let Err(e) = result
545            && first_error.is_none()
546        {
547            first_error = Some(e);
548        }
549    }
550    match first_error {
551        Some(e) => Err(e.into()),
552        None => Ok(()),
553    }
554}
555
556/// Refuse a staged path that would resolve outside the root. Defers to
557/// [`crate::path::escapes_root`], so a caller that guards its *reads* with the
558/// same function clamps both directions to the exact same boundary.
559fn guard_in_root(path: &Path) -> Result<()> {
560    if crate::path::escapes_root(path) {
561        return Err(Error::Escape(path.to_path_buf()));
562    }
563    Ok(())
564}
565
566/// Create `full`'s parent directory if it is missing. Unconditional (rather than
567/// staged as its own op) because a directory is not part of the document graph:
568/// it is an artifact of *where* a write lands, so it belongs to the write.
569async fn ensure_parent<FS: Storage>(fs: &FS, full: &Path) -> Result<()> {
570    if let Some(dir) = full.parent() {
571        fs.create_dir_all(dir).await?;
572    }
573    Ok(())
574}
575
576#[cfg(test)]
577mod tests {
578    use super::*;
579    use crate::exec::block_on;
580    use crate::fs::StdFs;
581    use crate::fs_faults::{FailAtWrite, FsEvent, RecordingFs};
582    use crate::journal::Journal;
583
584    fn tmp(name: &str) -> PathBuf {
585        let dir = std::env::temp_dir().join(format!("fstx-change-{name}"));
586        let _ = std::fs::remove_dir_all(&dir);
587        std::fs::create_dir_all(&dir).unwrap();
588        dir
589    }
590
591    fn read(root: &Path, rel: &str) -> Option<String> {
592        std::fs::read_to_string(root.join(rel)).ok()
593    }
594
595    #[test]
596    fn applies_every_op_in_order() {
597        let root = tmp("apply");
598        std::fs::write(root.join("parent.md"), "old parent").unwrap();
599        let mut cs = ChangeSet::new();
600        cs.write("child.md", "child");
601        cs.write("parent.md", "new parent");
602        block_on(cs.apply(&StdFs, &root)).unwrap();
603        assert_eq!(read(&root, "child.md").as_deref(), Some("child"));
604        assert_eq!(read(&root, "parent.md").as_deref(), Some("new parent"));
605    }
606
607    #[test]
608    fn creates_missing_parent_directories() {
609        let root = tmp("mkdir");
610        let mut cs = ChangeSet::new();
611        cs.write("deep/nested/child.md", "hi");
612        block_on(cs.apply(&StdFs, &root)).unwrap();
613        assert_eq!(read(&root, "deep/nested/child.md").as_deref(), Some("hi"));
614    }
615
616    #[test]
617    fn a_copy_lands_the_source_bytes_and_leaves_the_source_alone() {
618        let root = tmp("copy");
619        std::fs::create_dir_all(root.join("history/blobs/9f")).unwrap();
620        std::fs::write(root.join("history/blobs/9f/86d081"), "captured").unwrap();
621        std::fs::write(root.join("notes.md"), "damaged").unwrap();
622
623        let mut cs = ChangeSet::new();
624        cs.copy_from("notes.md", "history/blobs/9f/86d081");
625        // A path that does not exist yet gets its parent made, as a write does.
626        cs.copy_from("deep/fresh.md", "history/blobs/9f/86d081");
627        block_on(cs.apply(&StdFs, &root)).unwrap();
628
629        assert_eq!(read(&root, "notes.md").as_deref(), Some("captured"));
630        assert_eq!(read(&root, "deep/fresh.md").as_deref(), Some("captured"));
631        // The blob is shared by every event naming it: read, never consumed.
632        assert_eq!(
633            read(&root, "history/blobs/9f/86d081").as_deref(),
634            Some("captured")
635        );
636    }
637
638    #[test]
639    fn a_failed_copy_rolls_back_exactly_as_a_failed_write_does() {
640        // A copy is a write whose payload was fetched late, so it must record the
641        // same undo: restore what it overwrote, delete what it created.
642        let root = tmp("rollback-copy");
643        std::fs::create_dir_all(root.join("history/blobs/9f")).unwrap();
644        std::fs::write(root.join("history/blobs/9f/86d081"), "captured").unwrap();
645        std::fs::write(root.join("notes.md"), "damaged").unwrap();
646
647        let mut cs = ChangeSet::new();
648        cs.copy_from("notes.md", "history/blobs/9f/86d081");
649        cs.copy_from("fresh.md", "history/blobs/9f/86d081");
650        cs.write("doomed.md", "never lands");
651        let err = block_on(cs.apply(&FailAtWrite::nth(2), &root)).unwrap_err();
652        assert!(err.to_string().contains("disk full"), "{err}");
653
654        assert_eq!(read(&root, "notes.md").as_deref(), Some("damaged"));
655        assert_eq!(read(&root, "fresh.md"), None);
656    }
657
658    #[test]
659    fn a_copy_from_a_missing_source_fails_before_the_target_is_touched() {
660        // The half-synced event: the manifest names a blob the transport has not
661        // delivered. Better to fail the set than to write a hole into the tree.
662        let root = tmp("copy-missing-source");
663        std::fs::write(root.join("notes.md"), "damaged").unwrap();
664        let mut cs = ChangeSet::new();
665        cs.copy_from("notes.md", "history/blobs/9f/86d081");
666        assert!(block_on(cs.apply(&StdFs, &root)).is_err());
667        assert_eq!(read(&root, "notes.md").as_deref(), Some("damaged"));
668    }
669
670    #[test]
671    fn a_copy_cannot_read_from_outside_the_root() {
672        // The source is read, so it is clamped like every written path: a set a
673        // caller assembled must not be able to pull the host's files into the tree.
674        let root = tmp("copy-escape");
675        let mut cs = ChangeSet::new();
676        cs.copy_from("stolen.md", "../../../etc/passwd");
677        let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
678        assert!(matches!(err, Error::Escape(_)), "{err:?}");
679        assert_eq!(read(&root, "stolen.md"), None);
680    }
681
682    #[test]
683    fn a_failed_write_restores_the_files_already_written() {
684        let root = tmp("rollback-write");
685        std::fs::write(root.join("parent.md"), "old parent").unwrap();
686        std::fs::write(root.join("child.md"), "old child").unwrap();
687
688        // Three writes staged; the third fails.
689        let mut cs = ChangeSet::new();
690        cs.write("child.md", "new child");
691        cs.write("parent.md", "new parent");
692        cs.write("third.md", "third");
693        let err = block_on(cs.apply(&FailAtWrite::nth(2), &root)).unwrap_err();
694        assert!(err.to_string().contains("disk full"), "{err}");
695
696        // Everything is as it was found — no half-linked tree.
697        assert_eq!(read(&root, "child.md").as_deref(), Some("old child"));
698        assert_eq!(read(&root, "parent.md").as_deref(), Some("old parent"));
699    }
700
701    #[test]
702    fn a_failed_write_deletes_files_the_set_had_created() {
703        let root = tmp("rollback-create");
704        let mut cs = ChangeSet::new();
705        cs.write("fresh.md", "fresh");
706        cs.write("doomed.md", "doomed");
707        let err = block_on(cs.apply(&FailAtWrite::nth(1), &root)).unwrap_err();
708        assert!(err.to_string().contains("disk full"), "{err}");
709        // The file the set created before failing is gone, not orphaned.
710        assert_eq!(read(&root, "fresh.md"), None);
711    }
712
713    #[test]
714    fn a_clean_rollback_reports_the_cause_not_a_tear() {
715        // `Torn` means "this crate cannot say what is on disk" — it must be reserved
716        // for a rollback that genuinely failed. The commonest rollback of all is a
717        // write to a *new* file that failed before creating it, whose undo then
718        // finds nothing to delete; calling that a tear would cry wolf on every
719        // ordinary full disk. Asserted on the variant, because `Torn`'s message
720        // embeds the cause and so still matches a "disk full" substring check.
721        let root = tmp("clean-rollback");
722        std::fs::write(root.join("existing.md"), "before").unwrap();
723        let mut cs = ChangeSet::new();
724        cs.write("existing.md", "after");
725        cs.write("brand-new.md", "never lands");
726        let err = block_on(cs.apply(&FailAtWrite::nth(1), &root)).unwrap_err();
727
728        assert!(
729            matches!(err, Error::Io(_)),
730            "a clean rollback should surface the cause itself, got: {err:?}"
731        );
732        assert_eq!(read(&root, "existing.md").as_deref(), Some("before"));
733        assert_eq!(read(&root, "brand-new.md"), None);
734    }
735
736    #[test]
737    fn a_failed_write_after_a_rename_moves_the_file_back() {
738        // The ordering `mutate::rename` actually uses: move the file, then
739        // rewrite it with its re-relativized links. The write's undo must
740        // restore the *renamed* bytes so the rename's undo has something to
741        // move back — the reason undo is recorded per-op, not up front.
742        let root = tmp("rollback-rename");
743        std::fs::write(root.join("a.md"), "original").unwrap();
744        let mut cs = ChangeSet::new();
745        cs.rename("a.md", "sub/a.md");
746        cs.write("sub/a.md", "rewritten");
747        cs.write("parent.md", "never gets here");
748        let err = block_on(cs.apply(&FailAtWrite::nth(1), &root)).unwrap_err();
749        assert!(err.to_string().contains("disk full"), "{err}");
750
751        assert_eq!(read(&root, "a.md").as_deref(), Some("original"));
752        assert_eq!(read(&root, "sub/a.md"), None);
753    }
754
755    #[test]
756    fn a_failed_write_restores_a_removed_file() {
757        let root = tmp("rollback-remove");
758        std::fs::write(root.join("gone.md"), "precious").unwrap();
759        let mut cs = ChangeSet::new();
760        cs.remove("gone.md");
761        cs.write("parent.md", "boom");
762        let err = block_on(cs.apply(&FailAtWrite::nth(0), &root)).unwrap_err();
763        assert!(err.to_string().contains("disk full"), "{err}");
764        assert_eq!(read(&root, "gone.md").as_deref(), Some("precious"));
765    }
766
767    #[test]
768    fn every_document_write_lands_atomically_and_leaves_no_temp_files() {
769        // The payoff of routing `FileOp::Write` through `write_atomic`: applying a
770        // set stages each document through a sibling and renames it into place, so
771        // no reader ever catches one half-written, and a clean apply leaves not one
772        // staging file behind.
773        let root = tmp("apply-atomic");
774        std::fs::write(root.join("parent.md"), "old parent").unwrap();
775        let fs = RecordingFs::local();
776        let mut cs = ChangeSet::new();
777        cs.write("child.md", "child");
778        cs.write("parent.md", "new parent");
779        block_on(cs.apply(&fs, &root)).unwrap();
780
781        assert_eq!(read(&root, "child.md").as_deref(), Some("child"));
782        assert_eq!(read(&root, "parent.md").as_deref(), Some("new parent"));
783
784        // Every write in the log is either a staging sibling or a rename target —
785        // never a plain write straight to a document path.
786        for event in fs.events() {
787            if let FsEvent::Write(p) = event {
788                let name = p.file_name().unwrap().to_string_lossy();
789                assert!(
790                    name.contains("fstx-tmp"),
791                    "wrote a document non-atomically: {name}"
792                );
793            }
794        }
795        // And nothing staging survives.
796        let leftovers: Vec<_> = std::fs::read_dir(&root)
797            .unwrap()
798            .map(|e| e.unwrap().file_name().to_string_lossy().into_owned())
799            .filter(|n| n.contains("fstx-tmp"))
800            .collect();
801        assert!(
802            leftovers.is_empty(),
803            "staging files survived apply: {leftovers:?}"
804        );
805    }
806
807    #[test]
808    fn apply_journals_before_touching_documents_and_clears_it_after() {
809        // The commit-point protocol: the journal is written and renamed into
810        // place *before* the first document write, and removed *after* the last —
811        // so a crash is always found with the journal either whole (roll forward)
812        // or absent (nothing began).
813        let root = tmp("journal-order");
814        std::fs::write(root.join("parent.md"), "old parent").unwrap();
815        let fs = RecordingFs::local();
816        let mut cs = ChangeSet::new();
817        cs.write("child.md", "child");
818        cs.write("parent.md", "new parent");
819        block_on(cs.apply(&fs, &root)).unwrap();
820
821        let events = fs.events();
822        let journal = Journal::default().path_in(&root);
823
824        // The journal is renamed into place before any document write happens.
825        let journal_committed = events
826            .iter()
827            .position(|e| matches!(e, FsEvent::Rename(_, to) if *to == journal))
828            .expect("journal must be committed");
829        let first_doc_write = events
830            .iter()
831            .position(|e| matches!(e, FsEvent::Write(p) if !Journal::default().owns_path(p)))
832            .expect("a document must be written");
833        assert!(
834            journal_committed < first_doc_write,
835            "the journal must be durable before any document is touched"
836        );
837
838        // And it is removed at the very end — nothing survives a clean apply.
839        assert_eq!(events.last(), Some(&FsEvent::Remove(journal.clone())));
840        assert!(!journal.exists());
841    }
842
843    #[test]
844    fn a_set_of_one_lands_without_a_journal_at_all() {
845        // The counterpart to the test above, and the reason it stages two ops: a
846        // lone op is already indivisible, so the journal that makes *several* land
847        // together has nothing left to guarantee and is skipped. The assertion is
848        // the exact event list, because what is being claimed is an absence —
849        // "contains no journal write" would still pass if the set quietly grew a
850        // second file operation somewhere else.
851        let root = tmp("journal-single");
852        std::fs::write(root.join("doc.md"), "old").unwrap();
853        let fs = RecordingFs::local();
854        let mut cs = ChangeSet::new();
855        cs.write("doc.md", "new");
856        block_on(cs.apply(&fs, &root)).unwrap();
857
858        let (target, temp) = (root.join("doc.md"), root.join(".doc.md.fstx-tmp"));
859        assert_eq!(std::fs::read_to_string(&target).unwrap(), "new");
860        assert_eq!(
861            fs.events(),
862            vec![
863                FsEvent::Write(temp.clone()),
864                FsEvent::Sync(temp.clone(), crate::fs::Durability::Ordered),
865                FsEvent::Rename(temp, target),
866                FsEvent::Sync(root.clone(), crate::fs::Durability::Durable),
867            ],
868            "a set of one must cost exactly one atomic write and nothing else"
869        );
870        assert!(!Journal::default().path_in(&root).exists());
871    }
872
873    #[test]
874    fn a_set_of_one_still_refuses_to_run_over_a_stale_journal() {
875        // Skipping the journal must not also skip the *check* for one. An earlier
876        // change crashed mid-apply and recovery has yet to roll it forward; a save
877        // that slipped past would be silently overwritten when it finally does.
878        let root = tmp("journal-single-stale");
879        std::fs::write(root.join("doc.md"), "old").unwrap();
880        std::fs::write(
881            Journal::default().path_in(&root),
882            "a previous change's intent",
883        )
884        .unwrap();
885
886        let mut cs = ChangeSet::new();
887        cs.write("doc.md", "new");
888        let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
889
890        assert!(matches!(err, Error::StaleJournal(_)), "got {err:?}");
891        assert_eq!(
892            std::fs::read_to_string(root.join("doc.md")).unwrap(),
893            "old",
894            "the refused write must not have happened"
895        );
896    }
897
898    #[test]
899    fn a_set_of_one_does_not_read_the_file_it_is_about_to_replace() {
900        // The undo bookkeeping is what made a save read its own target back, and a
901        // set of one has no rollback to feed it to. Proven the only way an absent
902        // read can be: a target that cannot be read at all still writes fine.
903        let root = tmp("journal-single-unreadable");
904        let target = root.join("doc.md");
905        std::fs::write(&target, "old").unwrap();
906        let mut perms = std::fs::metadata(&target).unwrap().permissions();
907        #[cfg(unix)]
908        {
909            use std::os::unix::fs::PermissionsExt;
910            perms.set_mode(0o200); // write-only: any read of it fails
911        }
912        std::fs::set_permissions(&target, perms).unwrap();
913
914        let mut cs = ChangeSet::new();
915        cs.write("doc.md", "new");
916        block_on(cs.apply(&StdFs, &root)).expect("a write-only target is still replaceable");
917
918        // Reading the result back needs the readability restored first: an
919        // atomic write preserves the target's mode, so a write-only document is
920        // still write-only afterwards. That is the point of the mode being
921        // carried across, and it is why this cannot simply read the file.
922        #[cfg(unix)]
923        {
924            use std::os::unix::fs::PermissionsExt;
925            assert_eq!(
926                std::fs::metadata(&target).unwrap().permissions().mode() & 0o777,
927                0o200,
928                "replacing the contents must not have changed who may read it"
929            );
930            std::fs::set_permissions(&target, std::fs::Permissions::from_mode(0o600)).unwrap();
931        }
932        assert_eq!(std::fs::read_to_string(&target).unwrap(), "new");
933    }
934
935    #[test]
936    fn a_caught_error_reverts_and_leaves_no_journal_behind() {
937        // An error mid-apply unwinds to the pre-change state *and* clears the
938        // journal — so a later recovery cannot roll the aborted set forward.
939        let root = tmp("journal-abort");
940        std::fs::write(root.join("existing.md"), "before").unwrap();
941        let mut cs = ChangeSet::new();
942        cs.write("existing.md", "after");
943        cs.write("brand-new.md", "never lands");
944        let err = block_on(cs.apply(&FailAtWrite::nth(1), &root)).unwrap_err();
945
946        assert!(err.to_string().contains("disk full"), "{err}");
947        assert_eq!(read(&root, "existing.md").as_deref(), Some("before"));
948        assert_eq!(read(&root, "brand-new.md"), None);
949        assert!(
950            !Journal::default().path_in(&root).exists(),
951            "a cleanly-reverted change must not leave a journal to roll forward"
952        );
953    }
954
955    #[test]
956    fn a_crash_mid_apply_is_recovered_forward_from_the_journal() {
957        // The end-to-end crash story: apply writes the journal, a crash strikes
958        // before the set finishes (modeled by leaving the journal and only the
959        // first write on disk), and `recover` rolls the rest forward.
960        let root = tmp("journal-crash");
961        std::fs::write(root.join("parent.md"), "old parent").unwrap();
962        let mut cs = ChangeSet::new();
963        cs.write("child.md", "child");
964        cs.write("parent.md", "new parent");
965
966        // The journal the real apply would have committed at its commit point.
967        std::fs::write(
968            Journal::default().path_in(&root),
969            crate::journal::encode(cs.ops()).unwrap(),
970        )
971        .unwrap();
972        // A crash after the first document landed but before the second.
973        std::fs::write(root.join("child.md"), "child").unwrap();
974
975        let outcome = block_on(crate::journal::recover(&StdFs, &root)).unwrap();
976        assert_eq!(outcome, crate::journal::Recovered::Applied(2));
977        assert_eq!(read(&root, "child.md").as_deref(), Some("child"));
978        assert_eq!(read(&root, "parent.md").as_deref(), Some("new parent"));
979        assert!(!Journal::default().path_in(&root).exists());
980    }
981
982    #[test]
983    fn apply_refuses_a_path_that_escapes_the_root() {
984        // A staged op whose path climbs above the root (a hostile link target that
985        // resolved to `../escape.md`) is refused before anything — journal or
986        // document — is written.
987        let root = tmp("escape-write");
988        let mut cs = ChangeSet::new();
989        cs.write("../escape.md", "should never land");
990        let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
991        assert!(
992            matches!(err, Error::Escape(_)),
993            "expected Escape, got {err:?}"
994        );
995        // Nothing was written, in or out of the root, and no journal remains.
996        assert!(!root.parent().unwrap().join("escape.md").exists());
997        assert!(!Journal::default().path_in(&root).exists());
998    }
999
1000    #[test]
1001    fn apply_refuses_an_absolute_path() {
1002        // An absolute path would ignore the root under `root.join`; it escapes too.
1003        let root = tmp("escape-abs");
1004        let mut cs = ChangeSet::new();
1005        cs.write("/tmp/fstx-abs-escape-should-not-exist.md", "nope");
1006        let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
1007        assert!(
1008            matches!(err, Error::Escape(_)),
1009            "expected Escape, got {err:?}"
1010        );
1011    }
1012
1013    #[test]
1014    fn apply_refuses_to_clobber_a_stale_journal() {
1015        // A journal from a *previous* interrupted change is on disk. Applying a new
1016        // set must refuse rather than overwrite it — the old change would otherwise
1017        // be stranded with no record to recover from.
1018        let root = tmp("stale-journal");
1019        std::fs::write(root.join("doc.md"), "before").unwrap();
1020        // Pretend a prior change crashed mid-apply, leaving a valid journal.
1021        let prior = vec![FileOp::Write {
1022            path: "other.md".into(),
1023            bytes: b"prior".to_vec(),
1024        }];
1025        std::fs::write(
1026            Journal::default().path_in(&root),
1027            crate::journal::encode(&prior).unwrap(),
1028        )
1029        .unwrap();
1030
1031        let mut cs = ChangeSet::new();
1032        cs.write("doc.md", "after");
1033        let err = block_on(cs.apply(&StdFs, &root)).unwrap_err();
1034        assert!(
1035            matches!(err, Error::StaleJournal(_)),
1036            "expected StaleJournal, got {err:?}"
1037        );
1038        // The new set did not land, and the old journal is untouched — recovery can
1039        // still complete the interrupted change.
1040        assert_eq!(read(&root, "doc.md").as_deref(), Some("before"));
1041        assert!(Journal::default().path_in(&root).exists());
1042    }
1043
1044    #[test]
1045    fn apply_proceeds_once_the_stale_journal_is_recovered() {
1046        // After recovery clears the journal, the same set applies cleanly — the
1047        // refusal is about an *unrecovered* interruption, not a permanent lock.
1048        let root = tmp("stale-journal-cleared");
1049        std::fs::write(root.join("doc.md"), "before").unwrap();
1050        let prior = vec![FileOp::Write {
1051            path: "other.md".into(),
1052            bytes: b"prior".to_vec(),
1053        }];
1054        std::fs::write(
1055            Journal::default().path_in(&root),
1056            crate::journal::encode(&prior).unwrap(),
1057        )
1058        .unwrap();
1059        block_on(crate::journal::recover(&StdFs, &root)).unwrap();
1060
1061        let mut cs = ChangeSet::new();
1062        cs.write("doc.md", "after");
1063        block_on(cs.apply(&StdFs, &root)).unwrap();
1064        assert_eq!(read(&root, "doc.md").as_deref(), Some("after"));
1065        assert_eq!(read(&root, "other.md").as_deref(), Some("prior"));
1066    }
1067
1068    #[test]
1069    fn staged_ops_are_readable_without_applying() {
1070        // The dry-run view: a set describes writes without performing them.
1071        let root = tmp("dry-run");
1072        let mut cs = ChangeSet::new();
1073        cs.write("child.md", "child");
1074        cs.remove("old.md");
1075        assert_eq!(cs.len(), 2);
1076        assert_eq!(
1077            cs.ops().iter().map(FileOp::path).collect::<Vec<_>>(),
1078            [Path::new("child.md"), Path::new("old.md")]
1079        );
1080        assert_eq!(read(&root, "child.md"), None);
1081    }
1082}