nodedb_wal/double_write/
mod.rs

1// SPDX-License-Identifier: Apache-2.0
2
3//! Double-write buffer for torn write protection.
4//!
5//! NVMe drives guarantee atomic 4 KiB sector writes but NOT atomic writes
6//! for larger pages (e.g., 16 KiB). If power fails mid-write on a 16 KiB
7//! page, the WAL page can be partially written (torn).
8//!
9//! CRC32C detects torn writes during replay, but without the double-write
10//! buffer, the record is lost — even though it was acknowledged to the client.
11//!
12//! The double-write buffer solves this:
13//! 1. Before writing to WAL, write the record to the double-write file.
14//! 2. `fsync` the double-write file.
15//! 3. Write to the WAL file.
16//! 4. `fsync` the WAL file.
17//!
18//! On recovery, if a WAL record's CRC fails:
19//! - Check the double-write buffer for an intact copy (verify CRC).
20//! - If found, use the double-write copy to reconstruct the WAL page.
21//! - If not found, the record is truly lost (pre-fsync crash).
22//!
23//! The double-write file is a fixed-size circular buffer. Only the most
24//! recent N records are kept — older ones are overwritten. This is fine
25//! because torn writes can only happen on the most recent write.
26//!
27//! Slots are staged in memory as records are appended and reach disk at the
28//! next flush, ahead of the WAL fsync. A flush writes only the slots that
29//! survive in the ring, so its cost is at most `N` slot writes however many
30//! records the batch holds. See the `staging` module.
31//!
32//! ## Why the WAL alone is not enough
33//!
34//! Losing the DWB is not a cosmetic downgrade. A CRC failure with no DWB copy
35//! is indistinguishable, to [`crate::reader`] and [`crate::torn_tail`], from
36//! the end of the committed prefix: nothing higher-LSN follows the last record
37//! in a segment, so a torn final record is classified as an unfsynced tail and
38//! dropped without a word. Every degradation of the DWB therefore has to be
39//! observable — see [`status`] and [`metrics`].
40//!
41//! ## O_DIRECT mode
42//!
43//! When the parent WAL uses `O_DIRECT`, the DWB can also be opened with
44//! `O_DIRECT` (`DwbMode::Direct`). This:
45//! - Keeps the page cache free of DWB bytes — the O_DIRECT WAL was
46//!   specifically designed not to warm the cache, and a buffered DWB
47//!   undoes that by writing the exact same payload through the cache.
48//! - Surfaces DWB bytes in block-layer iostat traffic alongside the WAL.
49//!
50//! The on-disk layout is the same in both modes (one aligned header block
51//! followed by fixed-stride slots, all block-aligned) so a DWB written in
52//! one mode can be read in the other.
53
54pub mod buffer;
55pub mod layout;
56pub mod metrics;
57pub mod mode;
58pub(crate) mod raw_io;
59pub mod recover;
60pub(crate) mod staging;
61pub mod status;
62
63pub use buffer::DoubleWriteBuffer;
64pub use layout::{slot_record_max, slot_stride};
65pub use metrics::{
66    wal_dwb_bytes_written_total, wal_dwb_degradations_total, wal_dwb_unprotected_records_total,
67};
68pub use mode::DwbMode;
69pub use status::{DwbDegradation, DwbMirror, DwbProtection, DwbSkipReason};
nodedb_wal/double_write/mod.rs

nodedb_wal/double_write/
mod.rs
