pub struct WriteAheadLog { /* private fields */ }Expand description
Write-ahead log for crash recovery
Provides durable storage for mutations before they reach the memtable. Every mutation is serialized to an append-only log and fsync’d to disk.
§Usage
use cqlite_core::storage::write_engine::{WriteAheadLog, Mutation};
use std::path::Path;
// Create a new WAL
let mut wal = WriteAheadLog::create(Path::new("/data"))?;
// Append mutations (serialized with CRC32)
// let mutation = Mutation::new(...);
// wal.append(&mutation)?;
// Explicit sync to disk
wal.sync()?;
// On recovery, replay all valid entries
// let mutations = wal.replay()?;Implementations§
Source§impl WriteAheadLog
impl WriteAheadLog
Sourcepub const DEFAULT_BUFFER_SIZE: usize = 4096
pub const DEFAULT_BUFFER_SIZE: usize = 4096
Default buffer size (4 KB)
Sourcepub const WAL_FILENAME: &'static str = "commitlog.wal"
pub const WAL_FILENAME: &'static str = "commitlog.wal"
WAL file name
Sourcepub fn create(dir: &Path) -> Result<Self>
pub fn create(dir: &Path) -> Result<Self>
Create a new WAL in the specified directory
This creates a new WAL file with the default buffer size (4 KB). If a WAL already exists in the directory, it will be truncated.
§Arguments
dir- Directory where the WAL file will be created
§Returns
A new WriteAheadLog instance ready for appending.
§Errors
Returns an error if the directory doesn’t exist or the file cannot be created.
Sourcepub fn create_with_buffer_size(dir: &Path, buffer_size: usize) -> Result<Self>
pub fn create_with_buffer_size(dir: &Path, buffer_size: usize) -> Result<Self>
Create a new WAL with a custom buffer size
§Arguments
dir- Directory where the WAL file will be createdbuffer_size- Size of the append buffer in bytes
Sourcepub fn open_existing(path: &Path) -> Result<Self>
pub fn open_existing(path: &Path) -> Result<Self>
Open an existing WAL file for appending
This opens an existing WAL and seeks to the end, ready for new appends. Use this for recovery scenarios where you want to append to an existing log.
§Arguments
path- Path to the existing WAL file
§Returns
A WriteAheadLog positioned at the end of the file.
§Errors
Returns an error if the file doesn’t exist or cannot be opened.
Sourcepub fn has_pending_corrupt_tail(&self) -> bool
pub fn has_pending_corrupt_tail(&self) -> bool
True if open_existing found mid-stream corruption whose valid prefix has
not yet been reset (issue #1391). While this holds, the LIVE WAL still
carries the corrupt tail and appends would land after it (lost on the next
replay) — the caller must preserve the segment aside and then call
reset_to_valid_prefix before any write.
Sourcepub fn reset_to_valid_prefix(&mut self) -> Result<Option<u64>>
pub fn reset_to_valid_prefix(&mut self) -> Result<Option<u64>>
After a lossy (Corruption) recovery, trim the LIVE WAL back to its last
CRC-valid prefix so post-recovery appends resume at a replayable position
(issue #1391). This is a no-op unless open_existing recorded a pending
corrupt tail; the caller MUST first preserve the raw segment aside (the
trim is destructive to the on-disk corrupt bytes, though the aside copy
and the retained RecoveryReport survive).
Returns the new (valid-prefix) length when a trim occurred, or None when
there was nothing pending.
§Errors
Returns an error if the buffer flush, set_len, or fsync fails.
Sourcepub fn append(&mut self, mutation: &Mutation) -> Result<()>
pub fn append(&mut self, mutation: &Mutation) -> Result<()>
Append a mutation to the WAL
This serializes the mutation using bincode and writes it to the buffer.
The entry is not guaranteed to be on disk until sync() is called.
§Entry Format
[u32 LE: entry_length]
[u32 LE: crc32]
[bytes: serialized mutation]§Arguments
mutation- The mutation to append
§Errors
Returns an error if serialization fails or the write fails.
Sourcepub fn sync(&mut self) -> Result<()>
pub fn sync(&mut self) -> Result<()>
Sync the WAL to disk (fsync)
This flushes the buffer and calls fsync to ensure all data is written to persistent storage. This is required for durability guarantees.
§Errors
Returns an error if the flush or sync operation fails.
Sourcepub fn replay(&self) -> Result<RecoveryReport>
pub fn replay(&self) -> Result<RecoveryReport>
Replay all valid entries from the WAL
Reads the WAL from the beginning and deserializes all valid entries. This is used during crash recovery to rebuild the memtable.
§Corruption Handling (issue #1391)
Recovery is fail-fast-then-report — see RecoveryReport for the full
rationale. In summary:
- CRC mismatch / implausible length on a fully-present entry: the
framing past this point cannot be resynced authoritatively (no sync
markers), so replay STOPS (
RecoveryReport::stopped_early) and records the loss. It does NOT advance by the untrusted length (that was the misalignment bug that decoded garbage). - CRC-valid but undecodable payload: the entry boundary is
trustworthy, so this single entry is skipped
(
RecoveryReport::corrupt_entries) and replay continues. - Complete header + short payload at EOF: a complete-header entry
whose declared length cannot be satisfied is CORRUPTION (issue #1391
r5), not a clean torn tail — it is indistinguishable from a length
bit-flip that overshoots EOF and swallows valid successors. Replay stops
early and records the loss (
RecoveryReport::stopped_early). - Torn header (short 8-byte header at EOF): an interrupted final append that never finished its header — there is provably no complete entry to lose, so replay stops cleanly (not counted as corruption).
- Valid entries: deserialized and returned in order.
§Returns
A RecoveryReport. Callers MUST check RecoveryReport::is_clean
before treating recovery as complete; a non-clean report signals data
loss that must be surfaced, not silently truncated over.
§Errors
Returns an error if the WAL file cannot be opened or read (an I/O fault distinct from in-band corruption, which is reported, not errored).
Sourcepub fn replay_each<F>(&self, f: F) -> Result<RecoveryReport>
pub fn replay_each<F>(&self, f: F) -> Result<RecoveryReport>
Streaming form of replay: invoke f once per recovered
mutation instead of accumulating a whole-log Vec<Mutation> (issue #1661).
Control flow — decode/skip/stop/error semantics and the on-disk framing —
is byte-for-byte identical to replay; this is a pure
memory refactor. The two differences are internal:
- A single payload buffer is reused across entries (its capacity grows to the largest entry seen and is retained), so the peak read buffer is bounded by the largest single entry rather than allocated fresh per entry.
- Each successfully decoded mutation is passed to
frather than pushed onto aVec, so the caller (e.g. crash recovery) can insert it into the memtable immediately and never hold the whole log resident.
The returned RecoveryReport carries the same corruption metadata
(corrupt_entries,
stopped_early,
bytes_skipped) as replay, but its
mutations vec is left empty — the mutations
were streamed to f. Callers MUST still consult
RecoveryReport::is_clean.
If f returns an error, replay stops and the error propagates unchanged.
§Errors
Returns an error if the WAL file cannot be opened or read (an I/O fault
distinct from in-band corruption, which is reported, not errored), or if
f returns an error for a recovered mutation.
Sourcepub fn truncate(&mut self) -> Result<()>
pub fn truncate(&mut self) -> Result<()>
Truncate the WAL (clear all entries)
This is used after a successful flush to memtable/SSTable, removing old entries that are no longer needed for recovery.
On success this also lifts any pending corrupt-tail guard (issue #1391):
an emptied WAL has no corrupt tail, so subsequent append() calls must
not be rejected by the stale fail-closed guard.
§Errors
Returns an error if the truncate operation fails. This flattens the
phase distinction of WriteAheadLog::truncate_checked; the
durability handoff (issue #1392) calls truncate_checked directly so it
can react differently to a failure that occurs after the WAL contents
have already been zeroed.
Sourcepub fn truncate_checked(&mut self) -> Result<(), TruncateError>
pub fn truncate_checked(&mut self) -> Result<(), TruncateError>
Truncate the WAL, distinguishing failures before the WAL contents
are mutated from failures after set_len(0) has already zeroed it
(issue #1392).
The sequence is: flush pending writes, set_len(0), fsync, seek to
start. The set_len(0) call is the point of no return: once it
succeeds the on-disk WAL is empty and is no longer a replayable
recovery marker, even if a following fsync/seek fails.
- A failure at or before
set_len(0)returnsTruncateError::BeforeMutation— the WAL is still intact and can be safely left in place for replay. - A failure after
set_len(0)returnsTruncateError::AfterMutation— the WAL has been zeroed and callers MUST NOT treat it as intact.
Sourcepub fn rotate(self, dir: &Path) -> Result<Self>
pub fn rotate(self, dir: &Path) -> Result<Self>
Rotate the WAL (create a new one, keeping the old)
This creates a new WAL file with a timestamp suffix and returns a new
WriteAheadLog instance. The old WAL file is left intact for archival
or backup purposes.
The old file is renamed to: commitlog.wal.{timestamp}
§Arguments
dir- Directory where the new WAL will be created
§Returns
A new WriteAheadLog instance ready for appending.
§Errors
Returns an error if the rotation fails.