Struct WalWriter 

pub struct WalWriter { /* private fields */ }

Writer for the Write-Ahead Log

Wraps the underlying file in a BufWriter so each append does not pay a write syscall — bytes accumulate in a 64 KiB user-space buffer until sync() (or flush_until()) drains them and then calls sync_all() on the raw file. This is how postgres turns per-record append cost from ~500 ns down to ~5 ns; reddb’s previous per-append write_all directly to the file paid the syscall on every record.

Critical contract: every code path that calls sync_all() on the underlying file must drain the BufWriter first via BufWriter::flush(). Otherwise the bytes in user-space never reach the kernel before fsync, and durability is silently broken.

Implementations

impl WalWriter

pub fn open<P: AsRef<Path>>(path: P) -> Result<Self>

Open a WAL file for writing. Creates it if it doesn’t exist.

pub fn append(&mut self, record: &WalRecord) -> Result<u64>

Append a record to the WAL.

Bytes go into the BufWriter — they are NOT durable on disk after this call returns. Callers that need durability must follow up with WalWriter::sync or WalWriter::flush_until.

Returns the LSN (Log Sequence Number) of the record.

pub fn append_bytes(&mut self, bytes: &[u8]) -> Result<u64>

Write already-encoded bytes and advance the LSN counter to match. Used by the lock-free append path: writers encode + atomically reserve an LSN range outside this writer, the group-commit coordinator drains the pending queue in LSN order, then calls append_bytes for each batch.

The bytes MUST be a valid WalRecord::encode() payload (or a concatenation of such) — no structural validation happens here. The caller is responsible for keeping the on-disk byte offset synchronised with the externally-tracked LSN counter; this method just appends and advances.

pub fn set_current_lsn(&mut self, lsn: u64)

Rewind the writer’s LSN counter to a specific value. Used by the lock-free append path to resync the writer with the externally-tracked next_lsn after a drain batch; the coordinator knows the exact byte offset it just wrote to and needs current_lsn to match so subsequent direct callers of append stay consistent.

pub fn sync(&mut self) -> Result<()>

Force sync to disk.

Drains the user-space BufWriter first, then calls sync_all() on the underlying file so every byte appended since the last sync is durable. Updates durable_lsn so subsequent flush_until calls become no-ops up to current_lsn.

pub fn flush_until(&mut self, target: u64) -> Result<()>

Ensure the WAL is durable on disk at least up to byte offset target. No-op when target <= durable_lsn.

This is the postgres XLogFlush(LSN) analogue. Pager flush paths call this with max(dirty.header.lsn) before writing any data page so the WAL record describing the change is guaranteed to be on disk before the page itself.

pub fn durable_lsn(&self) -> u64

Highest byte offset that is durable on disk. Used by the pager to decide whether a flush_until call would actually need a fsync.

pub fn current_lsn(&self) -> u64

Get current LSN (end of file offset)

pub fn drain_for_group_sync(&mut self) -> Result<(u64, Arc<File>)>

Drain the BufWriter into the kernel and return the captured LSN plus a cloned file handle for the caller to fsync without holding the WAL writer mutex.

Used by the group-commit leader path. The flow is:

1. Take the WAL writer mutex.
2. Call this method — drains user-space buffer to the kernel and captures (target_lsn, sync_handle).
3. Release the WAL writer mutex.
4. Call sync_handle.sync_all() — this is the expensive ~100 µs syscall, and other writers can keep appending while it runs.
5. Take the WAL writer mutex briefly and call [WalWriter::mark_durable(target_lsn)] to publish the new durable position.

The cloned sync_handle shares the same kernel inode with the writer’s file, so sync_all() on the clone flushes ALL bytes that have reached the kernel for that file — including bytes appended by other writers AFTER step 3. This is the coalescing window that makes group commit win.

pub fn mark_durable(&mut self, lsn: u64)

Manually advance durable_lsn after a successful out-of-lock sync_all() performed via WalWriter::drain_for_group_sync.

Monotonic — never lowers durable_lsn. Safe to call with a stale lsn; just becomes a no-op.

pub fn truncate(&mut self) -> Result<()>

Truncate the WAL (usually after checkpoint).

Drains the BufWriter first so no pending bytes hit the file after the truncate. Then resets the underlying file, rewrites the header through the buffered writer (header is small; the followup flush + sync_all makes it durable), and resets LSN bookkeeping.