wal-db 1.0.0

Write-ahead log primitive for Rust storage engines. Durable, recoverable, lock-free append path. The WAL substrate under lsm-db, txn-db, raft-io, and Hive DB.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130

# wal-db v1.0.0 — Stable

**The first stable release.** wal-db is a write-ahead log primitive for Rust
storage engines: the durable, append-only substrate a database writes to before it
acknowledges anything. With 1.0 the public API is frozen until 2.0 and the on-disk
format is frozen for the 1.x line — a log written by any 1.x release reads back on
any other.

There are no functional changes since 0.9.1. 1.0.0 is the commitment to a surface
that has been built, measured, and hardened across the 0.x series.

## What wal-db gives you

A WAL is the workhorse no database avoids: every state change is appended to a
durable log *before* it is acknowledged, and that log rebuilds state after a
crash. Most Rust databases bury their WAL inside the engine; wal-db publishes it as
a clean, composable primitive so an LSM tree, a B-tree store, a transaction
manager, and a Raft log can all share one well-tested implementation.

- **Lock-free multi-writer append.** Many threads append at once with no global
  lock. Each append reserves its byte range with a single atomic step, and that
  range's start offset *is* the record's LSN — so reservations never overlap or
  reorder, and seeking to an LSN is O(1).
- **Explicit, two-step durability.** `append` returns when the record is in the OS
  page cache; `sync` returns when it is on stable storage. The two are never
  conflated — the single most common way to lose data with a WAL.
- **Group commit.** Concurrent `sync` calls coalesce into one fsync, so the cost of
  durability is amortised across everyone committing together.
- **Platform-correct flush.** `fdatasync` on Linux, `FlushFileBuffers` on Windows,
  and `fcntl(F_FULLFSYNC)` on macOS — not plain `fsync`, which leaves data in the
  drive's write cache.
- **Torn-write detection and self-healing recovery.** A CRC32C checksum per record;
  on open, recovery stops at the first incomplete or corrupt record and truncates
  that torn tail. A corrupt length can never drive a wild allocation.
- **Recovery policies.** Stop at the first damaged record, or skip past it for
  forensic partial recovery.
- **Segment rotation.** Optionally stripe the log across bounded segment files for
  bounded recovery and archival, with records spanning boundaries freely.
- **Compaction, both ends.** `iter_from` replays from any LSN; `truncate_after`
  drops a tail (a Raft conflict), and `truncate_before` reclaims a prefix (a
  checkpoint), durably and crash-safely.
- **Optional typed records.** Serialise any value via `pack-io` behind a feature;
  the byte-record API is unchanged when it is off.
- **Pluggable backend.** File-backed by default, with a `WalStore` trait for custom
  and in-memory stores.

## The common case is four calls

```rust
use wal_db::Wal;

# fn apply(_lsn: wal_db::Lsn, _bytes: &[u8]) {}
# fn main() -> Result<(), wal_db::WalError> {
# let dir = tempfile::tempdir().map_err(wal_db::WalError::from)?;
# let path = dir.path().join("app.wal");
let wal = Wal::open(&path)?;          // open (recovers any torn tail)
let _lsn = wal.append(b"a change")?;  // append (into the page cache)
wal.sync()?;                          // sync (the durability barrier)

for entry in wal.iter()? {            // iter (replay on restart)
    let entry = entry?;
    apply(entry.lsn(), entry.data());
}
# Ok(())
# }
```

## Stability guarantees

- **API frozen until 2.0.** Every public item is stable. `WalError` and
  `RecoveryPolicy` are `#[non_exhaustive]` and `WalConfig` is a builder, so the
  surface can still grow additively within the 1.x line without a breaking change.
- **On-disk format frozen for 1.x.** The record framing (frozen since 0.3.0) and
  the segment layout (since 0.3.1) will not change in a backward-incompatible way
  within 1.x. The normative specification is [`docs/ON_DISK_FORMAT.md`](../ON_DISK_FORMAT.md).
- **MSRV 1.85**, Rust 2024 edition. Cross-platform: Linux, macOS, Windows.

## How it was hardened

- **Fuzzing** — a continuous `cargo-fuzz` recovery harness; recovery never panics
  or over-allocates on arbitrary bytes across millions of inputs.
- **Model checking** — loom verifies the lock-free append and group-commit paths
  under exhaustive interleavings.
- **Adversarial and fault-injection tests** — hostile recovery inputs (garbage
  prefixes, implausible lengths, corrupt and torn markers) and injected I/O
  failures (disk-full, fsync failure) through the `WalStore` seam.
- **Property tests** — prefix compaction across random record sets, segment sizes,
  and truncation points, reopened and checked for completeness.
- **Benchmarked honestly** — against a hand-rolled inline WAL; numbers and method
  in [`docs/BENCHMARKS.md`](../BENCHMARKS.md).

The full suite is green on Windows and Linux, on stable and MSRV 1.85, with macOS
covered by the CI matrix.

## Performance, in one line

The LSN reservation is a single ~4 ns atomic; a file append is syscall-bound (the
`pwrite` the durability contract requires); and group commit beats a hand-rolled
inline WAL by ~1.9× for concurrent durable commits, widening with more writers and
faster storage. Details in [`docs/BENCHMARKS.md`](../BENCHMARKS.md).

## Upgrading from 0.9.x

Nothing to do. There are no API or on-disk changes from 0.9.1; 1.0.0 only freezes
the surface.

## Installation

```toml
[dependencies]
wal-db = "1.0"

# Typed records:
wal-db = { version = "1.0", features = ["pack-io"] }
```

MSRV: Rust 1.85.

## Documentation

- [README](https://github.com/jamesgober/wal-db/blob/main/README.md)
- [API Reference](https://github.com/jamesgober/wal-db/blob/main/docs/API.md)
- [On-Disk Format](https://github.com/jamesgober/wal-db/blob/main/docs/ON_DISK_FORMAT.md)
- [Benchmarks](https://github.com/jamesgober/wal-db/blob/main/docs/BENCHMARKS.md)
- [CHANGELOG](https://github.com/jamesgober/wal-db/blob/main/CHANGELOG.md)

---

**Full diff:** [`v0.9.1...v1.0.0`](https://github.com/jamesgober/wal-db/compare/v0.9.1...v1.0.0).
**Changelog:** [`CHANGELOG.md`](https://github.com/jamesgober/wal-db/blob/main/CHANGELOG.md#100---2026-06-05).