# wal-db v0.3.0 — Core
**The concurrent WAL.** v0.3.0 turns the single-writer foundation into the real
thing: lock-free multi-writer append, group commit, and a record format frozen
for the 1.x line. Many threads share one log behind an `Arc` and append with no
global lock; when they commit, their fsyncs coalesce into one. The durability and
torn-write recovery from 0.2 sit underneath, now with fail-stop integrity on a
failed write. The concurrency core is verified with `loom`. Segment rotation
follows in 0.3.1.
This release has breaking changes. They are deliberate, they are what make the
append path lock-free, and at pre-1.0 they are the right time to make them.
## What is wal-db?
A write-ahead log primitive for Rust storage engines — the durability substrate
under `lsm-db`, `txn-db`, `raft-io`, and Hive DB. Every state change is appended
to a durable log before it is acknowledged, and that log rebuilds state after a
crash. `wal-db` publishes that primitive as one audited, benchmarked crate so the
engines that need it stop re-deriving the durability contract and getting it
subtly wrong. The core is synchronous; async is left to the consumer.
## What's new in 0.3.0
### Lock-free multi-writer append
The append path no longer takes a lock. Each `append` reserves its byte range
with a single atomic `fetch_add`, frames the record into a reused thread-local
buffer (so steady-state appends do not allocate), and writes it — concurrently
with every other appender.
```rust
use std::sync::Arc;
use std::thread;
use wal_db::Wal;
# fn main() -> Result<(), wal_db::WalError> {
# let dir = tempfile::tempdir().map_err(wal_db::WalError::from)?;
# let path = dir.path().join("app.wal");
let wal = Arc::new(Wal::open(&path)?);
let workers: Vec<_> = (0..8)
.map(|t| {
let wal = Arc::clone(&wal);
thread::spawn(move || {
for i in 0..1000 {
wal.append(format!("worker {t} record {i}").as_bytes()).unwrap();
}
})
})
.collect();
for w in workers {
w.join().unwrap();
}
wal.sync()?;
assert_eq!(wal.iter()?.count(), 8000);
# Ok(())
# }
```
Because the reservation is a single atomic, two records can never share a range
or land out of order — the `loom` model check below proves it across every
interleaving.
### LSNs are byte offsets
The reservation returns the record's **byte offset**, and that offset is its
[`Lsn`]. LSNs are still monotonic and unique, but no longer consecutive: the
first record is `0`, the next sits at its end. This is the change that makes a
lock-free, reorder-free, variable-size append possible with one atomic, and it is
the same design PostgreSQL uses. It also sets up O(1) seeking by LSN in a later
milestone.
This is a breaking change for code that assumed dense `0, 1, 2, …` LSNs.
### Group commit
When several threads call `sync` at once, only one issues the fsync; the others
wait for it and return. A short mutex+condvar coordinator tracks the
contiguous-written watermark (a record is durable only once every byte before it
is written, since recovery stops at the first gap) and elects the fsync leader.
The framing, the write syscall, and the fsync all happen outside the lock.
`append_and_sync` does an append and a group-commit-aware sync in one call:
```rust
use wal_db::Wal;
# fn main() -> Result<(), wal_db::WalError> {
# let dir = tempfile::tempdir().map_err(wal_db::WalError::from)?;
# let path = dir.path().join("app.wal");
let wal = Wal::open(&path)?;
let lsn = wal.append_and_sync(b"committed immediately")?;
# let _ = lsn;
# Ok(())
# }
```
### Fail-stop data integrity
If a record's write fails, its reserved range becomes a permanent gap. The log is
poisoned from that offset on: recovery stops at the gap, and any `sync` whose
range covers it returns `WalError::Corruption` rather than reporting durability it
cannot provide. A WAL that lies about durability is worse than one that stops, so
it stops.
### Record format frozen for 1.x
The on-disk record is now an 8-byte header (CRC32C + length) followed by the
payload — the redundant stored LSN is gone, since a record's LSN is its offset.
The full byte-level specification, with the exact CRC32C parameters and the
recovery algorithm, is in [`docs/ON_DISK_FORMAT.md`](../ON_DISK_FORMAT.md), and
**the record format is frozen for the entire 1.x line.** The multi-file segment
layout is added in 0.3.1.
### Model-checked concurrency
`tests/loom_wal.rs` runs under `loom`, which explores every meaningful thread
interleaving rather than relying on luck:
- **Append** — two concurrent writers always get distinct LSNs, occupy disjoint
byte ranges, and both records recover intact. No overlap, no reorder, no loss.
- **Group commit** — two concurrent `append_and_sync` calls issue at most one
fsync each (coalescing makes it fewer), and both records end up durable.
## Breaking changes
- **`Lsn` is a byte offset**, not a dense counter. Monotonic and unique, not
consecutive.
- **`WalStore` trait**: methods take `&self`, `append(&mut self, bytes)` became
`write_at(&self, offset, bytes)`, and the trait requires `Send + Sync`.
- **`Wal::len` / `Wal::is_empty`** return `u64` / `bool` directly instead of
`Result`.
- **On-disk record format**: 8-byte header, no stored LSN. Logs written by 0.2
are not readable by 0.3. This format is now frozen for 1.x.
## Verification
Run on Windows x86_64 and Linux (WSL2 Ubuntu), Rust stable 1.95.x and MSRV
1.85.0; macOS is covered by the CI matrix:
```bash
cargo fmt --all -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-features
RUSTFLAGS="--cfg loom" cargo test --test loom_wal
RUSTDOCFLAGS="-D warnings" cargo doc --no-deps --all-features
cargo +1.85 build --all-features
cargo audit
cargo deny check
cargo bench --bench wal_bench
```
All green on both platforms. Counts at this tag:
- 40 unit tests
- 6 integration tests (4 round-trip, 1 torn-write property test, 1 cross-process
durability)
- 2 loom model-check tests
- 19 doctests
Baseline criterion numbers (256-byte records, development machine):
- `append/single` (lock-free hot path, no I/O): ~107 ns — faster than 0.2's
~130 ns now that the mutex is gone.
- `append/multi` (8 writers): ~3.6 M appends/s aggregate.
- `commit/single` (append + fsync each time): ~0.9 ms.
- `commit/group` (8 writers, append-and-sync): ~4× the single-writer commit rate,
the group-commit fsync amortisation. The figure is bounded by this machine's
fsync latency and grows with faster storage and more concurrent writers; an
honest head-to-head against other engines is the subject of the 0.6 comparison
milestone.
## What's next
- **0.3.1 — Segment rotation.** Bounded segment files so recovery time and
archival stay bounded, with the segment-file layout added to the on-disk format
spec. The byte-offset LSN design is already built for it.
## Installation
```toml
[dependencies]
wal-db = "0.3"
```
MSRV: Rust 1.85.
## Documentation
- [README](https://github.com/jamesgober/wal-db/blob/main/README.md)
- [API Reference](https://github.com/jamesgober/wal-db/blob/main/docs/API.md)
- [On-Disk Format](https://github.com/jamesgober/wal-db/blob/main/docs/ON_DISK_FORMAT.md)
- [CHANGELOG](https://github.com/jamesgober/wal-db/blob/main/CHANGELOG.md)
---
**Full diff:** [`v0.2.0...v0.3.0`](https://github.com/jamesgober/wal-db/compare/v0.2.0...v0.3.0).
**Changelog:** [`CHANGELOG.md`](https://github.com/jamesgober/wal-db/blob/main/CHANGELOG.md#030---2026-06-05).