# `db.tx()` — crash-atomic multi-collection transactions
A transaction writes several collections **all-or-none across a crash**. It is
not an isolation mechanism: there is no snapshot, no read set, no MVCC. What it
guarantees is that after any crash the database contains either every write of
the transaction or none of them.
For the history of the `atomic2`/`atomic3`/`atomic4` API this replaced, see
[`cross-collection-transactions.md`](./cross-collection-transactions.md).
```rust
db.tx()
.put(&accounts, &from, &Account { balance: a - amount })?
.put(&accounts, &to, &Account { balance: b + amount })?
.put(&ledger, &entry_id, &Entry { from, to, amount })?
.durable()
.commit()?;
```
## What is guaranteed
| **Atomicity across a crash** | yes — the record is durable in the transaction log before anything is applied, and replay finishes it at the next open |
| **Atomicity of an error before the commit point** | yes — zero writes, zero index mutations, zero hooks |
| **Durability against power loss** | only with `.durable()`, which waits for the log fsync |
| **Isolation from concurrent readers** | **no** — writes become visible per shard as they are applied |
| **Rollback after the commit point** | **no** — this is a redo log; a failed apply is finished by replay, never undone |
| **Write-hook effects** | **not** covered — hooks replay in-process after the locks drop; a crash between apply and hook loses the hook, not the write |
Isolation is worth restating: a reader that touches two of the participating
collections can observe one collection's write before the other's. The
transaction bounds what survives a crash, not what a concurrent reader sees.
## Who can participate
The six managed families — `TypedTree`, `TypedMap`, `ZeroTree`, `ZeroMap`,
`VarTypedTree`, `VarTypedMap` — opened through **the same `Db`**. Membership is
checked by `Arc` identity, so a handle from another `Db` is rejected at
`commit()` with `DbError::Client`.
**FixedStore** collections (`open_zero_tree_fixed` / `open_zero_map_fixed`) are
eligible alongside the Bitcask families above. A transaction can therefore span
both backends — the shape the consumer actually writes.
An **encrypted** Bitcask collection may participate when `TxConfig::log_encryption_key`
is set; without it `commit()` refuses before any lock is taken. FixedStore
collections are never encrypted.
Not eligible today:
- **`LsmTree`** — never participates.
- Raw `VarTree` / `VarMap` — wrap them in a `VarTyped*` to take part.
`txlog` is a reserved collection name.
## Operations
| `put(&coll, &key, &value)` | write, encoded at the call site |
| `delete(&coll, &key)` | delete; a key that is absent is a legal no-op |
| `upsert(&coll, &key, f)` | `f(Option<&T>) -> Upsert<T>`, evaluated under the locks |
| `guard(&coll, &key, f)` | `f(Option<&T>) -> DbResult<()>`; an `Err` fails the whole transaction before anything is written |
Duplicate keys on one collection collapse — the last decision wins, at the
position of the key's first appearance — and `upsert`/`guard` read the
transaction's own pending writes before the stored value.
### Closures run under the shard locks
`upsert` and `guard` closures are evaluated at `commit()` while this thread holds
every shard the transaction touches. **Do not call back into the database from
one.** `atomic()` and a nested `commit()` return `DbError::ReentrantTx` instead
of deadlocking, but a facade call (`put` / `get` / `delete` straight on a
collection handle) is not instrumented and will park forever on a lock this
thread already owns. Read what you need *before* building the transaction.
The same guard works in the other direction: `commit()` inside an `atomic()`
block is `ReentrantTx`.
## Limits
Checked at the builder or before any lock, never after:
| Operations per transaction (before duplicate collapse) | `max_ops` | 1024 |
| Encoded record size | `max_bytes` | 8 MiB |
| Key length | — | 64 KiB, fixed |
`TxConfig::durable_by_default` sets the starting value of the `durable` flag;
it is **off**, so a plain `commit()` is atomic against a process crash but not
against power loss. `checkpoint_bytes` decides how large the log grows before a
commit triggers a cut.
## How it works
1. **Plan** — group operations by participant, order participants by
`collection_id` (position in `Db`'s publish list) and shards ascending. This
canonical order is what makes two overlapping transactions deadlock-free.
2. **Lock** — take every shard in that order.
3. **Evaluate** — run `upsert`/`guard` under the locks, with read-your-writes
over the pending set.
4. **Reserve** — per backend. Bitcask reserves a contiguous GSN block per shard,
so a shard's entries keep ascending GSNs; replay depends on that. FixedStore
resolves each key to `(slot, target_meta)` and holds the slot in memory until
the record is logged. An abort returns a freshly allocated slot to the allocator
and writes nothing — with the one documented exception that a file which grew
to make room stays grown.
5. **Log** — encode one record and append it. Any failure up to here aborts with
nothing written. With `.durable()`, fsync before continuing.
6. **Apply** — write through the same per-shard mutation seam `atomic()` uses.
A failure here is *not* rolled back: the record is durable, so replay
finishes it at the next open. The failing shard is **fenced** — it applies no
further operation in this transaction and refuses every later write with
`DbError::ShardFenced` until the database is reopened. A Bitcask shard's
replay rule compares the reserved GSN with the shard's recovered watermark, so
any entry written after the gap would bury it and the next open would drop the
committed operation. FixedStore is not fenced: its rule reads the reserved
slot's exact state, which no unrelated write can disturb. Reads and other
shards are unaffected.
7. **Release, then hooks** — every lock drops, then buffered hook events replay
in operation order.
8. **Settle and maybe cut** — mark the record applied (or, on a partial apply,
leave it pending but unpin the floor) and checkpoint if the log is over
`checkpoint_bytes`.
### Replay
When a collection is opened, pending records naming it are replayed **before its
migrations run** — the bytes in the log are in the collection's stored,
pre-migration encoding. An operation is replayed when its reserved GSN is above
the shard's recovered watermark; at or below means it is already durably present
(or was superseded, which is equally a reason to leave it alone).
An operation that **does** need replay and whose logged `schema_fingerprint`
disagrees with the collection's stored `typ_hash` **fails the open** with
`DbError::TxSchemaMismatch` rather than injecting a stale encoding.
`TxConfig::discard_pending_on_schema_mismatch` turns that into a drop-and-log; it
is off, because losing a committed transaction silently is worse than refusing to
start.
The fingerprint is checked per operation and only after the replay decision,
because it guards *decoding*. A record can outlive the migration that replayed
it — a participant that never opens keeps the record pending while the collection
migrates — and refusing every later open over bytes nobody will decode would turn
a schema change into an unopenable database.
A record is retired only after every collection it names has been opened in this
process and the checkpoint has durably flushed them. A record naming a collection
this process never opens stays in the log.
**FixedStore** operations replay against the slot's recovered state, not a
per-shard GSN watermark. The logged `(slot_id, target_meta)` is compared under
the shard lock:
| `slot_id == NO_SLOT` | skip (logged no-op) |
| `current_meta == target_meta`, put, CRC valid, **another key** in the slot | `DbError::TxFixedSlot` — fail closed (or skip, if *superseded*) |
| `current_meta == target_meta` and state matches the kind | skip (already durable) |
| `is_newer(current_version, target_version)` | skip (slot moved on) |
| `is_newer(target_version, current_version)` | replay (our write never landed) |
| `current_version == target_version`, state differs | replay (torn write, or status differs) |
| otherwise (unorderable) | `DbError::TxFixedSlot` — fail closed (or skip, if *superseded*) |
**Superseded** is the one condition that overrides the rest: if the collection's
index maps the operation's key to a *different* slot, the key has a newer
incarnation and this reservation is an orphan — every "replay" verdict above
becomes a skip, and both fail-closed verdicts become a skip too. Replaying an
orphan would overwrite whatever now lives in the reserved slot. (An orphan arises
when an apply failed after the log point: the record stays durable, the slot stays
allocated, and a later commit for the same key lands elsewhere.)
"State matches the kind" means: for a put, the slot's CRC validates and its stored
key equals the operation's key; for a delete, the status is `STATUS_DELETED`. A
torn write at the target version recovers as FREE while preserving the version,
so equality on version alone would wrongly skip — the state check catches it.
A slot holding a **different** key's intact row at the exact target version means
the reservation was recycled while the record stayed durable. Neither answer is
safe to guess — replaying overwrites a live row that is not ours, skipping drops a
committed write — so unless the key is superseded the open fails with
`DbError::TxFixedSlot`. Reservations are log-aware (they are never handed back to
the allocator once the record is durable), so this state should not arise; the
check is the backstop that keeps a bug from becoming cross-key corruption.
Versions more than 2²⁹ apart cannot be ordered in the 30-bit ring, so — again
unless the key is superseded — the open fails with `DbError::TxFixedSlot` rather
than guessing.
## Encryption
When any participant reports `encrypted()`, `TxConfig::log_encryption_key` must
be set (behind the `encryption` feature). The key seals every log record with
AES-256-GCM: HKDF over a `txlog`-local salt stored in `txlog/db.salt`, the record
header as AAD, and the `tx_id` as the nonce. Without a log key `commit()` refuses,
so a collection's values are never written to the log in the clear. A pending
encrypted record with a missing or wrong key fails the open. FixedStore
collections are never encrypted.
### Operating an encrypted log
- **Back up `txlog/db.salt` with the segments.** The key alone cannot open a
sealed record; losing the salt is losing the pending log.
- **Rotating the key requires a drained log.** A record sealed under the old key
fails the open once the new key is configured, and nothing migrates it. Drain
first (every record applied and checkpointed away), then change the key.
- **A plaintext record stays readable** after the key is configured, so turning
encryption on does not orphan a log written before it.
- **The plaintext is resident in memory** for the log's lifetime: `TxLog` keeps
every pending record so it can replay it. The envelope protects the log at rest.
- **What it defends against.** Confidentiality of logged keys and values, plus
per-record integrity: a modified record fails its GCM tag and fails the open,
even if its CRC was repaired. It is **not** an anti-truncation or anti-rollback
mechanism — an attacker with write access to `txlog/` can still corrupt the
newest segment's tail (which recovery discards as torn) or unlink a segment
file, and no authenticated chain links the segments.
## Errors
| `ReentrantTx` | `commit()` inside `atomic()`, or a collection call inside an `upsert`/`guard` closure |
| `Config` | an encrypted participant with no log key |
| `Client` | limit exceeded, or a handle from another `Db` |
| `TxSchemaMismatch` | a pending record's encoding no longer matches the collection (raised at open) |
| `TxFixedSlot` | a pending FixedStore operation cannot be ordered against the slot's current version (raised at open) |