Expand description
turndb — a content-addressed columnar store for AI traces.
§The shape
ingest ── WAL (durability) ── memtable
│ flush
▼
part (immutable, columnar)
│ merge
▼
larger parts
fold ── stable content locations shared by every partTwo ideas fused. Content addressing: a piece of content is identified by BLAKE3 of its bytes and stored exactly once, however many records reference it. Columnar: the typed metadata around that content lives in per-field columns that compress and scan well.
§Why the fusion is worth it
In a conventional LSM, merging rewrites data — merge two 1 GiB parts, write 2 GiB. That write amplification is the tax every tiering strategy exists to manage.
Here, part merging never touches content. Pieces live in the fold, addressed by hash, written once until refold replaces the generation or declared content punch removes an unreachable payload. A part is a bundle of references plus columns, so merging rewrites references and columns and nothing else — on trace-shaped data that is a small fraction of the bytes. Content addressing is what decouples part-merge cost from data volume, and that is what lets a trace store behave like a database instead of a write-once archive.
§The cardinal invariant
Byte-exact reconstruction. Reading a record reproduces the original bytes exactly — including attribute order and duplicate keys. Every layer below is in service of it, and no change ships that breaks it.
Re-exports§
pub use types::AttrValue;pub use types::Content;pub use types::ContentHash;pub use types::ContentOp;pub use types::PieceHash;pub use types::Record;pub use types::BODY_CONTENT;
Modules§
- backup
- Online backup and restore result types and refusal contracts.
- capabilities
- Machine-readable build and platform capabilities.
- carve
- The carve: how a body becomes spans — an opinion with escape hatches, never a lock-in.
- container
- A store’s artifacts in one mutable file.
- control
- Cooperative interruption shared by scans and lifecycle operations.
- error
- Stable, domain-neutral error classification for embedding boundaries.
- fold
- The fold — an append-only, content-addressed piece store with block compression.
- observability
- Pull-based, telemetry-neutral operation metrics.
- part
- A part: an immutable, self-contained, id-sorted columnar slice of the store.
- query
- The query lens: parts, read a column at a time, as Arrow.
- read_
limits - Runtime admission for atomic storage frames and persistent object collections.
- readat
- Positioned reads behind a trait — the seam every future backend goes through.
- scan
- Storage-native structured paging.
- schema
- Feature-independent discovery of the store’s self-described field universe.
- store
- The store: WAL, pending change set, publication, manifest revisions, and WAL replay — the layer that turns a fold and parts into a database.
- types
- The vocabulary every layer shares.
- vfs
- Every WRITE-SIDE filesystem operation, behind one thin seam.