Crate holt

Expand description

§holt — adaptive radix tree metadata storage engine

holt is an embedded Rust library that stores path-shaped metadata with sub-microsecond lookups, per-blob concurrency, and crash-safe persistence. It is built around an Adaptive Radix Tree that spans multiple 512 KB blob frames.

See README.md for the elevator pitch and ARCHITECTURE.md for the deep dive.

§Current status

All core layers — layout, walker (insert / lookup / erase / range / spillover / compact / merge), persistent store (O_DIRECT + optional io_uring), WAL with replay, sharded buffer manager, background checkpointer, copy-on-write snapshots — are in place and covered by integration + property tests.

Durability::Wal controls how Holt acknowledges its local write-ahead log. File-backed trees replay that WAL on open; manual and background checkpoints make older log records redundant.

§Quick taste

use holt::{RangeEntry, TreeBuilder};

let tree = TreeBuilder::new("/var/lib/myapp/meta.holt").open()?;
tree.put(b"img/01.jpg", b"rgb_data")?;
let v: Vec<u8> = tree.get(b"img/01.jpg")?.unwrap();
for entry in tree.scan(b"img/").into_iter().take(10) {
    if let RangeEntry::Key { key, value, .. } = entry? {
        println!("{key:?} -> {value:?}");
    }
}

§Module map

The supported import surface is the flat crate root: Tree, DB, TreeBuilder, AtomicBatch, AtomicErrorKind, Record, RecordVersion, View, Snapshot, KeyPathBuf, KeyPrefixBuf, range iterator types, stats snapshots, and the optional metrics renderer.

All implementation modules are crate-private. This keeps the on-disk format, WAL codec, walker, and buffer-manager internals free to change in minor releases without breaking downstream code.

Internal modules (pub(crate), not part of the SemVer surface):

layout — extern struct layouts (BlobHeader, SlotEntry, per-NodeType bodies). Pinned at compile time via const _: () = assert!(...) blocks per file.
journal — WAL codec + replay scanner + writer.
store — buffer manager + blob-frame allocator + store trait machinery. The supported store surface (BlobStore, MemoryBlobStore, FileBlobStore, AlignedBlobBuf) is re-exported at the crate root for users who want to plug in a custom store.
engine — recursive walker (insert / lookup / erase / scan / rename / compact). Record and key-only range iterator types and stats snapshots are re-exported at the crate root.
concurrency — HybridLatch 3-mode lock plus the tree-wide maintenance gate.
checkpoint — 3-thread background checkpointer for dirty blob drain, WAL flush/truncate, and cache eviction.

§Platform support

holt is Unix-only by design: Linux (O_DIRECT fast path, io_uring on the file store) and macOS (F_NOCACHE). Windows is out of scope and the crate refuses to compile there — see the platform stance in ROADMAP.md.

Modules§

metrics: Prometheus text-format renderer for TreeStats. Enabled via the metrics feature flag. Prometheus text-format renderer for TreeStats.

Structs§

AlignedBlobBuf: A heap-allocated, 4 KB-aligned, PAGE_SIZE-byte buffer.
AtomicBatch: Builder for an atomic batch. See super::tree::Tree::atomic.
BlobStats: Per-blob counters captured by Tree::stats.
CheckpointConfig: Background checkpointer policy + cadence.
CheckpointImage: A serialized whole-DB checkpoint. See the module docs.
CheckpointerStats: Snapshot of the background checkpointer’s accumulated counters. Returned inside TreeStats::checkpointer when the thread group is enabled. All counters are cumulative since the threads were spawned.
DB: A storage instance containing multiple named Tree roots.
DBAtomicBatch: Builder for DB::atomic.
DBStats: DB-wide resource counters from DB::stats.
DBView: Immutable read transaction over one or more named tree scopes.
FileBlobStore: NVMe-backed, O_DIRECT, single-packed-file blob store.
JournalAnchor: One position in an application’s hash-chained recovery stream.
JournalEnvelope: Opaque application recovery bytes anchored before and after one DB batch.
JournalEnvelopePage: One bounded page of attached recovery envelopes.
JournalState: Checkpointed and live positions of one attached recovery stream.
JournalStats: Snapshot of WAL append and sync counters.
KeyPathBuf: Owned builder for slash-separated byte keys.
KeyPrefixBuf: Owned slash-terminated scan prefix.
KeyRangeBuilder: Builder produced by crate::Tree::range_keys.
KeyRangeIter: Active key-only iteration state — see KeyRangeBuilder::into_iter.
KeyScanOutcome: Result of a borrowed key-only range visit.
MemoryBlobStore: Concurrent in-memory blob store.
OpenStats: Reopen-time recovery telemetry captured while opening a tree.
PrefixCount: Bounded count of live keys under a prefix.
PrefixRecord: Longest-prefix lookup result.
RangeBuilder: Builder produced by crate::Tree::range.
RangeIter: Active iteration state — see RangeBuilder::into_iter.
Record: Value plus the live record version observed by one lookup.
RecordVersion: Opaque per-record version returned by Tree::get_version.
RouteCacheStats: Prefix-route-cache counters captured by Tree::stats.
ScanStats: Per-scan work accounting for a single range cursor.
Snapshot: A stable copy-on-write snapshot of a tree or prefix subtree.
StoreStats: Store-level space and slot counters.
Tree: An holt tree — your handle to one metadata store.
TreeBuilder: Fluent constructor for Tree.
TreeConfig: Configuration passed to crate::Tree::open.
TreeStats: Tree-wide aggregate counters from Tree::stats.
VacuumStats: Physical space reclaimed by an explicit vacuum pass.
View: Immutable read transaction over one captured prefix.
ViewKeyRangeBuilder: Key-only range builder scoped to a View.
ViewRangeBuilder: Record range builder scoped to a View.

Enums§

AccessMode: Access policy for a file-backed tree.
AtomicErrorKind: Whether a failed crate::DB::atomic call may have applied its batch.
Durability: How Holt’s local write-ahead log is acknowledged.
Error: Top-level error type covering the union of every failure mode.
JournalEnvelopeError: Validation failure while constructing an attached journal envelope.
KeyPathError: Error returned when a path-shaped key segment is not valid.
KeyRangeEntry: An entry yielded by KeyRangeIter.
KeyRangeEntryRef: Borrowed key-only range entry passed to KeyRangeBuilder::visit.
PutOutcome: Per-key result of Tree::put_many_if_absent.
RangeEntry: An entry yielded by RangeIter.
Storage: Where the tree’s data lives.

Constants§

JOURNAL_DIGEST_BYTES: Byte width of a journal anchor digest.
MAX_JOURNAL_RECORD_BYTES: Maximum encoded size of one WAL record accepted by Holt.

Traits§

BlobStore: A blob-granular storage interface.

Type Aliases§

BlobGuid: 128-bit blob identifier (stored as 16 bytes).
Result: Result alias used throughout the crate.

Crate holt

Crate holt Copy item path

§holt — adaptive radix tree metadata storage engine

§Current status

§Quick taste

§Module map

§Platform support

Modules§

Structs§

Enums§

Constants§

Traits§

Type Aliases§

Crate holt