Trine KV

Trine KV is an embedded Rust key-value database for applications that need ordered local storage without running a separate server. It gives small programs a default bucket, and lets larger applications add named buckets with their own prefix, filter, compression, and large-value settings.

The crate is implemented and verified by the repository test suite, benchmark harness, and durability notes. To see the main path work end to end:

cargo run --example quickstart

Then read docs/usage.md for the API path and docs/platform-io.md when you need the feature/runtime choice for async native-file I/O. docs/durability.md explains persistence guarantees and limits. Release packaging notes live in docs/release.md.

Common Capabilities

• Async-first default-bucket reads and writes with Db::put, Db::get, Db::range, and Db::prefix.
• Explicit sync adapters with *_sync names, such as Db::open_sync, db.put_sync, bucket.get_sync, and db.flush_sync.
• Optional named buckets through db.bucket("users").await? when data needs logical separation or independent tuning.
• Atomic write batches across the default bucket and named buckets.
• MVCC snapshots that keep old reads stable while newer writes commit.
• Public ReadVersion cursors for reopening a retained committed state with Db::snapshot_at.
• Named checkpoints that pin important read versions until the application deletes them.
• Configurable recent read-version retention with DbOptions::with_keep_last_read_versions.
• Snapshot-bound point readers that can avoid copying inline table values when callers can work with borrowed bytes.
• Optimistic transactions with point and range conflict checks.
• Ordered range scans and prefix scans.
• Value-lazy range and prefix scans for large-value workloads that need keys before reading blob bytes.
• Persistent mode with WAL replay, manifest recovery, directory locking, safe native defaults, background maintenance, backpressure, flush, compaction, and read-only open.
• Explicit in-memory mode for tests, examples, and short-lived data that should disappear when the Db is dropped.
• Async open/read/write/scan/transaction/maintenance entry points. Native persistent async APIs enter operation-level storage waits through Trine's storage boundary; platform-io chooses native, partial native, or managed thread-pool completion below that boundary.
• Block-based SSTables with partitioned index/filter blocks, data-block hash lookup for point reads, high-priority metadata caching, compression, and linear/binary/auto index seek policies.
• Large values can be separated into Titan-like blob files with BlobIndex records in SSTables.
• Automatic blob Level Merge can rewrite retained large values into output blob files during compaction when it improves locality or removes stale blob refs.
• Snapshot-safe blob GC rewrites still-live large values out of stale blob files and delays old-file deletion while a read can still reach them.
• Live stats report table, cache, filter, blob read, blob byte, and blob GC counters.
• Explicit WASI and browser persistent options. WASI uses a host-preopened filesystem path on WASI targets and supports Db::open through the host storage boundary. Browser persistence uses the async API and the browser persistent backend on wasm32-unknown-unknown.

Install

Add Trine KV from crates.io:

cargo add trine-kv

cargo install is for crates that provide command-line binaries. Trine KV is a library crate, so application projects should depend on it instead.

For local development, depend on a path:

[dependencies]
trine-kv = { path = "../trine-kv" }

Feature Flags

Most applications can start with no feature flags:

[dependencies]
trine-kv = "0.4"

Enable platform-io when native-file storage should complete through Trine's portable async platform boundary. This baseline uses a bounded Trine-owned thread pool and avoids native async backend dependencies:

[dependencies]
trine-kv = { version = "0.4", features = ["platform-io"] }

Enable platform-io-native when you want native async where Trine has audited operation support, with the same thread-pool backend for the remaining rows:

[dependencies]
trine-kv = { version = "0.4", features = ["platform-io-native"] }

After enabling either feature, select the runtime for a database:

use trine_kv::{Db, DbOptions, RuntimeOptions};

let mut options = DbOptions::new("./trine-data");
options.runtime = RuntimeOptions::platform_io();

let db = Db::open(options).await?;

Verify the path with:

cargo run --example platform_io --features platform-io
cargo run --example platform_io --features platform-io-native

See docs/platform-io.md for the operation matrix and stats fields.

Common API Example

use trine_kv::{Db, KeyRange, TransactionOptions, WriteBatch, WriteOptions};

async fn run() -> trine_kv::Result<()> {
    let db = Db::open("./trine-data").await?;

    // Simple applications can use the built-in default bucket directly.
    db.put(b"settings:theme", b"dark").await?;
    assert_eq!(db.get(b"settings:theme").await?, Some(b"dark".to_vec()));

    // Named buckets are created on demand when you need logical separation.
    let users = db.bucket("users").await?;
    users.put(b"user:001", b"Ada").await?;

    // Snapshots keep a stable read version while newer writes continue.
    let snapshot = db.snapshot();
    users.put(b"user:002", b"Lin").await?;
    assert_eq!(snapshot.get(&users, b"user:002").await?, None);

    // Batches can atomically span buckets.
    let mut batch = WriteBatch::new();
    batch.put(b"audit:001", b"user-created");
    batch.put_bucket("users", b"user:003", b"Grace")?;
    db.write(batch, WriteOptions::default()).await?;

    // Transactions validate their read set when they commit.
    let mut txn = db.transaction(TransactionOptions::default());
    assert_eq!(
        txn.get_bucket("users", b"user:001").await?,
        Some(b"Ada".to_vec())
    );
    txn.put_bucket("users", b"user:004", b"Barbara")?;
    txn.commit().await?;

    let mut rows = users
        .range(&KeyRange::half_open(b"user:001", b"user:999"))
        .await?;
    let mut row_count = 0;
    while rows.next().await?.is_some() {
        row_count += 1;
    }
    assert_eq!(row_count, 4);

    Ok(())
}

For the runnable async-first persistent path, use:

cargo run --example quickstart

For tests or short-lived data, opt into memory mode explicitly:

use trine_kv::{Db, DbOptions};

let db = Db::open(DbOptions::memory()).await?;

For the explicit sync-adapter path, use:

cargo run --example sync_quickstart

Common Commands

cargo fmt --check
cargo clippy
cargo test
cargo run --example quickstart
cargo run --example sync_quickstart
cargo run --example platform_io --features platform-io
cargo run --example read_versions
cargo run --example user_store
cargo run --example event_index
cargo run --example durability
cargo bench --bench v1_bench

Examples

• quickstart: first pass through Db::open, async writes, lazy scans, transaction commit, maintenance, read-only reopen, and storage runtime stats.
• sync_quickstart: first pass through the explicit sync adapters, including persistent open, buckets, scans, transactions, flush, reopen, and stats.
• platform_io: runs a small write/read/flush path and, when a platform-io feature is enabled, checks RuntimeOptions::platform_io() operation counters.
• read_versions: captures public read-version cursors, creates a named checkpoint, reopens from the checkpoint, and shows expiration after deletion.
• user_store: wraps Trine KV behind a small repository-style API.
• event_index: stores event payloads and a secondary account index with one atomic write batch.
• durability: shows the non-strict default versus strict (SyncAllStrict / F_FULLFSYNC) writes, per write and as a database-wide floor.

Documentation

• Usage guide
• Platform I/O and feature selection
• Durability notes
• Release packaging
• 0.1 benchmark baseline
• Large-value direct read tuning
• Blob maintenance and lazy value benchmark
• Read-pruning measurement
• Cold table open read
• Cold manifest/open reopen
• Read-only cold reopen
• Read-only cold open breakdown
• Batched point reads
• Get-many internal batching

Current Boundaries

• Persistent mode uses a single local database directory.
• Native persistent open defaults to SyncAll (non-strict) for confirmed writes: on macOS that is a plain fsync, which is fast and survives a crash or kernel panic but is not guaranteed across sudden power loss. For power-loss durability use SyncAllStrict (macOS F_FULLFSYNC), per write with WriteOptions::sync_all_strict() or database-wide with DbOptions::with_durability. Buffered is an explicit advanced mode for data that can tolerate losing recent writes after a crash. See docs/durability.md.
• WASI and browser persistent backends do not claim the device-sync modes (SyncData, SyncAll, SyncAllStrict).
• WASI persistent Db::open uses the host-preopened filesystem on WASI targets; current WASI file work completes inline and does not advertise platform async I/O.
• Browser persistence is async-only: use Db::open plus async mutation and maintenance methods. Synchronous browser persistent open, mutation, and maintenance *_sync APIs return typed unsupported errors.
• Read-only open is for inspecting a stable directory state; the current pre-1.0 line does not define live multi-process reads against an active writer.
• Repair is intentionally narrow and only removes known safe temporary files when explicitly requested.