turbokv 0.2.0

A fast, embedded key-value store in Rust

TurboKV is a high-performance, embedded key-value database written in Rust. It provides a clean API with configurable durability guarantees.

Features

• Simple API: Familiar get, insert, remove, range operations
• Configurable Durability: Choose between fast, durable, or paranoid modes
• LSM-Tree Architecture: Optimized for write-heavy workloads
• Async/Await: Built on Tokio for modern async Rust
• Batch Operations: Atomic write batches for transactional writes
• Range Scans: Efficient prefix and range queries
• Block Cache: Configurable caching for read performance
• Bloom Filters: Fast negative lookups
• Compression: LZ4, Snappy, and Zstd support

Quick Start

Add TurboKV to your Cargo.toml:

[dependencies]
turbokv = "0.2"
tokio = { version = "1", features = ["full"] }

Basic Usage

use turbokv::{Db, DbOptions};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Open database with default options (durable mode)
    let db = Db::open("./my_data").await?;

    // Insert key-value pairs
    db.insert(b"hello", b"world").await?;
    db.insert(b"user:1", b"alice").await?;

    // Get values
    if let Some(value) = db.get(b"hello").await? {
        println!("Got: {}", String::from_utf8_lossy(&value));
    }

    // Delete keys
    db.remove(b"hello").await?;

    // Range scan
    for (key, value) in db.range(b"user:", b"user:~").await? {
        println!("{}: {}",
            String::from_utf8_lossy(&key),
            String::from_utf8_lossy(&value)
        );
    }

    // Prefix scan
    let users = db.scan_prefix(b"user:").await?;

    Ok(())
}

Batch Writes

use turbokv::{Db, WriteBatch};

let db = Db::open("./my_data").await?;

// Atomic batch write
let mut batch = WriteBatch::new();
batch.put(b"key1", b"value1");
batch.put(b"key2", b"value2");
batch.delete(b"old_key");
db.write_batch(&batch).await?;

Configuration Options

TurboKV provides three durability modes to balance speed and safety:

Recommended for most users: fast() or durable() mode.

• Use fast() when data can be regenerated or occasional loss is acceptable
• Use durable() for production data that must survive process crashes

The paranoid() mode is for specialized use cases where you need power-loss durability. This mode is significantly slower due to fsync overhead (~257 ops/sec vs ~1.1M ops/sec).

use turbokv::{Db, DbOptions};

// Fast mode - maximum speed, no durability guarantees
// Best for: caches, temporary data, benchmarks
let db = Db::open_with_options("./data", DbOptions::fast()).await?;

// Durable mode (RECOMMENDED) - WAL protects against process crashes
// Best for: most production workloads
let db = Db::open_with_options("./data", DbOptions::durable()).await?;

// Paranoid mode - fsync on every write
// Best for: financial transactions, critical records
let db = Db::open_with_options("./data", DbOptions::paranoid()).await?;

Custom Configuration

use turbokv::{DbOptions, Compression};

let options = DbOptions {
    wal_enabled: true,           // Write-ahead log for durability
    sync_writes: false,          // Periodic sync (true = fsync every write)
    memtable_size: 64 * 1024 * 1024,   // 64MB memtable
    block_cache_size: 64 * 1024 * 1024, // 64MB block cache
    compression: Compression::Lz4,      // Lz4, Snappy, Zstd, or None
};

let db = Db::open_with_options("./data", options).await?;

Compression options:

• Compression::None - No compression (fastest writes, largest files)
• Compression::Lz4 - Very fast compression (default, best for most workloads)
• Compression::Snappy - Fast compression (good balance)
• Compression::Zstd - High compression ratio (smaller files, slower)

Components

• WAL (Write-Ahead Log): Ensures durability by logging writes before applying
• MemTable: In-memory skip list for fast writes
• SSTable: Sorted, immutable files on disk with bloom filters
• Block Cache: LRU cache for frequently accessed data blocks
• Compaction: Background process to merge SSTables and reclaim space

Performance

TurboKV is optimized for high write throughput and outperforms both RocksDB and fjall.

Production-scale benchmark: 10M keys, 400-byte values (4.2GB total)

Benchmarks on Apple Silicon Mac, SSD storage, RocksDB-comparable parameters (20-byte keys, 400-byte values)

Understanding the Numbers

How does TurboKV compare?

• TurboKV is 2x faster than RocksDB with equivalent durability (WAL enabled)
• TurboKV is 2.25x faster than fjall
• TurboKV provides a simpler async API with zero-allocation write paths

Why is paranoid mode so slow? Every write calls fsync() which takes 3-5ms on SSDs. This is a hardware limitation that affects all databases equally. RocksDB and fjall hit the same bottleneck (~200-300 ops/sec) when configured for power-loss durability.

Why is durable mode much faster? It writes to the WAL but relies on the OS to fsync() periodically (every few seconds). This "periodic sync" approach means data survives process crashes but not sudden power loss. This is what RocksDB does by default (sync_wal: false).

Comparison

API Reference

Db

DbOptions

WriteBatch

Development

# Build
cargo build --release

# Run tests
cargo test

# Run benchmarks
cargo bench

# Format code
cargo fmt

# Lint
cargo clippy