seerdb 0.0.7

Research-grade storage engine with learned data structures
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100

# seerdb

Research-grade LSM storage engine with learned data structures.

[![Crates.io](https://img.shields.io/crates/v/seerdb.svg)](https://crates.io/crates/seerdb)
[![Docs.rs](https://docs.rs/seerdb/badge.svg)](https://docs.rs/seerdb)
[![CI](https://github.com/omendb/seerdb/actions/workflows/ci.yml/badge.svg)](https://github.com/omendb/seerdb/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)

Embedded key-value storage integrating learned indexes (ALEX), key-value separation (WiscKey), and workload-aware compaction from recent systems research.

## Installation

```toml
[dependencies]
seerdb = "0.0.4"
```

Requires nightly Rust:

```bash
rustup override set nightly
```

## Quick Start

```rust
use seerdb::{DB, DBOptions};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let db = DB::open(DBOptions::default())?;

    // Basic operations
    db.put(b"key", b"value")?;
    let val = db.get(b"key")?;
    db.delete(b"key")?;

    // Batch writes (atomic)
    let mut batch = db.batch();
    batch.put(b"user:1", b"alice");
    batch.put(b"user:2", b"bob");
    batch.commit()?;

    // Range scan
    for result in db.prefix(b"user:")? {
        let (key, value) = result?;
        println!("{:?} = {:?}", key, value);
    }

    // Point-in-time snapshot
    let snapshot = db.snapshot();
    db.put(b"key", b"new_value")?;
    assert_eq!(snapshot.get(b"key")?, val); // snapshot sees old state

    Ok(())
}
```

## Features

- **Learned indexes** (ALEX) for adaptive key distribution
- **Key-value separation** (WiscKey) for reduced write amplification
- **Merge operators** for atomic read-modify-write (counters, lists)
- **OCC transactions** with snapshot isolation
- **Point-in-time snapshots**
- **Range queries** and prefix scans
- **Tiered storage** with S3/GCS/Azure support
- **Compression** (ZSTD/LZ4) and SIMD optimizations

## Architecture

seerdb is a 7-level LSM tree with:

- **Memtable**: Partitioned skip list (16 partitions) with lock-free reads via `ArcSwap`
- **WAL**: Write-ahead log with configurable sync policies
- **SSTable**: Two-level index with ALEX learned index for data blocks, bloom filters, LZ4/ZSTD compression
- **VLog**: Optional WiscKey-style value log for large values (reduces write amplification)
- **Compaction**: Dostoevsky-inspired tiered/leveled hybrid with lazy leveling

## Benchmarks

Run benchmarks with:

```bash
cargo bench                              # All benchmarks
cargo bench --bench ycsb_benchmark       # YCSB workloads
cargo bench --bench mixed_workload       # Read/write mix
```

## Testing

```bash
cargo test --lib                         # Unit tests (213 tests)
cargo test                               # All tests including integration
cargo test --features failpoints         # Crash/recovery tests
```

## License

[Apache License 2.0](LICENSE)