EdgeStore

Local-first embedded KV + vector database in Rust.

EdgeStore is an SSD-aware, append-only embedded database for edge deployments. It pairs local NVMe fast-path writes with S3-safe recovery, and uses deathtime-cohort compaction (VLDB 2026) to drive device write amplification toward 1.0 — no existing embedded database does this.

Library-first. No mandatory server. No mandatory async runtime.

Quick Start

use edgestore::{EdgestoreConfig, Engine};

let config = EdgestoreConfig::new("/tmp/mydb");
let mut db = Engine::open(config)?;

db.put(b"default", b"hello", b"world")?;
let value = db.get(b"default", b"hello")?;
assert_eq!(value, Some(b"world".to_vec()));

db.flush()?; // WAL fsync + optional memtable flush

See edgestore/examples/ for runnable examples (KV, vector search, replication).

For a rich documentation site with feature guides and paper references, open website/index.html in your browser.

Choose Your Crate

EdgeStore is a Cargo workspace. Most users need only the first crate.

edgestore-repl is optional. The core edgestore crate has zero network dependencies and zero async runtime dependencies. It is a library you embed and call directly — no daemon, no port binding, no server process.

Crate details

Feature Matrix

Architecture

┌─────────────────────────────────────────────────────────────┐
│  Application                                                │
│    ┌──────────────┐  ┌──────────────┐  ┌──────────────┐   │
│    │   KV API     │  │ Vector API   │  │  Text API    │   │
│    └──────┬───────┘  └──────┬───────┘  └──────┬───────┘   │
└───────────┼─────────────────┼─────────────────┼───────────┘
            │                 │                 │
            └─────────────────┼─────────────────┘
                              ▼
┌─────────────────────────────────────────────────────────────┐
│  Engine (single writer + group commit)                      │
│    • Transactions, namespace isolation, LWW conflict res.   │
└─────────────────────┬───────────────────────────────────────┘
                      │ writes batches
                      ▼
┌─────────────────────────────────────────────────────────────┐
│  WAL (LZ4, CRC32C)                    Memtable (BTreeMap)   │
│  • Append-only, rotated at 64 MB / 60 s  • In-memory buf   │
│  • Crash recovery source                 • Flushed → segment│
└─────────────────────────────────────────────────────────────┘
                      │ flushes
                      ▼
┌─────────────────────────────────────────────────────────────┐
│  Segment Store                                              │
│    • Immutable SSTables (ZSTD L1, 4 KiB blocks, 16 MB)      │
│    • Sparse index + xor filter + BLAKE3 content addressing  │
│    • Manifest: live segment tracking, Merkle roots           │
└─────────────────────┬───────────────────────────────────────┘
                      │
                      ▼
┌─────────────────────────────────────────────────────────────┐
│  Local Storage (SSD / NVMe)                                 │
│    • Deathtime-cohort compaction → WAF → 1.0                │
│    • FDP placement hints on supported hardware              │
└─────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────┐
│  Optional: edgestore-repl (network layer)                   │
│    • HTTP replication client + server                        │
│    • S3 / filesystem RemoteStore backends                    │
│    • Merkle delta sync, anti-entropy loop                    │
└─────────────────────────────────────────────────────────────┘

For a deep dive, see ARCHITECTURE.md.

Using S3 (Optional)

Only needed if you are replicating segments to S3 or using S3 as a cold archive. Add the s3 feature to edgestore-repl:

[dependencies]
edgestore-repl = { version = "1.0", features = ["s3"] }

use edgestore::RemoteStore;
use edgestore_repl::S3RemoteStore;

let store = S3RemoteStore::new(
    "my-bucket",           // S3 bucket name
    Some("mydb/"),         // optional key prefix
    None,                  // None for AWS; Some("http://localhost:4566") for LocalStack
).expect("S3RemoteStore::new");

store.upload(&hash, &data)?;
let bytes = store.download(&hash)?;

S3 path layout

s3://{bucket}/{prefix}segments/{blake3_hash_hex}.dat

{blake3_hash_hex} is the 64-character lowercase hex encoding of the 32-byte BLAKE3 hash.

What edgestore-repl does and does not do

It does: provide upload/download/list/delete primitives so you can move segments to/from S3.

It does not: implement tiering policy, cache eviction, or transparent get() fallback to S3. For that, use edgestore-tier below.

Environment variables

LocalStack testing

make s3-test

This starts a LocalStack container, runs all S3 integration tests, and tears it down.

Tiered Storage (Optional)

Use edgestore-tier when your dataset exceeds local disk and you need transparent read-through to S3:

[dependencies]
edgestore-tier = "1.1"

use edgestore::{Engine, EdgestoreConfig};
use edgestore_repl::S3RemoteStore;
use edgestore_tier::TieredEngine;

let local = Engine::open(EdgestoreConfig::new("/tmp/db")).unwrap();
let remote = S3RemoteStore::new("bucket", Some("prefix/"), None).unwrap();
let mut tiered = TieredEngine::new(local, Box::new(remote));

// Writes go to local only
tiered.put(b"ns", b"key", b"val").unwrap();

// Reads try local first; on miss, check archived segments in S3
tiered.get(b"ns", b"key").unwrap();

// Archive local segments to S3, then delete locally to reclaim space
let metas = tiered.local().list_segment_metas();
tiered.archive_segments(&metas).unwrap();
// (caller deletes local .dat files if desired)

edgestore-tier is a thin orchestration layer. It does not decide when to archive or evict — those are application-level policies. It simply provides the read-through mechanism using the RemoteStore primitives from edgestore-repl and the import_segment path from edgestore.

CLI Installation

The edgestore-cli administrative tool can be installed from source:

# Clone the repository
git clone https://github.com/gleicon/edgestore.git
cd edgestore

# Install locally from source
cargo install --path edgestore-cli

# Or build the optimized release binary
cargo build --release -p edgestore-cli
# Binary will be at: target/release/edgestore-cli

The CLI provides commands for:

• Database management: create, stats, compact
• KV operations: put, get, delete, range
• Data exchange: export, import (JSON and binary formats)
• Vector search: vector-put, vector-get, vector-search
• Text search: text-search

Run edgestore-cli --help for full command reference.

Documentation

• API reference: docs.rs/edgestore
• Architecture & file formats: ARCHITECTURE.md
• Changelog: CHANGELOG.md
• Design spec: prod.md

License

Licensed under either of

• Apache License, Version 2.0 (LICENSE-APACHE)
• MIT license (LICENSE-MIT)

at your option.

Contributing

Issues and pull requests are welcome. Please open an issue or PR on GitHub.

EdgeStore is not affiliated with the VLDB organization. The deathtime-cohort technique is described in Lee et al., VLDB 2026.