Modular, Asynchronous Implementation of a Log-Structured Merge Tree

Note: This is an experimental implementation and not intended for production environments. Please use the leveldb or rocksdb crate for this purpose.

This implementation does not aim to reimplement LevelDB. The major differences are:

• Separation of keys and values: like WiscKey, values are store separately to increase compaction speed
• Concurrent compaction: Multiple threads can compact at the same time for higher write throughput (not fully implemented yet)
• Async-support: All API calls are exposed as async functions. Note, that internally tokio still uses blocking IO in a separate thread pool. Eventually, there will be support for io_uring.

Supported Architectures:

Currently, the code is only tested on Linux machines, but it should run on all systems supported by the rust compiler.

Planned Features:

• Bloom filters for faster lookups
• FLSM: Like PebblesDB LSM-rs will fragment the keyspace to reduce write amplification and increase compaction speed
• More modularity and configuration options

Feature Flags

• snappy-compression: Use the snappy to compress data on disk (enabled by default)
• sync: Expose the synchronous API instead of async one (Note: in this case the implementation will launch a tokio instance internally and hide it from the caller)

Tests

This library ships with several tests. Note, that you cannot run them concurrently as they will access the same on-disk location. To run tests (on Linux) execute the following command.

env RUST_TEST_THREADS=1 RUST_BACKTRACE=1 RUST_LOG=debug cargo test 

Similar Crates

• rust-rocksdb: Rust bindings for RocksDB
• leveldb: Rust bindings for LevelDB
• wickdb: Rust re-implementation of vanilla LevelDB