Crate tiny_lsm[−][src]
Expand description
tiny-lsm is a dead-simple in-memory LSM for managing
fixed-size metadata in more complex systems.
Uses crc32fast to checksum all key-value pairs in the log and sstables. Uses zstd to compress all sstables. Performs sstable compression in the background in a low-priority thread.
Because the data is in-memory, there is no need to put bloom filters on the sstables, and get / range cannot fail due to IO issues. This is a bad choice for large data sets if you require quick recovery time because it needs to read all of the sstables and the write ahead log when starting up.
The benefit to using tiered sstables at all, despite being in-memory, is that they act as an effective log-deduplication mechanism, keeping space amplification very low.
Maximum throughput is not the goal of this project. Low space amplification and very simple code is the goal, because this is intended to maintain metadata in more complex systems.
There is currently no compaction throttling. You can play with the constants around compaction to change compaction characteristics.
Never change the constant size of keys or values for an existing database.
Basically untested, but on the flip side there are less than 500 lines of bugs. Let it be a case study for how many bugs can exist in a codebase with 0 usage of unsafe.
Examples
// open up the LSM
let mut lsm = tiny_lsm::Lsm::recover("path/to/base/dir").expect("recover lsm");
// store some things
let key: [u8; 8] = 8_u64.to_le_bytes();
let value: [u8; 1] = 255_u8.to_le_bytes();
lsm.insert(key, value);
assert_eq!(lsm.get(&key), Some(&value));