Crate camelliakv

Expand description

CamelliaKV Lump Storage.

CamelliaKV is a local key-value storage designed to provide predictable latency.

§Features

Storage (local KVS) used to store the lump group with 128-bit ID
- Adopt a fixed length ID to reduce memory usage
(Mostly) the operation of HTTP GET/POT/DELETE is performed on lump
Each operation can specify the time prioritydeadline
Request to allocate a device management thread for a physical device (e.g., HDD)
- All I/O instructions for a physical device are processed in series on this management thread
- Serialization is compatibility with HDDs (other words, the performance may cannot be used for SSD)
Without cache layer, it can accurately estimate number of I/O on the disk during each operation (roughly):
- GET/DELETE: Once
- PUT: Twice(maximum)
- ※ In fact, I/O may occur background processing (eg. GC), so that the above values are the number of deductions.
Define and use the “storage format (v1.0)” named “lusf.
Physical devices (e.g., HDD) with a maximum capacity of 512TB
No redundancy or data integrity(consistency) guarantee

device => storage => nvm

deviceModules:
- Provide Device structure
- CamelliaKV users directly touch this structure
- Start the management thread to control Storage and take charge of scheduling request group for it
storageModules:
- Provide Storage structure
- Use nvm as the persistence layer and implemented with [storage format (v1.0)] format
nvmModules:
- Provide NonVolatileMemory structure and FileNvm implementation
- The purpose of providing a persistence layer for storage
- Although it is not implemented at present, it is possible to completely bypass the OS layer by preparing a NonVolatileMemory implementation that directly operation the block device

block: Blocks of storage and NVM related (minimum unit of reading and writing).
deadline: Deadline of various operations for devices.
lump: Lump data structures.
metrics: Metrics for Prometheus.
nvm: Non-volatile memory interface definition and implementation .
storage: Lump storage.