kvbm-engine
Distributed coordination primitives for KV cache block management (KVBM).
This crate implements the leader/worker architecture for managing KV cache blocks across a tiered storage hierarchy:
G1 (GPU HBM) → G2 (Pinned DRAM) → G3 (NVMe/SSD) → G4 (S3/MinIO)
Leaders own block metadata and make placement decisions. Workers execute data transfers (RDMA, NVMe, object storage). Sessions coordinate multi-instance block transfers.
Feature Flags
| Flag | Purpose |
|---|---|
s3 (default) |
S3/MinIO object storage (G4 tier) |
testing |
Test utilities and mock infrastructure |
nats |
NATS-based pub/sub transport |
collectives |
NIXL + NCCL multi-GPU collectives |
nccl |
NCCL via cudarc |
nvtx |
NVIDIA Tools Extension profiling markers |
Documentation
Detailed module documentation lives in [docs/](docs/):
- Architecture — Overall system design
- Leader — Block coordination and metadata management
- Session — Distributed onboarding protocol
- Worker — Transfer execution
- Worker Group — SPMD parallel workers
- Offload — Async tier-demotion pipeline
- Offload Developer Guide — Contributing to the offload module
- Object Storage — S3/MinIO integration
- Runtime — Runtime bundle (tokio, Velo, NIXL)
- Testing — Test utilities and fixtures