Crate rhei_cdc_rocksdb 

RocksDB-backed CDC event log for the Rhei HTAP engine.

This crate provides RocksDbCdcLog, a durable, high-throughput alternative to the trigger-based SQLite CDC log used in standard HTAP mode. Benchmarks show 5–7× higher write throughput compared to SQLite triggers, because RocksDB’s LSM-tree write path has lower per-write overhead than SQLite’s WAL trigger execution.

When to use this crate

Scenario	Recommended CDC backend
Development / low-volume workloads	SQLite trigger-based CDC (default)
High-throughput OLTP with many small writes	RocksDbCdcLog (this crate)
Sidecar mode (external source, no local OLTP)	N/A — sidecar polls the external DB directly

This crate is compiled only when the parent workspace enables the rocksdb-cdc feature. The RocksDB C++ library is bundled via rust-rocksdb so no external installation is required.

Storage layout and key encoding

All data lives in RocksDB’s default column family (no custom CFs). Two key namespaces co-exist:

Key range	Content
[0x00, 0xFF) (8-byte big-endian i64)	CDC event payloads serialised as JSON
\xff__meta__/*	Metadata: latest sequence number, bridge watermark

Big-endian encoding is chosen so that RocksDB’s default byte-wise comparator iterates events in sequence order without a custom comparator. Metadata keys start with 0xFF, which sorts after all valid sequence keys (which are non-negative i64 values encoded big-endian and therefore never start with 0xFF), so a forward iterator stops naturally before reaching metadata.

Durability and crash recovery

Every write goes through a WriteBatch, which RocksDB commits atomically. On crash recovery, RocksDbCdcLog::open reads the persisted META_LATEST_SEQ key to restore the in-memory sequence counter, so appends resume from the correct position without scanning all events.

The bridge watermark (RocksDbCdcLog::append_and_set_bridge_watermark) is written in the same WriteBatch as the events it covers, making the bridge operation idempotent: after a mid-write crash, either both the events and the watermark are present, or neither is.

Quick start

use rhei_cdc_rocksdb::{RocksDbCdcLog, RocksDbCdcConfig};

let config = RocksDbCdcConfig {
    path: "/var/lib/rhei/cdc_rocksdb".to_string(),
    create_if_missing: true,
};
let log = RocksDbCdcLog::open(&config).unwrap();

Re-exports

pub use config::RocksDbCdcConfig;

pub use error::RocksDbCdcError;

pub use log::RocksDbCdcLog;

Modules

config

Configuration for opening a RocksDbCdcLog. Configuration for opening a crate::RocksDbCdcLog.

error

Error types returned by RocksDbCdcLog operations. Error types for the RocksDB CDC log.

keys

Key encoding and metadata-key constants for the RocksDB storage layout. Key encoding and metadata-key constants for the RocksDB storage layout.

log

RocksDbCdcLog implementation and its rhei_core::CdcConsumer trait impl. RocksDbCdcLog — the main entry point of this crate.