Rhei — lightweight serverless HTAP engine for Rust.

Rhei pairs Rusqlite (SQLite) for low-latency OLTP writes with a pluggable OLAP backend (Apache DataFusion or DuckDB) for analytical queries, bridged by trigger-based Change Data Capture (CDC) replication and automatic SQL query routing. No separate server process is required — the engine runs fully in-process.

Architecture

Standard HTAP mode:

Client
  |
  v
HtapEngine  (facade — this crate)
  |-- SqlParserRouter  ──► AST-based routing to OLTP or OLAP
  |-- RusqliteEngine   ──► WAL-mode SQLite, write conn + read pool
  |     └── RusqliteCdcProducer  ──► trigger-based _rhei_cdc_log
  |-- OlapBackend      ──► DuckDB or DataFusion (feature-gated)
  └── CdcSyncEngine    ──► polls CDC, applies DML to OLAP

Sidecar mode:

External DB (SQLite / PostgreSQL)
  │  polls by updated_at > watermark
  ▼
TimestampCdcConsumer<S: SourceConnector>
  │  CdcEvent stream
  ▼
CdcSyncEngine  (temporal or destructive)
  │  DML -> OLAP
  ▼
OlapBackend (DuckDB / DataFusion)

Operating modes

Standard HTAP mode

The default mode. A local SQLite database handles writes; CDC triggers capture every INSERT / UPDATE / DELETE into _rhei_cdc_log. A background sync loop (or manual [HtapEngine::sync_now]) applies those events to the OLAP engine so analytical queries see fresh data.

Sidecar mode

When [HtapConfig::sidecar] is Some(SidecarConfig { .. }), the engine follows an external database (SQLite or PostgreSQL) via timestamp-based CDC polling. No local SQLite write path is needed unless [SidecarConfig::enable_local_oltp] is true. Useful for adding OLAP capabilities to an existing application database without schema changes.

Sync modes

Feature flags

Quick start

use rhei::{HtapConfig, HtapEngine, TableSchema};
use arrow::datatypes::{DataType, Field, Schema};
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // 1. Configure the engine (in-memory OLAP, default DataFusion backend)
    let config = HtapConfig {
        oltp_path: "my_app.db".to_string(),
        olap_in_memory: true,
        ..HtapConfig::default()
    };

    // 2. Start the engine
    let mut engine = HtapEngine::new(config).await?;

    // 3. Register a table for HTAP replication
    let schema = TableSchema {
        name: "events".to_string(),
        arrow_schema: Arc::new(Schema::new(vec![
            Field::new("id",    DataType::Int64,  false),
            Field::new("label", DataType::Utf8,   true),
        ])),
        primary_key: vec!["id".to_string()],
    };
    engine.register_table(schema).await?;

    // 4. Write via OLTP (auto-routed by SqlParserRouter)
    engine.execute("INSERT INTO events VALUES (1, 'hello')", &[]).await?;

    // 5. Sync CDC events to OLAP
    engine.sync_now().await?;

    // 6. Query via OLAP (aggregate → routed to DataFusion)
    let batches = engine.query("SELECT COUNT(*) FROM events").await?;
    println!("{} batches returned", batches.len());

    // 7. Shut down cleanly
    engine.shutdown().await;
    Ok(())
}

Key types

• [HtapEngine] — the main engine facade; start here.
• [HtapConfig] — builder-style configuration.
• [SidecarConfig] — sidecar-specific settings (watermark persistence, delete detection, OLTP toggle).
• [CdcSource] — unified CDC consumer wrapper used internally by the sync engine.