Module docs 

D-Engine

d-engine is a lightweight and strongly consistent Raft consensus engine written in Rust. It is a base to build reliable and scalable distributed systems. Designed for resource efficiency, d-engine employs a single-threaded event-driven architecture that maximizes single CPU core performance while minimizing resource overhead. It plans to provide a production-ready implementation of the Raft consensus algorithm, with support for pluggable storage backends, observability, and runtime flexibility.

---

Features

• Strong Consistency: Full production-grade implementation of the Raft consensus protocol, including leader election, log replication, membership changes, and fault-tolerant state replication.
• Pluggable Storage: Supports custom storage backends (e.g., RocksDB).
• Configurable Snapshotting: Built-in snapshot generation with customizable policies, on-demand creation, or the option to disable snapshots entirely.
• Dynamic Cluster Membership: New nodes can join as learners and be promoted to followers based on configurable synchronization conditions.
• Optimized Resource Usage: Single-threaded, event-driven architecture maximizing single-core throughput while minimizing memory and disk overhead.
• Deep Observability: Exposes rich metrics, structured logs, and tracing data for integration with your own monitoring stack (e.g., Prometheus + Grafana).

---

Quick Start

use d_engine::NodeBuilder;
use d_engine::FileStorageEngine;
use d_engine::FileStateMachine;
use tokio::sync::watch;
use tracing::error;
use tracing::info;
use std::path::PathBuf;
use std::sync::Arc;


#[tokio::main(flavor = "current_thread")]
async fn main() {
    let (graceful_tx, graceful_rx) = watch::channel(());

    let path = PathBuf::from("/tmp/db");
    let storage_engine = Arc::new(FileStorageEngine::new(path.join("storage")).unwrap());
    let state_machine =
        Arc::new(FileStateMachine::new(path.join("state_machine"), 1).await.unwrap());

    let node = NodeBuilder::new(None, graceful_rx.clone())
        .storage_engine(storage_engine)
        .state_machine(state_machine)
        .build()
        .start_rpc_server()
        .await
        .ready()
        .expect("start node failed.");


    if let Err(e) = node.run().await {
        error!("node stops: {:?}", e);
    } else {
        info!("node stops.");
    }
}

Core Concepts

For production deployments, a minimum cluster size of 3 nodes is required.

Modules

architecture

This project strictly follows the Single Responsibility Principle (SRP) to ensure modularity and maintainability. Developers extending the codebase must adhere to this principle to preserve clean separation of concerns. Below are key guidelines and examples.

performance

Raft Throughput Optimization Guide for Tonic gRPC in Rust

server_guide

Implementing Custom Storage Engines