Crate d_engine 

D-Engine

d-engine is a lightweight and strongly consistent Raft consensus engine written in Rust. It is a base to build reliable and scalable distributed systems. Designed for resource efficiency, d-engine employs a single-threaded event-driven architecture that maximizes single CPU core performance while minimizing resource overhead. It plans to provide a production-ready implementation of the Raft consensus algorithm, with support for pluggable storage backends, observability, and runtime flexibility.

---

Features

• Strong Consistency: Full production-grade implementation of the Raft consensus protocol, including leader election, log replication, membership changes, and fault-tolerant state replication.
• Pluggable Storage: Supports custom storage backends (e.g., RocksDB).
• Configurable Snapshotting: Built-in snapshot generation with customizable policies, on-demand creation, or the option to disable snapshots entirely.
• Dynamic Cluster Membership: New nodes can join as learners and be promoted to followers based on configurable synchronization conditions.
• Optimized Resource Usage: Single-threaded, event-driven architecture maximizing single-core throughput while minimizing memory and disk overhead.
• Deep Observability: Exposes rich metrics, structured logs, and tracing data for integration with your own monitoring stack (e.g., Prometheus + Grafana).

---

Quick Start

use d_engine::NodeBuilder;
use d_engine::FileStorageEngine;
use d_engine::FileStateMachine;
use tokio::sync::watch;
use tracing::error;
use tracing::info;
use std::path::PathBuf;
use std::sync::Arc;


#[tokio::main(flavor = "current_thread")]
async fn main() {
    let (graceful_tx, graceful_rx) = watch::channel(());

    let path = PathBuf::from("/tmp/db");
    let storage_engine = Arc::new(FileStorageEngine::new(path.join("storage")).unwrap());
    let state_machine =
        Arc::new(FileStateMachine::new(path.join("state_machine"), 1).await.unwrap());

    let node = NodeBuilder::new(None, graceful_rx.clone())
        .storage_engine(storage_engine)
        .state_machine(state_machine)
        .build()
        .start_rpc_server()
        .await
        .ready()
        .expect("start node failed.");


    if let Err(e) = node.run().await {
        error!("node stops: {:?}", e);
    } else {
        info!("node stops.");
    }
}

Core Concepts

For production deployments, a minimum cluster size of 3 nodes is required.

Re-exports

pub use storage::*;

Modules

client

Client module for distributed consensus system

config

Configuration management module for distributed Raft consensus engine.

docs

D-Engine

node

Raft node container and lifecycle management.

proto

Protocol Buffer definitions and generated code for RPC services.

storage

Storage layer for d-engine.

Enums

ConsensusError

ConvertError

Error type for value conversion operations

ElectionError

Error

FileError

IdAllocationError

MembershipError

NetworkError

ProstError

Wrapper for prost encoding/decoding errors

ReadSendError

ReplicationError

SerializationError

SnapshotError

StorageError

SystemError

WriteSendError