Crate d_engine_core 

d-engine-core

Pure Raft consensus algorithm - for building custom Raft-based systems

⚠️ Internal Crate - Not Ready for Standalone Use

Use d-engine instead.

This crate contains the pure Raft consensus algorithm used internally by d-engine. The API is unstable before v1.0.

# ❌ Don't use this directly
[dependencies]
d-engine-core = "0.2"

# ✅ Use this instead
[dependencies]
d-engine = "0.2"

For Contributors

What this crate provides

This crate focuses solely on the Raft consensus algorithm:

• Leader Election - Automatic leader election with randomized timeouts
• Log Replication - Reliable log replication to followers
• Membership Changes - Dynamic cluster membership changes
• Snapshot Support - Log compaction via snapshots

Storage, networking, and state machine implementation are your responsibility.

Reference integration: See how d-engine-server uses this crate.

Future Vision

Post-1.0 goal: Become a standalone Raft library with stable API.

Current status: Internal to d-engine, API may change between minor versions.

Key Traits

• StorageEngine - Persistent storage for Raft logs
• StateMachine - Application-specific state transitions
• LogStore - Log entry persistence
• MetaStore - Metadata persistence (term, voted_for)

Documentation

For comprehensive guides:

• Customize Storage Engine
• Customize State Machine

Re-exports

pub use config::*;

pub use storage::*;

pub use watch::*;

Modules

config

Configuration management module for distributed Raft consensus engine.

storage

watch

Watch mechanism for monitoring key changes

Structs

AppendResponseWithUpdates

AppendEntries response with possible state changes

AppendResult

AppendResults

BatchBuffer

ClusterMetadata

Cached cluster topology metadata for hot path optimization.

ClusterUpdateResult

CommitHandlerDependencies

DefaultCommitHandler

DefaultPurgeExecutor

Default implementation using the configured log storage

DefaultStateMachineHandler

ElectionHandler

LeaderInfo

Leader election information Used at: Application layer (internal Raft protocol notifications) Purpose: Notify applications about leader changes via watch channel Fields: Minimal - only what Raft protocol needs

LogSizePolicy

MaybeCloneOneshot

MaybeCloneOneshotReceiver

MaybeCloneOneshotSender

NewCommitData

PeerUpdate

Raft

RaftContext

RaftCoreHandlers

RaftRequestWithSignal

Request with response channel that can handle all Raft payload types

RaftStorageHandles

ReplicationData

ReplicationHandler

SignalParams

SnapshotContext

SnapshotTransferMeta

Unified snapshot metadata with precomputed values

StateUpdate

State transition data for election outcomes

StreamResponseSender

VoteResult

Enums

ConnectionType

ConsensusError

ConvertError

Error type for value conversion operations

ElectionError

Error

FileError

IdAllocationError

MembershipError

NetworkError

ProstError

Wrapper for prost encoding/decoding errors

QuorumStatus

RaftEvent

ReadIndexError

Errors that can occur during ReadIndex batching for linearizable reads

ReadSendError

ReplicationError

RoleEvent

SerializationError

SnapshotError

StorageError

SystemError

WriteSendError

Traits

ElectionCore

Membership

PurgeExecutor

Defines the behavior for log entry compaction and physical deletion.

RaftOneshot

ReplicationCore

Core replication protocol operations

SnapshotPolicy

StateMachineHandler

Transport

Functions

client_command_to_entry_payloads

Converts a vector of client WriteCommands into a vector of EntryPayloads. Each WriteCommand is serialized into bytes and wrapped in an EntryPayload::Command variant.

ensure_safe_join

grpc_task_with_timeout_and_exponential_backoff

As soon as task has return we should return from this function