Struct agreed::raft::Raft

pub struct Raft<D: AppData, R: AppDataResponse, N: RaftNetwork<D>, S: RaftStorage<D, R>> { /* fields omitted */ }

The Raft API.

This type implements the full Raft spec, and is the interface to a running Raft node. Applications building on top of Raft will use this to spawn a Raft task and interact with the spawned task.

For more information on the Raft protocol, see the specification here (pdf warning).

For details and discussion on this API, see the Raft API section of the guide.

clone

This type implements Clone, and should be cloned liberally. The clone itself is very cheap and helps to facilitate use with async workflows.

shutting down

If any of the interfaces returns a RaftError::ShuttingDown, this indicates that the Raft node is shutting down (potentially for data safety reasons due to a storage error), and the shutdown method should be called on this type to await the shutdown of the node. If the parent application needs to shutdown the Raft node for any reason, calling shutdown will do the trick.

Implementations

impl<D: AppData, R: AppDataResponse, N: RaftNetwork<D>, S: RaftStorage<D, R>> Raft<D, R, N, S>

pub fn new(
    id: NodeId,
    config: Arc<Config>,
    network: Arc<N>,
    storage: Arc<S>
) -> Self

Create and spawn a new Raft task.

id

The ID which the spawned Raft task will use to identify itself within the cluster. Applications must guarantee that the ID provided to this function is stable, and should be persisted in a well known location, probably alongside the Raft log and the application’s state machine. This ensures that restarts of the node will yield the same ID every time.

config

Raft’s runtime config. See the docs on the Config object for more details.

network

An implementation of the RaftNetwork trait which will be used by Raft for sending RPCs to peer nodes within the cluster. See the docs on the RaftNetwork trait for more details.

storage

An implementation of the RaftStorage trait which will be used by Raft for data storage. See the docs on the RaftStorage trait for more details.

pub async fn append_entries(
    &self,
    rpc: AppendEntriesRequest<D>
) -> Result<AppendEntriesResponse, RaftError>

Submit an AppendEntries RPC to this Raft node.

These RPCs are sent by the cluster leader to replicate log entries (§5.3), and are also used as heartbeats (§5.2).

pub async fn vote(&self, rpc: VoteRequest) -> Result<VoteResponse, RaftError>

Submit a VoteRequest (RequestVote in the spec) RPC to this Raft node.

These RPCs are sent by cluster peers which are in candidate state attempting to gather votes (§5.2).

pub async fn install_snapshot(
    &self,
    rpc: InstallSnapshotRequest
) -> Result<InstallSnapshotResponse, RaftError>

Submit an InstallSnapshot RPC to this Raft node.

These RPCs are sent by the cluster leader in order to bring a new node or a slow node up-to-speed with the leader (§7).

pub async fn current_leader(&self) -> Option<NodeId>

Get the ID of the current leader from this Raft node.

This method is based on the Raft metrics system which does a good job at staying up-to-date; however, the client_read method must still be used to guard against stale reads. This method is perfect for making decisions on where to route client requests.

pub async fn client_read(&self) -> Result<(), ClientReadError>

Check to ensure this node is still the cluster leader, in order to guard against stale reads (§8).

The actual read operation itself is up to the application, this method just ensures that the read will not be stale.

pub async fn client_write(
    &self,
    rpc: ClientWriteRequest<D>
) -> Result<ClientWriteResponse<R>, ClientWriteError<D>>

Submit a mutating client request to Raft to update the state of the system (§5.1).

It will be appended to the log, committed to the cluster, and then applied to the application state machine. The result of applying the request to the state machine will be returned as the response from this method.

Our goal for Raft is to implement linearizable semantics. If the leader crashes after committing a log entry but before responding to the client, the client may retry the command with a new leader, causing it to be executed a second time. As such, clients should assign unique serial numbers to every command. Then, the state machine should track the latest serial number processed for each client, along with the associated response. If it receives a command whose serial number has already been executed, it responds immediately without reexecuting the request (§8). The RaftStorage::apply_entry_to_state_machine method is the perfect place to implement this.

These are application specific requirements, and must be implemented by the application which is being built on top of Raft.

pub async fn initialize(
    &self,
    members: HashSet<NodeId>
) -> Result<(), InitializeError>

Initialize a pristine Raft node with the given config.

This command should be called on pristine nodes — where the log index is 0 and the node is in NonVoter state — as if either of those constraints are false, it indicates that the cluster is already formed and in motion. If InitializeError::NotAllowed is returned from this function, it is safe to ignore, as it simply indicates that the cluster is already up and running, which is ultimately the goal of this function.

This command will work for single-node or multi-node cluster formation. This command should be called with all discovered nodes which need to be part of cluster, and as such it is recommended that applications be configured with an initial cluster formation delay which will allow time for the initial members of the cluster to be discovered (by the parent application) for this call.

If successful, this routine will set the given config as the active config, only in memory, and will start an election.

It is recommended that applications call this function based on an initial call to RaftStorage.get_initial_state. If the initial state indicates that the hard state’s current term is 0 and the last_log_index is 0, then this routine should be called in order to initialize the cluster.

Once a node becomes leader and detects that its index is 0, it will commit a new config entry (instead of the normal blank entry created by new leaders).

Every member of the cluster should perform these actions. This routine is race-condition free, and Raft guarantees that the first node to become the cluster leader will propagate only its own config.

pub async fn add_non_voter(&self, id: NodeId) -> Result<(), ChangeConfigError>

Synchronize a new Raft node, bringing it up-to-speed (§6).

Applications built on top of Raft will typically have some peer discovery mechanism for detecting when new nodes come online and need to be added to the cluster. This API facilitates the ability to request that a new node be synchronized with the leader, so that it is up-to-date and ready to be added to the cluster.

Calling this API will add the target node as a non-voter, starting the syncing process. Once the node is up-to-speed, this function will return. It is the responsibility of the application to then call change_membership once all of the new nodes are synced.

If this Raft node is not the cluster leader, then this call will fail.

pub async fn remove_non_voter(
    &self,
    old_non_voter: NodeId
) -> Result<(), ChangeConfigError>

Cancel replication to the specified Non-Voter node.

Although the Non-Voter role and the machinery around it mainly serves the purpose of syncing and bringing up a node to speed prior to adding it as a Voter member (to prevent disrupting the cluster), it can very well be used as a read-only replication/change data capture mechanism. To support the latter use-case, this API can be used to remove Non-Voter nodes.

If the node is being synced as part of a configuration change (causing the leader to be in CatchingUp consensus state) then it cannot be removed. You have to wait for the configuration change to take place.

If this Raft node is not the cluster leader, then this call will fail.

pub async fn add_voter(
    &self,
    new_voter: NodeId
) -> Result<(), ChangeConfigError>

Request the addition of the specified node as a Voter member of the cluster. Will return once the configuration change has finished.

This will cause the leader to begin a cluster membership configuration change. If the new node is not already registered as a Non-Voter — from an earlier call to add_non_voter — then the new node will first be synced as a Non-Voter before actually performing the membership change. While the synchronization takes place, the Leader enters CatchingUp consensus state, preventing other configuration changes. As this process may take some time, it is recommended that add_non_voter be called first for the new node, and then once the new node has been synchronized, call this method to start reconfiguration.

If this Raft node is not the cluster leader or there is already another configuration change in progress, then the change will be rejected.

pub async fn remove_voter(
    &self,
    old_voter: NodeId
) -> Result<(), ChangeConfigError>

Request the removal of the specified Voter node from the cluster. Will return once the configuration change has finished.

Please note, that it is not guaranteed to be safe to immediately shut down the removed node once this call returns. The node is only safe to be shut down once it replicated the config change. Afterwards, it will not receive entries anymore. Shutting down the node early will cause the cluster leader to constantly attempt to replicate the config change to the removed node.

If this Raft node is not the cluster leader or there is already another configuration change in progress, then the change will be rejected.

pub fn metrics(&self) -> Receiver<RaftMetrics>

Get a handle to the metrics channel.

pub async fn shutdown(&self) -> Result<()>

Shutdown this Raft node.

Trait Implementations

impl<D: AppData, R: AppDataResponse, N: RaftNetwork<D>, S: RaftStorage<D, R>> Clone for Raft<D, R, N, S>

fn clone(&self) -> Self

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.