Expand description
Simple and fast BFT agreement inspired by Simplex Consensus.
Inspired by Simplex Consensus, simplex provides simple and fast BFT
agreement with network-speed view (i.e. block time) latency and optimal finalization latency in a
partially synchronous setting.
§Features
- Wicked Fast Block Times (2 Network Hops)
- Optimal Finalization Latency (3 Network Hops)
- Externalized Uptime and Fault Proofs
- Require Certification Before Finalization
- Decoupled Block Broadcast and Sync
- Lazy Message Verification
- Application-Defined Block Format
- Pluggable Hashing and Cryptography
- Embedded VRF (via scheme::bls12381_threshold::vrf)
§Design
§Protocol Description
§Genesis
Genesis (view 0) is implicitly finalized. There is no finalization certificate for genesis;
Config::floor supplies the initial finalized state. Voting begins
at view 1, with the first proposal referencing genesis as its parent.
§Specification for View v
Upon entering view v:
- Determine leader
lfor viewv - Set timer for leader proposal
t_l = 2Δand advancet_a = 3Δ- If leader
lhas not been active for the configured skip timeout while a quorum of participants has been, set botht_landt_ato 0.
- If leader
- If leader
l, broadcastnotarize(c,v)- If can’t propose container in view
vbecause missing notarization/nullification for a previous viewv_m, requestv_m
- If can’t propose container in view
Upon receiving first notarize(c,v) from l:
- Cancel
t_l - If the container’s parent
c_parentis finalized (or both notarized and certified) atv_parentand we have required nullifications covering the skipped views betweenv_parentandv(a nullification covers the rest of its term; whenvis not a term start,v_parentmust be exactlyv-1), verifycand broadcastnotarize(c,v)- If verification of
cfails, immediately broadcastnullify(v)
- If verification of
Upon receiving 2f+1 notarize(c,v):
- Mark
cas notarized - Broadcast
notarization(c,v)(even if we have not verifiedc) - Attempt to certify
c(see Certification), leavingt_aarmed so a stalled certification still times out the view- On success: enter
v+1and broadcastfinalize(c,v)(skipped if we have broadcastnullify(v)or observedlequivocate inv) - On failure: treat as immediate timeout expiry and broadcast
nullify(v)
- On success: enter
Upon receiving 2f+1 nullify(v):
- Broadcast
nullification(v) - Enter
next_term_start(v)(equivalent tov+1whenterm_length = 1)
Upon receiving 2f+1 finalize(c,v):
- Mark
cas finalized (and recursively finalize its parents) - Broadcast
finalization(c,v)(even if we have not verifiedc)
Upon t_l or t_a firing:
- Broadcast
nullify(v) - Every retry interval
t_rafternullify(v)broadcast that we are still in viewv:- Rebroadcast
nullify(v)alongside the best certificate for entering viewvwe hold at that time: the finalization ofv-1if we have one, otherwise the highest nullification we hold for the previous term (checked only whenvstarts a term, as every view does whenterm_lengthis 1), otherwise the notarization ofv-1. If we hold none (as in view 1), rebroadcastnullify(v)alone.
- Rebroadcast
When 2f+1 votes of a given type (notarize(c,v), nullify(v), or finalize(c,v)) have been collected
from unique participants, a certificate (notarization(c,v), nullification(v), or finalization(c,v)) can be assembled.
These certificates serve as a standalone proof of consensus progress that downstream systems can ingest without executing
the protocol.
§Joining Consensus
As soon as 2f+1 nullifies or finalizes are observed for some view v, the Voter will
enter the corresponding successor view (next_term_start(v) for nullification, v+1 for
finalization). Notarizations advance the view if-and-only-if the application certifies them.
This means that a new participant joining consensus will immediately jump ahead on the previous
view’s nullification or finalization and begin participating in consensus at the current view.
§Certification
After a payload is notarized, the application can optionally delay or prevent finalization via the
CertifiableAutomaton::certify method. By default, certify
returns true for all payloads, meaning finalization proceeds immediately after notarization.
Customizing certify is useful for systems that employ erasure coding, where participants may want
to wait until they have received enough shards to reconstruct and validate the full block before
voting to finalize.
If certify returns true, the participant broadcasts a finalize vote for the payload (unless it has
broadcast nullify or observed the leader equivocate) and enters the next view. If certify returns false, the participant broadcasts
nullify for the view instead (treating it as an immediate timeout), and will refuse to build upon the
proposal or notarize proposals that build upon it.
Thus, a payload can only be finalized if a quorum of participants certify it.
Certification of some notarization should only be abandoned once a finalization at the same or higher view is observed. Until then (say a nullification certificate for a view arrives before certification completes), the application should continue attempting to complete certification. This increases the likelihood that we can vote on the next honest proposer’s block (which may build on our in-flight certification or the nullification). If we did not do this, it is possible that different parts of the network (neither with quorum) would refuse to vote on each other’s blocks (halting consensus).
The decision returned by certify must be deterministic and consistent across all honest participants to ensure
liveness.
§Deviations from Simplex Consensus
- Fetch missing notarizations/nullifications as needed rather than assuming each proposal contains a set of all notarizations/nullifications for all historical blocks.
- Introduce distinct messages for
notarizeandnullifyrather than referring to both as avotefor either a “block” or a “dummy block”, respectively. - Introduce a “leader timeout” to trigger early view transitions for unresponsive leaders.
- Skip “leader timeout” and “certification timeout” if a designated leader has not participated for the configured skip timeout while a quorum of participants has (again to trigger early view transition for an unresponsive leader).
- Introduce message rebroadcast to continue making progress if messages from a given view are dropped (only way to ensure messages are reliably delivered is with a heavyweight reliable broadcast protocol).
- Treat local proposal failure as immediate timeout expiry and broadcast
nullify(v). - Treat local verification failure as immediate timeout expiry and broadcast
nullify(v). - Consider the current leader’s
nullify(v)as immediate timeout expiry and broadcastnullify(v). - Upon seeing
notarization(c,v), instead of moving to the viewv+1immediately, request certification from the application (see Certification). Only move to viewv+1and broadcastfinalize(c,v)if certification succeeds, otherwise broadcastnullify(v)and refuse to build uponc. - With stable leaders (
term_length > 1), a prior same-termnullifyvote blocks laterfinalizevotes until a covering finalization is observed; notarize votes are never withheld (see Same-Term Vote Safety). - With stable leaders, optionally verify proposals and broadcast
notarizevotes up tooptimistic_viewsviews ahead of certified ancestry within a term (configured alongside the term length, seeelector::Terms::stable); certification andfinalizevotes always wait for explicit parent certification (see Optimistic Validation). - If an entered view remains unfinalized for the stall timeout (configured alongside the term
length, see
elector::Terms) and we are still in the same term, we locally time out the current view and votenullify. In practice, this tracks the oldest unfinalized view we have entered in the current term. - Votes are tracked down to
view_retentionviews below the highest finalized view: late votes in that window are still reported, even though they are no longer verified or used for certificate construction. By default, certification releases full vote evidence, making later conflict reporting and peer blocking best effort.Config::track_historical_votesinstead retains each recorded vote until its round is pruned. Votes below the window are ignored on arrival, so downstream systems consuming per-vote activity (rewards, slashing) never observe them.
§Protocol Properties
§Forced Inclusion (Tail-Forking Resistance)
A notarized payload in view v must appear in the canonical chain if no nullification
certificate covers v. With stable leaders, a nullification covers the view it was created for
and the rest of that term. This follows directly from the protocol rules:
- To propose in view
v+k, the leader must reference a certified parent in some viewv_pand possess required nullifications covering the skipped views fromv_ptov+k. - A nullification certificate requires
2f+1nullifyvotes for the covered view or an earlier view in the same term. - An honest participant only broadcasts
nullifyon a nullify trigger: an expired timeout (t_l,t_a, or the stall timeout) or an event treated as immediate timeout expiry (proposal build failure, verification failure, certification failure, the leader’s ownnullify, or leader inactivity, as listed in Deviations).
Therefore, a nullification covering v can only form if at least f+1 honest participants
broadcast nullify at a single view u in [term_start(v), v]. If every view in that term
prefix completes without any nullify trigger (just view v itself when
term_length is 1), no honest participant has broadcast a covering nullify: at most f
covering votes exist at any single view, which is insufficient to form a nullification
certificate. Without that certificate, no future leader can skip view v, and the notarized
payload must be included as an ancestor in all subsequent proposals. Note that a clean view
v alone is not enough when term_length > 1: an honest nullify broadcast at an earlier
view of the term (say, after a transient timeout at a view that later notarized) covers v
even though no trigger fired at v itself.
§Same-Term Vote Safety
With stable leaders, a nullification covers the view it was created for and the rest of that
term: a later proposal may use it to skip all of those views at once. This is only safe if no
covered view is finalized, so the protocol must maintain the invariant that a finalization at
view v rules out a nullification at any view u <= v in the same term (otherwise a proposal
could fork around a finalized view).
The finalize gate maintains this invariant: a participant that voted nullify(u) withholds
finalize votes for later views in that term until it observes a same-term finalization at or
above its highest nullify vote. To see why the invariant holds, suppose both a nullification
at u and a finalization at v >= u form in the same term. Their quorums intersect in at
least one honest participant that voted both nullify(u) and finalize(c,v). If u = v,
this is impossible outright: no honest participant votes both nullify and finalize in a
single view. If u < v, the gate means that participant first observed a same-term
finalization at some v* with u <= v* < v: at or above u because the gate requires
covering its highest nullify vote, and below v because participants only vote finalize
for views above their highest observed finalization. Applying this same argument to the
finalization at v* shows, by induction, that no nullification can form at or below v*,
contradicting the nullification at u <= v*.
The gate also recovers from a nullify vote that never became a nullification (e.g., a
transient timeout on an otherwise healthy network): by the invariant, an observed same-term
finalization at or above the vote proves the nullification can never form, so the vote is
inert and the gate reopens (“heals”). This prevents one transient timeout from degrading the
rest of the term. In a healthy network this takes one view: peers broadcast finalize(v) when
they certify v, so the finalization for v typically arrives shortly after entering v+1.
Healing does not proactively revisit earlier views: a finalize vote for a view certified
while the gate was blocked is only emitted if a later message (such as a redelivered
notarization) touches that view again. If more than f participants were blocked, that view may never
gather its own finalization certificate, and neither may any later view in the term (healing
itself requires a same-term finalization, which cannot assemble while more than f
participants withhold finalize). Such a view is either finalized transitively by the
finalization of a descendant in a later term or skipped entirely by a covering nullification:
the timeouts that blocked the gate also mean forced inclusion does not apply to it.
§Optimistic Validation
With stable leaders, a leader can propose for view v+1 as soon as its proposal for view v
is notarized, but participants that wait for v’s certification before verifying the new
proposal add a round of certification latency to every view. When a nonzero optimistic_views
lookahead is configured (see elector::Terms::stable), a participant instead verifies a
proposal and broadcasts its notarize vote before the parent is certified, if all of the
following hold:
- The proposal’s view is in the same term as its parent (optimism never crosses a term boundary; a term start always requires explicitly certified ancestry).
- At most
optimistic_viewsviews lie between the proposal’s view and the last directly notarized view (a view with an observed notarization or finalization certificate; a view is indirectly notarized when only a descendant’s certificate implies it), bounding how far local votes run ahead of certified ancestry. This is the issuance window. - There is local evidence for the immediate parent: our own broadcast
notarizevote, an observed notarization certificate (unless our own certification rejected it), or (once the proposal’s view is current) the parent’s explicit certification.
Certification requests and finalize votes never run ahead: both require the parent’s
explicit certification first. If an optimistic ancestor fails to notarize or certify, the
usual timeout and nullification path skips it, and any optimistic votes above it are inert.
Peers admit and buffer votes up to optimistic_views views beyond their own current
view (the admission window), so optimistic votes are not dropped by participants that have
not yet observed the sender’s ancestry. The setting is local: mismatched values across
participants only degrade the optimization (votes beyond a peer’s window are dropped until it
catches up), never safety.
§Optimistic Finality
The forced inclusion property provides a weaker but faster form of finality: a payload
notarized at view v can be treated as speculatively final, because no future sequence of
proposals can exclude it from the canonical chain if every view in [term_start(v), v]
completes without any nullify trigger (just view v itself when term_length is 1).
This “speculative finality” is available after just 2 network hops (proposal + notarization),
compared to the 3 hops required for full finalization (proposal + notarization + finalization).
Observing the notarization does not by itself rule out exclusion: honest participants may
still be inside a view of the term prefix, where a trigger can still fire (say, a
certification that outlives t_a). Exclusion requires f+1 or more honest participants to
broadcast nullify at a single view of that prefix. Because certification is deterministic,
it either fails for all honest participants or none, so a certification failure always
produces a nullification. In the common case (no faults, no timeouts), exclusion cannot
happen.
A Byzantine leader, however, can exclude even its own valid, certifiable, and timely proposal:
honest participants treat the leader’s nullify(v) as an immediate timeout, so a leader can
single-handedly revoke its own notarized proposal. This is no new power. A leader can achieve
the same exclusion by delivering its proposal so late that honest participants notarize it but
time out before certifying. Speculative finality therefore assumes the term’s leader wants its
proposal to survive.
§Unchained Finalization
Finalization does not require consecutive honest views. When a participant certifies
notarization(c,v), it broadcasts finalize(c,v) and immediately enters v+1,
regardless of what happens in subsequent views. These finalize(c,v) votes accumulate
independently of the current view: even if views v+1 through v+k all time out
(producing nullifications), the finalize(c,v) votes still count toward the 2f+1
threshold needed to form finalization(c,v).
This means a payload notarized in view v can be finalized while the network is
in view v+k for any k >= 1. There is no requirement that a particular view
after v succeeds or that any subsequent leader cooperates. As long as 2f+1
participants eventually certify and broadcast finalize(c,v), the finalization
certificate will form.
§Architecture
All logic is split into four components: the Batcher, the Voter, the Resolver, and the Application (provided by the user).
The Batcher is responsible for collecting messages from peers and lazily verifying them when a quorum is met. The Voter
is responsible for directing participation in the current view. The Resolver is responsible for
fetching artifacts from previous views required to verify proposed blocks in the latest view. Lastly, the Application
is responsible for proposing new blocks and indicating whether some block is valid.
To drive great performance, all interactions between Batcher, Voter, Resolver, and Application are
non-blocking. This means that, for example, the Voter can continue processing messages while the
Application verifies a proposed block or the Resolver fetches a notarization.
+------------+ +++++++++++++++
| +--------->+ +
| Batcher | + Peers +
| |<---------+ +
+-------+----+ +++++++++++++++
| ^
| |
| |
| |
v |
+---------------+ +---------+ +++++++++++++++
| |<----------+ +----------->+ +
| Application | | Voter | + Peers +
| +---------->| |<-----------+ +
+---------------+ +--+------+ +++++++++++++++
| ^
| |
| |
| |
v |
+-------+----+ +++++++++++++++
| +--------->+ +
| Resolver | + Peers +
| |<---------+ +
+------------+ +++++++++++++++§Batched Verification
Unlike other consensus constructions that verify all incoming messages received from peers, for schemes
where Verifier::is_batchable() returns true
(such as scheme::ed25519, scheme::bls12381_multisig and scheme::bls12381_threshold), simplex lazily
verifies messages (only when a quorum is met), enabling efficient batch verification. For schemes where
is_batchable() returns false (such as scheme::secp256r1), signatures are verified eagerly as they
arrive since there is no batching benefit.
If an invalid signature is detected, the Batcher will perform repeated bisections over collected
messages to find the offending message (and block the peer(s) that sent it via commonware_p2p::Blocker).
If using a p2p implementation that is not authenticated, it is not safe to employ this optimization as any attacking peer could simply reconnect from a different address. We recommend commonware_p2p::authenticated.
§Fetching Missing Certificates
Background repair fetches nullifications above the local certified or finalized floor. If honest participants complete a view with different certificate types, both sides can still consider it complete while rejecting the other’s proposal ancestry.
Proposal verification repairs this split by requesting the first missing nullification or named parent from the proposal’s elected leader, even below the certified floor. The voter rechecks the full ancestry after each delivery and votes only once it is valid. The voter does not request an uncertified parent inside the optimistic issuance window: its certificate is still forming from live votes (see Optimistic Validation).
The same split can block certification. A notarized view certifies only after its parent certifies, and certifying the parent requires its exact-view notarization. When the voter holds a view’s notarization but not its parent’s, the parent’s votes have stopped circulating, and peers broadcast a certificate only once. A leader that withheld the certificate may never answer a fetch, so the voter requests the parent’s notarization from any peer.
A resolver key identifies a view, not a certificate. A notarization and a covering nullification for one view answer opposite questions, so a peer can return valid evidence that does not settle the request. The requester records that evidence and retries without faulting the peer. A delivered notarization completes its fetch on arrival, because certification judges evidence already in hand. Matching evidence or finalization retires pending work.
§Pluggable Hashing and Cryptography
Hashing is abstracted via the commonware_cryptography::Hasher trait and cryptography is abstracted via the commonware_cryptography::certificate::Scheme trait, allowing deployments to employ approaches that best match their requirements (or to provide their own without modifying any consensus logic). The following schemes are supported out-of-the-box:
§scheme::ed25519
commonware_cryptography::ed25519 signatures are “High-speed high-security signatures” with 32 byte public keys and 64 byte signatures. While they are well-supported by commercial HSMs and offer efficient batch verification, the signatures are not aggregatable (and certificates grow linearly with the quorum size).
§scheme::bls12381_multisig
commonware_cryptography::bls12381 is a “digital signature scheme with aggregation properties”. Unlike commonware_cryptography::ed25519, signatures from multiple participants (say the signers in a certificate) can be aggregated into a single signature (reducing bandwidth usage per broadcast). That being said, commonware_cryptography::bls12381 is much slower to verify than commonware_cryptography::ed25519 and isn’t supported by most HSMs (a standardization effort expired in 2022).
§scheme::secp256r1
commonware_cryptography::secp256r1 signatures use the NIST P-256 elliptic curve (also known as prime256v1), which is widely supported by commercial HSMs and hardware security modules. Unlike commonware_cryptography::ed25519, Secp256r1 does not benefit from batch verification, so signatures are verified individually. Certificates grow linearly with quorum size (similar to ed25519).
§scheme::bls12381_threshold
scheme::bls12381_threshold employs threshold cryptography (BLS12-381 threshold signatures with a 2f+1 of 3f+1 quorum)
to generate succinct consensus certificates (verifiable with just the static public key). This scheme requires instantiating
the shared secret via commonware_cryptography::bls12381::dkg and resharing whenever participants change.
Two (non-attributable) variants are provided:
-
scheme::bls12381_threshold::standard: Certificates contain only a vote signature.
-
scheme::bls12381_threshold::vrf: Certificates contain a vote signature and a view signature (i.e. a seed that can be used as a VRF). This variant can be configured for random leader election (via elector::Random) and/or incorporate this randomness into execution.
§Embedded VRF (scheme::bls12381_threshold::vrf)
Every notarize(c,v), nullify(v), or finalize(c,v) message includes an attestation(v) (a partial signature over the view v).
After 2f+1 attestations are collected from unique participants, seed(v) can be recovered. Because attestation(v) is only over the
view v, the seed derived for a given view v is the same regardless of which block (if any) is notarized in view v. The 2f+1
attestations can come from mutually incompatible messages (notarize for different blocks, nullify, finalize), so seed(v) is
recoverable even when no certificate forms for view v.
The value of seed(v) cannot be known prior to message broadcast by any participant (including the leader) in view v and cannot be
manipulated by any participant (deterministic for any 2f+1 signers at a given view v), so it is a sound beacon for leader election
(where seed(v) determines the leader for v+1). It is not safe as a source of randomness for execution within view v itself: a
coalition of f Byzantine participants recovers seed(v) after only f+1 honest attestations, before the round resolves, letting a
malicious leader front-run the outcome. Consume seed(v) only in a later view or epoch (see scheme::bls12381_threshold::vrf for
extended discussion of the attack and the commit-then-reveal mitigation).
§Succinct Certificates
All broadcast consensus messages (notarize(c,v), nullify(v), finalize(c,v)) contain attestations (partial signatures) for a static
public key (derived from a group polynomial that can be recomputed during reconfiguration using dkg).
As soon as 2f+1 messages are collected, a threshold signature over notarization(c,v), nullification(v), and finalization(c,v)
can be recovered, respectively. Because the public key is static, any of these certificates can be verified by an external
process without following the consensus instance and/or tracking the current set of participants (as is typically required
to operate a lite client).
These threshold signatures over notarization(c,v), nullification(v), and finalization(c,v) (i.e. the consensus certificates)
can be used to secure interoperability between different consensus instances and user interactions with an infrastructure provider
(where any data served can be proven to derive from some finalized block of some consensus instance with a known static public key).
§Persistence
The Voter caches all data required to participate in consensus to avoid any disk reads on
on the critical path. To enable recovery, the Voter writes valid messages it receives from
consensus and messages it generates to a write-ahead log (WAL) implemented by commonware_storage::journal::segmented::variable::Journal.
Before sending a message, any pending Journal appends are synced to prevent inadvertent Byzantine
behavior on restart (especially in the case of unclean shutdown). All appends made in the same event
loop iteration are coalesced into a single sync that runs after messages are constructed and before
any are broadcast (even if there is nothing to broadcast). The proposal payload relay is not a
consensus message and is not gated on this sync: to lower view latency, it is requested as soon
as the automaton returns a payload, which is safe because extra payload bytes (unlike votes)
cannot form a conflicting certificate (see Plan::Propose).
§Automaton Failure Semantics
If a validator is the leader for a view but cannot build a valid payload yet (for example because
it is still syncing), it should decline the Automaton::propose
request by dropping the response channel. Simplex treats this as a missing proposal, broadcasts
nullify(v), and other validators can use the leader-nullify fast path to skip the view.
Once propose returns a payload, the local proposer is committed to that payload for verification
and certification. Automaton::verify and
CertifiableAutomaton::certify are stable verdict APIs,
not backpressure or syncing signals. While missing data may still arrive (and/or a validator cannot
immediately determine if a payload is valid), implementations should keep these requests pending rather
than returning false or closing the channel.
Returning false from verify means the proposal is permanently invalid and causes a local
nullify. Returning false from certify means the notarized payload is permanently
uncertifiable for that round and also causes a local nullify. Closing certify does not cause
nullify(v) to be broadcast before the normal round deadline and can halt progress because
certification requests are not retried during the same run. The safe way to stop working on
certification is to keep the request pending until Simplex drops it after finalizing the block
or a descendant.
Re-exports§
pub use config::Config;pub use config::Floor;pub use config::ForwardPolicy;pub use config::SkipBudget;pub use config::SkipPolicy;
Modules§
- config
- elector
- Leader election strategies for simplex consensus.
- scheme
- Signing scheme implementations for
simplex. - types
- Types used in crate::simplex.
Structs§
- Engine
- Instance of
simplexconsensus engine.
Enums§
- Plan
- Describes how a payload should be broadcast to the network.