Struct FaultDetection 

pub struct FaultDetection {
    pub communication_issues: BTreeMap<XorName, VecDeque<Instant>>,
    pub dkg_issues: BTreeMap<XorName, VecDeque<Instant>>,
    pub elder_voting_issues: BTreeMap<XorName, VecDeque<Instant>>,
    pub probe_issues: BTreeMap<XorName, VecDeque<Instant>>,
    pub network_knowledge_issues: BTreeMap<XorName, VecDeque<Instant>>,
    pub unfulfilled_ops: BTreeMap<XorName, VecDeque<Instant>>,
    /* private fields */
}

Faulty nodes tracking. Allows various potential issues to be tracked and weighted, with unresposive or suspect nodes being noted on request, against which action can then be taken.

Fields

communication_issues: BTreeMap<XorName, VecDeque<Instant>>

The communication issues logged against a node, along with a timestamp.

dkg_issues: BTreeMap<XorName, VecDeque<Instant>>

The dkg issues logged against a node, along with a timestamp to expire after some time.

elder_voting_issues: BTreeMap<XorName, VecDeque<Instant>>

The elder voting issues logged against a node, along with a timestamp to expire after some time.

probe_issues: BTreeMap<XorName, VecDeque<Instant>>

The probe issues logged against a node and as yet unfulfilled, along with a timestamp to expire after some time.

network_knowledge_issues: BTreeMap<XorName, VecDeque<Instant>>

The network knowledge issues logged against a node, along with a timestamp.

unfulfilled_ops: BTreeMap<XorName, VecDeque<Instant>>

The unfulfilled pending request operation issues logged against a node, along with an operation ID.

Implementations

impl FaultDetection

pub fn calculate_scores( &self, nodes_in_question: &BTreeSet<XorName>, ) -> ScoreResults

Calculate the scores of all nodes being tracked and return them in a node -> score map. There is a map for each type of issue.

The ‘score’ for a node is the number of issues logged against that node, minus the average of the number of issues at all the other nodes.

These scores can then be used to highlight nodes that have a higher score than some particular ratio.

pub fn get_faulty_nodes(&mut self) -> Vec<XorName>

Get a list of nodes that are faulty (the nodes must all ProposeOffline over a faulty node and then immediately vote it off. So any other membershipn changes in flight could block this. thus, we need to be callling this function often until nodes are removed.)

The vec is ordered from fauliest to least faulty (returning faulty elders first)

impl FaultDetection

pub fn new( non_elder_nodes: BTreeSet<XorName>, elders: BTreeSet<XorName>, ) -> Self

Set up a new tracker.

pub fn track_issue(&mut self, node_id: XorName, issue_type: IssueType)

Adds an issue to the fault tracker.

The op_id only applies when adding an operational issue.

pub fn dkg_ack_fulfilled(&mut self, node_id: &XorName)

Removes a DKG session from the node liveness records.

pub fn elder_vote_received(&mut self, node_id: &XorName)

Removes a Knowledge issue from the node liveness records.

pub fn ae_update_msg_received(&mut self, node_id: &XorName)

Removes a probe tracker from the node liveness records.

pub fn add_new_node(&mut self, node: XorName)

Add a new node to the tracker and recompute closest nodes.

pub fn update_and_only_retain_members( &mut self, non_elder_nodes: BTreeSet<XorName>, elders: BTreeSet<XorName>, )

Removes tracked nodes not present in current_members.

Tracked issues related to nodes that were removed will also be removed.

Trait Implementations

impl Clone for FaultDetection

fn clone(&self) -> FaultDetection

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for FaultDetection

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.