Crate datacake_node

Datacake Node

The core membership system used within Datacake.

This system allows you to build cluster extensions on top of this core functionality giving you access to the live membership watchers, node selectors, cluster clock, etc…

A good example of this is the datacake-eventual-consistency crate, it simply implements the ClusterExtension crate which lets it be added at runtime without issue.

Features

• Zero-copy RPC framework which allows for runtime adding and removing of services.
• Changeable node selector used for picking nodes out of a live membership to handle tasks.
• Pre-built data-center aware node selector for prioritisation of nodes in other availability zones.
• Distributed clock used for keeping an effective wall clock which respects causality.

Getting Started

To get started we’ll begin by creating our cluster:

use std::net::SocketAddr;

use datacake_node::{ConnectionConfig, DCAwareSelector, DatacakeNodeBuilder};

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let bind_addr = "127.0.0.1:8000".parse::<SocketAddr>().unwrap();

    // We setup our connection config for the node passing in the bind address, public address and seed nodes.
    // Here we're just using the bind address as our public address with no seed, but in the real world
    // this will be a different value when deployed across several servers with seeds to contact.
    let connection_cfg = ConnectionConfig::new(bind_addr, bind_addr, Vec::<String>::new());

    // Our builder lets us configure the node.
    //
    // We can configure the node selector, data center of the node, cluster ID, etc...
    let _my_node = DatacakeNodeBuilder::<DCAwareSelector>::new(1, connection_cfg).connect().await?;

    // Now we're connected we can add any extensions at runtime, our RPC server will already be
    // running and setup.
    //
    // Check out the `datacake-eventual-consistency` implementation for a demo.

    Ok(())
}

Creating A Extension

Creating a cluster extension is really simple, it’s one trait and it can do just about anything:

use datacake_node::{ClusterExtension, DatacakeNode};
use async_trait::async_trait;

pub struct MyExtension;

#[async_trait]
impl ClusterExtension for MyExtension {
    type Output = ();
    type Error = MyError;

    async fn init_extension(
        self,
        node: &DatacakeNode,
    ) -> Result<Self::Output, Self::Error> {
        // In here we can setup our system using the live node.
        // This gives us things like the cluster clock and RPC server:

        println!("Creating my extension!");

        let timestamp = node.clock().get_time().await;
        println!("My timestamp: {timestamp}");

        Ok(())
    }
}

pub struct MyError;

Structs

ChitchatNode

Clock

ClusterMember

ClusterStatistics

Live metrics around the cluster system.

ConnectionConfig

Configuration for the cluster network.

DCAwareSelector

A data-center aware node selector.

DatacakeHandle

DatacakeNode

DatacakeNodeBuilder

Build a Datacake node using provided settings.

MembershipChange

NodeSelectorHandle

A handle to the node selector actor responsible for working out what replicas should be prioritized when sending events based on a given consistency level.

RpcNetwork

A collection of RPC client connections which can be reused and multiplexed.

Enums

Consistency

The consistency level which should be reached before a write can be returned as successful.

ConsistencyError

NodeError

Statics

DEFAULT_CLUSTER_ID

DEFAULT_DATA_CENTER

Traits

ClusterExtension

An extension of the base node/cluster.

NodeSelector

Type Aliases

NodeId

Nodes