Crate connect

Connect framework is primarily aimed at easing the process of developing fault-tolerant distributed systems in Rust.

What is distributed system?

A distributed system is a group of machines that are virtually or geographically seperated and that work togather to provide the same service or application to clients.

Why are distributed systems needed?

With the exponential growth in volume of highly sensitive data, it has become increasingly important to distribute and replicate it across a cluster of machines to maintain reliable and fault-tolerant systems.

There are many cases n which the use of single computer would be possible in principle but the use of distributed system is benificial for practical reasons. For example, it may be more cost efficient to obtain the desired level of performance by using a cluster of several low-end computers, in comparison with a single high-end computer. A distributed system can provide more reliability than a non-distributed system, as there is no single point of failure. Moreover, a distributed system may be easier to expand and manage than a monolithic uniprocessor system.

How distributed systems work in harmony?

A fundamental problem in distributed systems is to create harmony between different nodes of a cluster. The underlying problem requires agreement among number of nodes for a single data value.

There are many algorithms especially created for handling consensus among nodes in a cluster. Connect framework uses Raft consensus algorithm to replicate state across the cluster.

Usage

Below is a simple example showing a way of creating a distributed atomic counter using connect framework.

use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::Arc;
use std::thread;
use std::time::Duration;
 
use bytes::Bytes;
use connect::{Cluster, MessageType, Server, ServiceType};
 
let counter = Arc::new(AtomicUsize::default());
 
let cluster = Cluster::with_file("Connect.toml");
 
let incoming = |bytes: Bytes| {
    if bytes[..4] == b"incr"[..] {
        MessageType::Write
    } else {
        MessageType::Read
    }
};
 
let on_read = {
    let counter = counter.clone();
    move |_| Bytes::from(format!("{}\n", counter.load(Ordering::SeqCst)))
};
 
let on_write = {
    let counter = counter.clone();
    move |_| {
        counter.store(counter.load(Ordering::SeqCst) + 1, Ordering::SeqCst);
 
        Bytes::from(format!("{}\n", counter.load(Ordering::SeqCst)))
    }
};
 
let server = Server::new(cluster, incoming, on_read, on_write, ServiceType::InMemory);
 
server.run();

Important notice

This crate is under active development. Please do not use it in production.

Structs

Cluster	Cluster represents a group of servers in a distributed system.
Connection	An object of Connection represents an incoming or outgoing connection.
ConnectionHandle	A handle to the Connection used to write some bytes on Connection's stream.
Node	Node represents a machine which is virtually or geographically seperated from other machines in a Cluster.
Server	This struct represents connect framework's server.

Enums

ConnectionMode	An enum representing different modes of connection.
Error	Defines errors in connect framework.
MemberType	Represents type of a member in a Cluster
MessageType	Represents type of the message received from a client.
ServiceType	This enum represents the type of service you're implementing using connect framework.

Type Definitions

NodeId

A type for unique identifiers for each node in a Cluster