Remote Hash Map

A distributed node cluster management system using gRPC, implementing a replicated key-value store.

Table of Contents

• Features
• Architecture
• Getting Started
  • Prerequisites
  • Installation
• Usage
  • Running the Node Group
  • Running Individual Nodes
  • Using the Client
• Docker Support
• Configuration
• Logging
• Troubleshooting
• Contributing
• License

Features

• Distributed key-value storage
• Node cluster management
• Health checking via regular pinging
• Data replication across nodes
• Configurable ping intervals
• Docker support for easy deployment

Architecture

The system consists of two main components:

1. Node: Individual servers capable of storing key-value pairs and responding to ping requests.
2. NodeGroup: A manager for the node cluster, handling node addition, retrieval, and health monitoring.

Getting Started

Prerequisites

• Rust (with Cargo package manager)
• protoc (Protocol Buffers compiler) for gRPC
• Docker (optional, for containerized deployment)

Installation

1. Clone the repository:

    git clone https://github.com/kwdowicz/remote_hash_map.git
    cd remote_hash_map
   

2. Build the project:

   cargo build --release
   

Usage

Running the Node Group

Start the NodeGroup service:

./target/release/ng --listen 127.0.0.1:5000

Running Individual Nodes

Run nodes and connect them to the NodeGroup:

./target/release/node --listen 127.0.0.1:6001 --ng 127.0.0.1:5000
./target/release/node --listen 127.0.0.1:6002 --ng 127.0.0.1:5000

Using the Client

1. Create a new project:

    mkdir rhm_test && cd rhm_test
    cargo init
    cargo add remote_hash_map tokio
   

2. Add the following to src/main.rs:

   use remote_hash_map::common::client::Client;
   
    #[tokio::main]
    async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut client = Client::connect("127.0.0.1:5000").await?;
   
    client.set("name", "Daria").await?;
    client.set("name", "Tosia").await?;
    client.set("name", "Gabi").await?;
   
    let result = client.get("name").await?;
    println!("Name: {}", result);
   
    Ok(())
    }
   

3. Run the client:

  cargo run

Using grpcurl example

grpcurl -plaintext -d '{"key": "name", "value": "Tas", "replication": true }' 127.0.0.1:5021 node_rpc.NodeRpc/Set
grpcurl -plaintext -d '{}' 127.0.0.1:5000 node_group_rpc.NodeGroupRpc/GetServer

Docker Support

Build and run using Docker:

docker build -t remote_hash_map .
docker network create --subnet=192.168.0.0/16 my_network
docker run --name ng --network my_network --ip 192.168.0.3 remote_hash_map ng
docker run --name node --network my_network --ip 192.168.0.4 -e NG_ADDRESS="192.168.0.3:5000" remote_hash_map node

Configuration

• Ping interval can be configured for the NodeGroup:

  ./target/release/ng --listen 127.0.0.1:5000 --ping-sec 4
  

Logging

The project uses env_logger. Set the RUST_LOG environment variable for different log levels:

RUST_LOG=debug ./target/release/node --listen 127.0.0.1:6001 --ng 127.0.0.1:5000

Troubleshooting

• Ensure all dependencies are installed and properly configured.
• Verify that specified ports are not in use or blocked by firewalls.
• Check log output for errors or warnings.

Contributing

Contributions are welcome! Please submit a pull request or open an issue to discuss proposed changes or improvements.

License

This project is licensed under the MIT License.