Module distributed 

Distributed spatial computing framework

This module provides a comprehensive distributed computing framework for spatial algorithms, enabling scaling across multiple nodes, automatic load balancing, fault tolerance, and efficient data partitioning for massive spatial datasets. It supports both message-passing and shared-memory paradigms with optimized communication patterns.

Features

• Distributed spatial data structures: Scale KD-trees, spatial indices across nodes
• Automatic data partitioning: Space-filling curves, load-balanced partitioning
• Fault-tolerant computation: Checkpointing, automatic recovery, redundancy
• Adaptive load balancing: Dynamic workload redistribution
• Communication optimization: Bandwidth-aware algorithms, compression
• Hierarchical clustering: Multi-level distributed algorithms
• Streaming spatial analytics: Real-time processing of spatial data streams
• Elastic scaling: Add/remove nodes dynamically

Architecture

The framework uses a hybrid architecture combining:

• Master-worker pattern for coordination
• Peer-to-peer communication for data exchange
• Hierarchical topology for scalability
• Event-driven programming for responsiveness

Examples

use scirs2_spatial::distributed::{DistributedSpatialCluster, NodeConfig};
use scirs2_core::ndarray::array;

// Create distributed spatial cluster
let clusterconfig = NodeConfig::new()
    .with_node_count(4)
    .with_fault_tolerance(true)
    .with_load_balancing(true)
    .with_compression(true);

let mut cluster = DistributedSpatialCluster::new(clusterconfig)?;

// Distribute large spatial dataset
let large_dataset = array![[0.0, 0.0], [1.0, 0.0]];
cluster.distribute_data(&large_dataset.view()).await?;

// Run distributed k-means clustering
let (centroids, assignments) = cluster.distributed_kmeans(5, 100).await?;
println!("Distributed clustering completed: {} centroids", centroids.nrows());

// Query distributed spatial index
let query_point = array![0.5, 0.5];
let nearest_neighbors = cluster.distributed_knn_search(&query_point.view(), 10).await?;
println!("Found {} nearest neighbors across cluster", nearest_neighbors.len());

Structs

ClusterPerformanceMetrics

Cluster performance metrics

ClusterState

Overall cluster state

ClusterStatistics

Cluster statistics

CommunicationLayer

Communication layer for inter-node communication

CommunicationStats

Statistics for communication performance

DataPartition

Data partition for distributed processing

DistributedSpatialCluster

Distributed spatial computing cluster

DistributedSpatialIndex

Distributed spatial index

FaultDetector

Fault detector for monitoring node health

GlobalIndexMetadata

Global index metadata shared across nodes

IndexStatistics

Index statistics

LoadBalancer

Load balancer for distributed workload management

LoadMetrics

Load metrics for nodes

LocalSpatialIndex

Local spatial index on each node

NodeConfig

Node configuration for distributed cluster

NodeHealth

Node health information

NodeInstance

Individual node in the distributed cluster

PartitionMigration

Partition migration instruction

QueryParameters

Query parameters

RebalancePlan

Load rebalancing plan

RecoveryAction

Recovery action

RecoveryPlan

Recovery plan for fault tolerance

RoutingTable

Routing table for distributed queries

SpatialBounds

Spatial bounds for data partition

SpatialKey

Spatial key for routing

Enums

DistributedMessage

Distributed message types

FailureType

Types of failures that can be detected

LoadBalancingStrategy

Load balancing strategies

NodeStatus

Node status enumeration

QueryResults

Query results

QueryType

Types of distributed queries

RecoveryStrategy

Recovery strategies for different failure types