Crate kwaai_distributed 

kwaai-distributed

Distributed ML operations for KwaaiNet, implementing Hivemind patterns.

This crate provides:

• Mixture of Experts (MoE): Distributed model layers across network
• Decentralized Averaging: Parameter sync without master node
• Fault Tolerance: Graceful handling of node failures

Architecture

┌─────────────────────────────────────────────────────┐
│                  kwaai-distributed                   │
├─────────────────┬─────────────────┬─────────────────┤
│   MoE Layer     │   Averaging     │  Fault Tolerance │
│  (Expert Routing)│ (Gradient Sync) │  (Retry/Fallback)│
├─────────────────┴─────────────────┴─────────────────┤
│                    kwaai-p2p                         │
│               (P2P Networking / DHT)                 │
└─────────────────────────────────────────────────────┘

Re-exports

pub use averaging::AveragingResult;

pub use averaging::DecentralizedAverager;

pub use averaging::ParameterAverager;

pub use coordinator::DistributedCoordinator;

pub use error::DistributedError;

pub use error::DistributedResult;

pub use expert::Expert;

pub use expert::ExpertId;

pub use expert::ExpertRegistry;

pub use moe::ExpertRouter;

pub use moe::MixtureOfExperts;

pub use moe::Routing;

Modules

averaging

Decentralized parameter averaging (Hivemind pattern)

coordinator

Distributed operations coordinator

error

Error types for distributed operations

expert

Expert management for Mixture of Experts

moe

Mixture of Experts (MoE) implementation

Structs

DistributedConfig

Configuration for distributed operations