kwaai-distributed

Distributed ML operations for KwaaiNet, implementing Hivemind patterns.

This crate provides:

• Mixture of Experts (MoE): Distributed model layers across network
• Decentralized Averaging: Parameter sync without master node
• Fault Tolerance: Graceful handling of node failures

Architecture

┌─────────────────────────────────────────────────────┐
│                  kwaai-distributed                   │
├─────────────────┬─────────────────┬─────────────────┤
│   MoE Layer     │   Averaging     │  Fault Tolerance │
│  (Expert Routing)│ (Gradient Sync) │  (Retry/Fallback)│
├─────────────────┴─────────────────┴─────────────────┤
│                    kwaai-p2p                         │
│               (P2P Networking / DHT)                 │
└─────────────────────────────────────────────────────┘