bytesandbrains provides composable, transport-agnostic building blocks for peer-to-peer overlay networks and distributed machine learning.
Workspace
| Crate | Description |
|---|---|
| bb-core | Shared traits and types: Address, PeerId, Peer, OverlayProtocol, PeerSampling, Embedding/EmbeddingSpace, and distance metrics (optional SIMD via simsimd). |
| bb-overlay | Protocol implementations built on bb-core. |
| bb-codec | Product Quantization: ProductQuantizer, PQCode, distance tables for approximate nearest neighbor search. |
| bb-index | Vector indexing: FlatIndex with add/search/remove/train operations. |
| bb-ml | Machine learning utilities: K-Means clustering. |
Protocols
Gossip Peer Sampling
A gossip-based peer sampling service that maintains self-healing, randomized overlay networks. Supports Push, Pull, and PushPull exchange modes with configurable selection strategies.
Protean
A distributed approximate nearest-neighbor search protocol. Combines a gossip-based random overlay with a Sparse Neighbor View (SNV) that uses occlusion-based pruning to organize peers by proximity in embedding space.
GOSSIP-PQ (in development)
A decentralized protocol for learning globally representative Product Quantization codebooks via gossip-based subspace exchange. Enables distributed ANN search without central coordination.
Feature Flags
Feature flags are configured per-crate. Key flags on bb-overlay:
| Flag | Default | Description |
|---|---|---|
gossip |
No | Gossip peer sampling protocol |
protean |
No | Protean kNN/kFN protocol |
proto |
No | Protocol Buffers serialization (via prost) |
full |
No | All of the above |
License
GPL-2.0