Expand description
§music-dht
A distributed music library directory on top of federation_net:
peers publish their local library (artists, releases, tracks — names and
small metadata, never files) into a Kademlia-style DHT and search each
other’s libraries. Every running node is simultaneously a client, a DHT
router and a storage node — there are no dedicated bootstrap, index or
search servers.
§How it works
- Every peer derives a stable 256-bit
NodeIdfrom its persistentfederation-netendpoint id. LibraryItemrecords are published under BLAKE3-derivedDhtKeys: one exact key for the whole normalized name plus one key per token of the name, every main/featured artist name and the track’s release title (so a track is found by its artists).- Track records may also carry a compact
b3:<hex>content id and are then published under a content key, so applications can find another peer with the exact same audio bytes. similaritydefines the bounded, model-neutral stream contract used by clients that independently generate compatible music embeddings.similarity_lshturns those embeddings into deterministic compact routing summaries, whilesimilarity_dhtstores and discovers signed peer summaries without owning model inference or an embedding database.- Records are replicated to the
Knodes whose ids are XOR-closest to each key. Publishers pick the targets from their routing table and send batched store requests (one pipeline per peer), so even a large library republishes in seconds; searches discover records with an iterative Kademlia-style lookup (never a broadcast). - Peers learn about each other through a Hello/PeerExchange gossip that runs automatically on every new connection; connections to further nodes are opened on demand from stored tickets.
- Deletions propagate as tombstones that win over active records of the same or lower revision; replicas expire by TTL and owners republish periodically.
§Example
use music_dht::{ItemKind, ItemSpec, MusicDhtConfig, MusicDhtService};
use federation_net::{NetworkId, RendezvousConfig};
let config = MusicDhtConfig::builder()
.data_dir("./peer-a")
.network_id(NetworkId::from_name("my-music-network"))
// Find other peers knowing only the network id.
.rendezvous(RendezvousConfig::default())
.build()?;
let (service, mut events) = MusicDhtService::start(config).await?;
// Publish (and keep in sync) the local library.
service
.sync_library(vec![ItemSpec {
local_key: "track:42".into(),
kind: ItemKind::Track,
name: "Teardrop".into(),
artist_names: vec!["Massive Attack".into()],
featured_artist_names: vec![],
year: Some(1998),
release_type: None,
release_title: Some("Mezzanine".into()),
track_number: Some(10),
disc_number: Some(1),
duration_seconds: Some(330.0),
content_id: None,
}])
.await?;
// Search the whole network.
let outcome = service.search_network("massive attack").await?;
for item in &outcome.network_results {
println!("{} {} owned by {}", item.kind, item.name, item.owner);
}Re-exports§
pub use similarity_lsh::SimilarityRouteEntry;
Modules§
- capabilities
- Runtime publication of application and protocol versions.
- catalog
- Shared wire shapes for Furumi catalog streams.
- device_
sync - Wire protocol shared by furumi personal-device synchronization clients.
- jam
- Ephemeral shared playback control between independent Furumi peers.
- playback
- Transport-independent personal playback coordination.
- similarity
- Shared wire contract for federated music-similarity queries.
- similarity_
dht - Decentralized storage and lookup of signed similarity-LSH peer summaries.
- similarity_
lsh - Deterministic, model-neutral LSH routing for federated similarity search.
Structs§
- Byte
Stream - A raw bidirectional byte stream to a peer, running over a dedicated connection on an auxiliary ALPN.
- Byte
Stream Connection Stats - Snapshot of low-level transport statistics for a byte stream connection.
- DhtKey
- A 256-bit DHT key. Lives in the same key space as
crate::NodeId. - Distance
- XOR distance between two points of the 256-bit key space, compared as unsigned big-endian integers.
- Endpoint
Addr - Network-level addressing information for an iroh endpoint.
- Find
Node Request - Asks for the closest known nodes to
target. - Find
Node Response - Reply to
FindNodeRequest. - Find
Value Request - Asks for records stored under
key, or the closest nodes to it. - Hello
- Introduction sent right after a connection is established.
- ItemId
- Stable identifier of one item record.
- Item
Spec - Description of one library item to publish, as provided by the application.
- Library
Item - One item record as stored by its owner.
- Music
DhtConfig - Configuration for an
crate::MusicDhtService. - Music
DhtConfig Builder - Builder for
MusicDhtConfig. - Music
DhtEvent Receiver - Receiving side of the service event channel.
- Music
DhtService - A distributed item directory node.
- Network
Health Snapshot - Point-in-time health snapshot of a running network engine.
- Network
Id - Identifier of a distinct P2P network.
- Node
Contact - Everything needed to reach another DHT node.
- NodeId
- A 256-bit DHT node identifier, derived from the peer’s endpoint id.
- Peer
Exchange - A batch of known contacts, shared after
Hello. - Peer
Ticket - A shareable invitation to connect to a peer.
- Ping
Request - Liveness probe.
- Pong
Response - Reply to
PingRequest. - Publish
Stats - Statistics of one publish or republish operation.
- Recv
Stream noqtypes that are used in the public iroh API. A stream that can only be used to receive data- Rendezvous
Config - Configuration of the mainline-DHT rendezvous.
- Request
Envelope - Wraps a request payload with its correlation id.
- Request
Id - Correlates a response with its request.
- Response
Envelope - Wraps a response payload with the correlation id of its request.
- Routing
Table - A simplified Kademlia routing table: 256 k-buckets of up to
Kcontacts. - Search
Outcome - Result of a combined local + network search.
- Secret
Key - A secret key.
- Send
Stream noqtypes that are used in the public iroh API. A stream that can only be used to send data- Store
Batch Request - Asks the receiver to store replicas of several records at once.
- Store
Batch Response - Reply to
StoreBatchRequest. - Store
Record Request - One record replica to store, always carried inside a
StoreBatchRequest. - Stored
Record - A replicated DHT entry: an item plus replication metadata.
- Stream
Acceptor - Receiver of incoming
ByteStreams for one declared stream protocol. - Sync
Stats - Result of one
MusicDhtService::sync_librarycall.
Enums§
- Connection
Path Kind - The currently selected transport path kind for a byte stream connection.
- Find
Value Response - Reply to
FindValueRequest. - Item
Kind - Kind of a library item.
- Music
DhtError - All errors that can be returned by the public API of this library.
- Music
DhtEvent - Events delivered to the application.
- Music
DhtMessage - Every message exchanged between music-dht peers.
- Network
Health State - Coarse health state of a running network engine.
- Store
Decision - Outcome of comparing an incoming record with the stored one.
Constants§
- ACTIVE_
RECORD_ TTL - Maximum lifetime of an active DHT record. Long enough that replicas — and the share links resolved through them — survive the owner going offline for a workday; the owner refreshes the expiry on every republish.
- ALPHA
- Lookup parallelism: how many candidates are queried concurrently.
- DEFAULT_
EXPIRE_ INTERVAL - Default interval between expired-record sweeps.
- DEFAULT_
LOOKUP_ TIMEOUT - Default timeout of a whole iterative lookup.
- DEFAULT_
REPUBLISH_ INTERVAL - Default interval between republish rounds.
- DEFAULT_
REQUEST_ TIMEOUT - Default timeout of a single DHT request.
- DEFAULT_
TRANSPORT_ TIMEOUT - Default timeout for transport operations (dialing, handshakes, sends).
- DHT_
PROTOCOL_ VERSION - Version of the music-dht protocol. Batch limits are enforced by the receiver, so raising them is a version bump too.
- K
- Replication factor: how many closest nodes store a record and how many contacts a single response may carry.
- MAX_
ARTISTS_ PER_ ITEM - Maximum number of artist names carried by one release/track record across main and featured roles.
- MAX_
CONTENT_ ID_ BYTES - Maximum canonical content id length in UTF-8 bytes.
- MAX_
ITEM_ NAME_ BYTES - Maximum item name length in UTF-8 bytes.
- MAX_
LOOKUP_ REQUESTS - Hard budget of requests a single iterative lookup may send.
- MAX_
PEER_ EXCHANGE_ CONTACTS - Maximum number of contacts in a single
PeerExchange. - MAX_
PENDING_ REQUESTS - Maximum number of simultaneously pending requests.
- MAX_
RECORDS_ PER_ BATCH - Maximum number of entries in a single
StoreBatchRequest. Sized so a batch of typical records nearly fills the byte budget: replicating a 20k-track library takes hundreds of requests, not tens of thousands. - MAX_
RECORDS_ PER_ RESPONSE - Maximum number of records in a single
FindValueResponse. - MAX_
TOKENS_ PER_ ITEM - Maximum number of tokens a single item name may produce.
- SCHEMA_
NAME - Fixed compatibility identifier of the music-dht protocol; peers with a
different schema are rejected by
federation-netduring the handshake. - TOMBSTONE_
TTL - Maximum lifetime of a tombstone. Must exceed
ACTIVE_RECORD_TTLso a stale active replica can never outlive the tombstone that deletes it.
Traits§
- Music
DhtStorage - Persistence backend used by
crate::MusicDhtServicefor local items, replicated DHT records and learned peer contacts.
Functions§
- decide_
store - Decides what to do with an incoming record given the stored state
(revision, deleted, expires_at_ms)for the same(key, artist_id, owner). - distance
- Computes the XOR distance between two 256-bit values.
- key_
distance - Convenience: distance from a node to a DHT key.
- normalize_
content_ id - Returns a canonical content id accepted by this DHT version.
- normalize_
name - Normalizes an item name for indexing and comparison.
- tokenize
- Splits a normalized name into search tokens.
Type Aliases§
- Endpoint
Id - The identifier for an endpoint in the (iroh) network.
- PeerId
- Identifier of the peer that owns a record (its
federation-netendpoint). - Result
- Convenient result alias used across the library.