Skip to main content

Crate music_dht

Crate music_dht 

Source
Expand description

§music-dht

A distributed music library directory on top of federation_net: peers publish their local library (artists, releases, tracks — names and small metadata, never files) into a Kademlia-style DHT and search each other’s libraries. Every running node is simultaneously a client, a DHT router and a storage node — there are no dedicated bootstrap, index or search servers.

§How it works

  • Every peer derives a stable 256-bit NodeId from its persistent federation-net endpoint id.
  • LibraryItem records are published under BLAKE3-derived DhtKeys: one exact key for the whole normalized name plus one key per token of the name, every main/featured artist name and the track’s release title (so a track is found by its artists).
  • Track records may also carry a compact b3:<hex> content id and are then published under a content key, so applications can find another peer with the exact same audio bytes.
  • similarity defines the bounded, model-neutral stream contract used by clients that independently generate compatible music embeddings.
  • similarity_lsh turns those embeddings into deterministic compact routing summaries, while similarity_dht stores and discovers signed peer summaries without owning model inference or an embedding database.
  • Records are replicated to the K nodes whose ids are XOR-closest to each key. Publishers pick the targets from their routing table and send batched store requests (one pipeline per peer), so even a large library republishes in seconds; searches discover records with an iterative Kademlia-style lookup (never a broadcast).
  • Peers learn about each other through a Hello/PeerExchange gossip that runs automatically on every new connection; connections to further nodes are opened on demand from stored tickets.
  • Deletions propagate as tombstones that win over active records of the same or lower revision; replicas expire by TTL and owners republish periodically.

§Example

use music_dht::{ItemKind, ItemSpec, MusicDhtConfig, MusicDhtService};
use federation_net::{NetworkId, RendezvousConfig};

let config = MusicDhtConfig::builder()
    .data_dir("./peer-a")
    .network_id(NetworkId::from_name("my-music-network"))
    // Find other peers knowing only the network id.
    .rendezvous(RendezvousConfig::default())
    .build()?;
let (service, mut events) = MusicDhtService::start(config).await?;

// Publish (and keep in sync) the local library.
service
    .sync_library(vec![ItemSpec {
        local_key: "track:42".into(),
        kind: ItemKind::Track,
        name: "Teardrop".into(),
        artist_names: vec!["Massive Attack".into()],
        featured_artist_names: vec![],
        year: Some(1998),
        release_type: None,
        release_title: Some("Mezzanine".into()),
        track_number: Some(10),
        disc_number: Some(1),
        duration_seconds: Some(330.0),
        content_id: None,
    }])
    .await?;

// Search the whole network.
let outcome = service.search_network("massive attack").await?;
for item in &outcome.network_results {
    println!("{} {} owned by {}", item.kind, item.name, item.owner);
}

Re-exports§

pub use similarity_lsh::SimilarityRouteEntry;

Modules§

capabilities
Runtime publication of application and protocol versions.
catalog
Shared wire shapes for Furumi catalog streams.
device_sync
Wire protocol shared by furumi personal-device synchronization clients.
jam
Ephemeral shared playback control between independent Furumi peers.
playback
Transport-independent personal playback coordination.
similarity
Shared wire contract for federated music-similarity queries.
similarity_dht
Decentralized storage and lookup of signed similarity-LSH peer summaries.
similarity_lsh
Deterministic, model-neutral LSH routing for federated similarity search.

Structs§

ByteStream
A raw bidirectional byte stream to a peer, running over a dedicated connection on an auxiliary ALPN.
ByteStreamConnectionStats
Snapshot of low-level transport statistics for a byte stream connection.
DhtKey
A 256-bit DHT key. Lives in the same key space as crate::NodeId.
Distance
XOR distance between two points of the 256-bit key space, compared as unsigned big-endian integers.
EndpointAddr
Network-level addressing information for an iroh endpoint.
FindNodeRequest
Asks for the closest known nodes to target.
FindNodeResponse
Reply to FindNodeRequest.
FindValueRequest
Asks for records stored under key, or the closest nodes to it.
Hello
Introduction sent right after a connection is established.
ItemId
Stable identifier of one item record.
ItemSpec
Description of one library item to publish, as provided by the application.
LibraryItem
One item record as stored by its owner.
MusicDhtConfig
Configuration for an crate::MusicDhtService.
MusicDhtConfigBuilder
Builder for MusicDhtConfig.
MusicDhtEventReceiver
Receiving side of the service event channel.
MusicDhtService
A distributed item directory node.
NetworkHealthSnapshot
Point-in-time health snapshot of a running network engine.
NetworkId
Identifier of a distinct P2P network.
NodeContact
Everything needed to reach another DHT node.
NodeId
A 256-bit DHT node identifier, derived from the peer’s endpoint id.
PeerExchange
A batch of known contacts, shared after Hello.
PeerTicket
A shareable invitation to connect to a peer.
PingRequest
Liveness probe.
PongResponse
Reply to PingRequest.
PublishStats
Statistics of one publish or republish operation.
RecvStream
noq types that are used in the public iroh API. A stream that can only be used to receive data
RendezvousConfig
Configuration of the mainline-DHT rendezvous.
RequestEnvelope
Wraps a request payload with its correlation id.
RequestId
Correlates a response with its request.
ResponseEnvelope
Wraps a response payload with the correlation id of its request.
RoutingTable
A simplified Kademlia routing table: 256 k-buckets of up to K contacts.
SearchOutcome
Result of a combined local + network search.
SecretKey
A secret key.
SendStream
noq types that are used in the public iroh API. A stream that can only be used to send data
StoreBatchRequest
Asks the receiver to store replicas of several records at once.
StoreBatchResponse
Reply to StoreBatchRequest.
StoreRecordRequest
One record replica to store, always carried inside a StoreBatchRequest.
StoredRecord
A replicated DHT entry: an item plus replication metadata.
StreamAcceptor
Receiver of incoming ByteStreams for one declared stream protocol.
SyncStats
Result of one MusicDhtService::sync_library call.

Enums§

ConnectionPathKind
The currently selected transport path kind for a byte stream connection.
FindValueResponse
Reply to FindValueRequest.
ItemKind
Kind of a library item.
MusicDhtError
All errors that can be returned by the public API of this library.
MusicDhtEvent
Events delivered to the application.
MusicDhtMessage
Every message exchanged between music-dht peers.
NetworkHealthState
Coarse health state of a running network engine.
StoreDecision
Outcome of comparing an incoming record with the stored one.

Constants§

ACTIVE_RECORD_TTL
Maximum lifetime of an active DHT record. Long enough that replicas — and the share links resolved through them — survive the owner going offline for a workday; the owner refreshes the expiry on every republish.
ALPHA
Lookup parallelism: how many candidates are queried concurrently.
DEFAULT_EXPIRE_INTERVAL
Default interval between expired-record sweeps.
DEFAULT_LOOKUP_TIMEOUT
Default timeout of a whole iterative lookup.
DEFAULT_REPUBLISH_INTERVAL
Default interval between republish rounds.
DEFAULT_REQUEST_TIMEOUT
Default timeout of a single DHT request.
DEFAULT_TRANSPORT_TIMEOUT
Default timeout for transport operations (dialing, handshakes, sends).
DHT_PROTOCOL_VERSION
Version of the music-dht protocol. Batch limits are enforced by the receiver, so raising them is a version bump too.
K
Replication factor: how many closest nodes store a record and how many contacts a single response may carry.
MAX_ARTISTS_PER_ITEM
Maximum number of artist names carried by one release/track record across main and featured roles.
MAX_CONTENT_ID_BYTES
Maximum canonical content id length in UTF-8 bytes.
MAX_ITEM_NAME_BYTES
Maximum item name length in UTF-8 bytes.
MAX_LOOKUP_REQUESTS
Hard budget of requests a single iterative lookup may send.
MAX_PEER_EXCHANGE_CONTACTS
Maximum number of contacts in a single PeerExchange.
MAX_PENDING_REQUESTS
Maximum number of simultaneously pending requests.
MAX_RECORDS_PER_BATCH
Maximum number of entries in a single StoreBatchRequest. Sized so a batch of typical records nearly fills the byte budget: replicating a 20k-track library takes hundreds of requests, not tens of thousands.
MAX_RECORDS_PER_RESPONSE
Maximum number of records in a single FindValueResponse.
MAX_TOKENS_PER_ITEM
Maximum number of tokens a single item name may produce.
SCHEMA_NAME
Fixed compatibility identifier of the music-dht protocol; peers with a different schema are rejected by federation-net during the handshake.
TOMBSTONE_TTL
Maximum lifetime of a tombstone. Must exceed ACTIVE_RECORD_TTL so a stale active replica can never outlive the tombstone that deletes it.

Traits§

MusicDhtStorage
Persistence backend used by crate::MusicDhtService for local items, replicated DHT records and learned peer contacts.

Functions§

decide_store
Decides what to do with an incoming record given the stored state (revision, deleted, expires_at_ms) for the same (key, artist_id, owner).
distance
Computes the XOR distance between two 256-bit values.
key_distance
Convenience: distance from a node to a DHT key.
normalize_content_id
Returns a canonical content id accepted by this DHT version.
normalize_name
Normalizes an item name for indexing and comparison.
tokenize
Splits a normalized name into search tokens.

Type Aliases§

EndpointId
The identifier for an endpoint in the (iroh) network.
PeerId
Identifier of the peer that owns a record (its federation-net endpoint).
Result
Convenient result alias used across the library.