Crate distributed_topic_tracker

Crate distributed_topic_tracker 

Source
Expand description

§distributed-topic-tracker

Crates.io Docs.rs

Decentralized auto-bootstrapping for iroh-gossip topics using the mainline BitTorrent DHT.

§Quick Start

Add dependencies to Cargo.toml:

[dependencies]
anyhow = "1"
tokio = "1"
rand = "0.9"
iroh = "*"
iroh-gossip = "*"

distributed-topic-tracker = "0.2"

Basic iroh-gossip integration:

use anyhow::Result;
use iroh::{Endpoint, SecretKey};
use iroh_gossip::net::Gossip;
use ed25519_dalek::SigningKey;

// Imports from distributed-topic-tracker
use distributed_topic_tracker::{TopicId, AutoDiscoveryGossip, RecordPublisher};

#[tokio::main]
async fn main() -> Result<()> {
    // Generate a new random secret key
    let secret_key = SecretKey::generate(&mut rand::rng());
    let signing_key = SigningKey::from_bytes(&secret_key.to_bytes());

    // Set up endpoint with discovery enabled
    let endpoint = Endpoint::builder()
        .secret_key(secret_key.clone())
        .bind()
        .await?;

    // Initialize gossip
    let gossip = Gossip::builder().spawn(endpoint.clone());

    // Set up protocol router
    let _router = iroh::protocol::Router::builder(endpoint.clone())
        .accept(iroh_gossip::ALPN, gossip.clone())
        .spawn();

    // Distributed Topic Tracker
    let topic_id = TopicId::new("my-iroh-gossip-topic".to_string());
    let initial_secret = b"my-initial-secret".to_vec();
    let record_publisher = RecordPublisher::new(
        topic_id.clone(),
        signing_key.verifying_key(),
        signing_key.clone(),
        None,
        initial_secret,
    );

    // Use new `subscribe_and_join_with_auto_discovery` on Gossip
    let topic = gossip
        .subscribe_and_join_with_auto_discovery(record_publisher)
        .await?;

    println!("[joined topic]");

    // Work with the topic (GossipSender/Receiver are clonable)
    let (_gossip_sender, _gossip_receiver) = topic.split().await?;
    
    Ok(())
}

§Protocol

§Features

  • Decentralized bootstrap for iroh-gossip topics
  • Ed25519 signing and HPKE shared-secret encryption
  • DHT rate limiting (per-minute record caps)
  • Resilient bootstrap with retries and jitter
  • Background publisher with bubble detection and peer merging

§Testing

§Unit Tests

Run core component tests:

cargo test

§End-to-End Tests

Verify peer discovery across Docker containers:

# Requires Docker and Docker Compose
./test-e2e.sh

The E2E test confirms multiple nodes discover each other via DHT and join the same gossip topic.

§Roadmap

  • Finalize crate name and publish to crates.io
  • Tests and CI
  • Add more examples
  • Optimize configuration
  • Major refactor
  • Make iroh-gossip integration a feature (repurposed for rustpatcher)
  • API docs

§License

Licensed under Apache-2.0 or MIT you choose.

§Contributing

  • Test and provide feedback: Issue #5
  • PRs, issues, and reports welcome.

Contributions are dual-licensed as above unless stated otherwise.

Structs§

Bootstrap
Manages the peer discovery and joining process.
BubbleMerge
Detects and merges small isolated peer groups (bubbles).
DefaultSecretRotation
Default implementation: SHA512-based KDF.
Dht
DHT client wrapper with actor-based concurrency.
EncryptedRecord
DHT record encrypted with HPKE.
GossipReceiver
Gossip receiver that collects incoming messages and neighbor info.
GossipRecordContent
Record content for peer discovery.
GossipSender
Gossip sender that broadcasts messages to peers.
MessageOverlapMerge
Detects network partitions by comparing message hashes across DHT records.
Publisher
Periodically publishes node state to DHT for peer discovery.
Record
A signed DHT record containing peer discovery information.
RecordPublisher
Publisher for creating and distributing signed DHT records.
RecordTopic
Topic identifier derived from a string via SHA512 hashing.
RotationHandle
Wrapper for custom or default secret rotation implementations.
Topic
Handle to a joined gossip topic with auto-discovery.
TopicId
Topic identifier derived from a string via SHA512 hashing.

Constants§

MAX_BOOTSTRAP_RECORDS
Maximum number of bootstrap records allowed per topic per time slot (minute).

Traits§

AutoDiscoveryGossip
Extension trait for iroh Gossip enabling auto-discovery.
SecretRotation
Trait for deriving time-rotated encryption keys.

Functions§

encryption_keypair
Derive Ed25519 key for HPKE encryption/decryption.
salt
Derive DHT salt for mutable record lookups.
signing_keypair
Derive Ed25519 signing key for DHT record authentication.
unix_minute
Get the current Unix minute timestamp, optionally offset.