torrentdht 0.1.1

DHT discovery with mTLS handshake handoff; optional BitTorrent when the host opts in
Documentation

torrentdht

Secure DHT discovery and mTLS 1.3 session handoff for host projects. After handshake, the host owns the connection and chooses the application protocol (BitTorrent is optional / opt-in).

Deploy scope: default mTLS + TCP. Also selectable: MSE + TCP and MSE + uTP (BEP 29 with windowing, SACK, and basic congestion control). mTLS + uTP is rejected (TLS is TCP-only).

Intended use: proprietary / private-network discovery and connection when member IPs are dynamic or otherwise not statically known. Last-known address, peer store, and DHT locate peers; mTLS establishes who they are before the host takes the socket. This is not a drop-in public Mainline BitTorrent client for untrusted strangers — MSE stacks are opt-in interop, not a substitute for mTLS identity on an open swarm.

Primary flow: last-known IP or DHT → TCP → mTLS 1.3 → host-defined handshake approval → sealed credential exchange → Handoff. On signal_broken, auto-reconnect (last IP, then DHT). BitTorrent swarm reconnect uses prefer-addr-only (no network-DHT fallback) and consumes reconnect handoffs into the matching torrent slot.

Required TLS / network env (for Client::open and CLI download/seed/run/daemon):

Variable Purpose
TORRENTDHT_CA_CERT PEM CA that signed peer certs
TORRENTDHT_CERT This node’s PEM certificate
TORRENTDHT_KEY This node’s PEM private key
TORRENTDHT_NETWORK_ID Private network name (capability-bearing)
TORRENTDHT_HANDSHAKE_KEY Shared secret for offer approval + credential seal

Optional: TORRENTDHT_PEER_STORE, TORRENTDHT_LISTEN_ADDR (when set, overrides KDL listen-port — a warning is logged).

Validate without starting a session: torrentdht env-check.

Security notes

  • NETWORK_ID is capability-bearing: SHA-1(NETWORK_ID) is announced on the public DHT. Use a high-entropy value; anyone who knows it can enumerate member addresses.
  • HANDSHAKE_KEY is a real shared secret (offer approval + credential seal). Treat it like production credentials; rotate if exposed.
  • Peer certificates need a matching IP SAN for default IP-as-SNI dials, or use SecureNode::dial_hostname / Client::secure_dial_hostname for DNS SAN certs. Wrong SANs fail dials on churny IPs.
  • Default env policy is network-wide: an approved peer can join any torrent loaded on this Client (not per-torrent ACL).
  • Handoff labels/credentials do not echo the handshake key.
  • Content/sidecar opens under download-dir refuse symlinks and hardlinks (nlink != 1).
  • Trust model + ingress zones: SECURITY.kdl. Remediation backlog after the 2026-07 review: REVIEW.kdl (closed). Historical inventory: security.rst (do not treat status flood as live).

BitTorrent architecture (opt-in)

                    ┌──────────────── Client ────────────────┐
                    │  DHT · mTLS SecureNode · peer id       │
                    │  many torrent slots (info-hash route)  │
                    └───────────┬──────────────────────────┘
                                │ info-hash routing
              ┌─────────────────┼─────────────────┐
              ▼                 ▼                 ▼
       ┌─────────────┐   ┌─────────────┐   ┌─────────────┐
       │  Session A  │   │  Session B  │   │  Session …  │
       │  (swarm)    │   │  (swarm)    │   │             │
       └──────┬──────┘   └──────┬──────┘   └─────────────┘
              │                 │
              ▼                 ▼
       {download-dir}/{info-hash}/   (isolated on-disk paths)

Each Session is one torrent swarm. A Client shares one DHT UDP socket and one mTLS accept loop; after handoff, BitTorrent wire is routed by info hash.

DHT namespaces: torrent info-hash discovers swarm peer addresses; network identity (SecurePolicy::network_dht_key) is used by SecureNode reconnect/discovery. Both share the same DHT node. Every peer address is dialed with mTLS before BitTorrent wire.

┌─────────────┐     ┌──────────────┐     ┌─────────────┐
│   Session   │────▶│ mTLS handoff │────▶│ Wire Proto  │
└──────┬──────┘     └──────────────┘     │ BEP 9/10    │
       │                                  └─────────────┘
       ├──────────────────┬──────────────────┐
       ▼                  ▼                  ▼
┌─────────────┐    ┌──────────────┐   ┌─────────────┐
│  DHT Node   │    │   Trackers   │   │ Staged Store│
│  (UDP/Kad)  │    │ HTTP/S/UDP   │   │  RAM → Disk │
└─────────────┘    └──────────────┘   └─────────────┘
  • Tokio — async UDP/TCP, session loop, background disk flush tasks
  • Rayon — parallel piece hash verification and batch flush work
  • Staged store — block-level RAM cache with a dedicated flush pipeline to disk
  • Hybrid discovery — merges tracker announces (including announce-list) with DHT get_peers
  • Magnets — v1 btih and/or v2 btmh (multihash 1220…). Hybrid torrents emit both; pure-v2 emits btmh only. Peers that speak BEP 9 after mTLS can fetch/serve ut_metadata.

Configuration

Settings live in KDL (config/default.kdl). Key options:

Key Default Description
listen-port 6881 TCP/UDP listen port (shared across torrents on one Client)
download-dir ./downloads Torrent content root; each torrent writes to {download-dir}/{info-hash}/
state-dir (under download-dir) Host ledger/state root: peers.kdl, dht-routing.kdl, {state-dir}/torrents/{hash}/ resume + integrity sidecars. Env: TORRENTDHT_STATE_DIR. API: Config::with_state_dir
dht-enabled true Bind and run Mainline DHT
peer-discovery "hybrid" hybrid, tracker, or dht
memory-cache-mb 256 RAM budget for staged blocks
flush-workers 2 Background disk writer tasks
block-size 16384 BitTorrent block size (16 KiB)
max-peers 55 Concurrent peer connections
max-upload-slots 8 Max peers unchoked for upload per swarm
upload-rotate-secs 30 Optimistic unchoke rotation interval (0 = off)
nat-enabled false Host opt-in: UPnP/NAT-PMP port mapping
swarm-security mtls mtls (TCP TLS) or mse (MSE/PE; works with tcp or utp)
swarm-transport tcp tcp or utp (BEP 29; requires swarm-security "mse")
utp-enabled false Legacy alias for swarm-transport "utp" (still requires mse)
pex-enabled false Host opt-in: peer exchange (BEP 11); forced off for private torrents
super-seed-enabled false Host opt-in: super-seeding (requires Fast extension)
web-seed-enabled false Host opt-in: fetch from torrent url-list (HTTP/HTTPS; outside mTLS)
web-seed-ftp-enabled false Host opt-in: allow ftp:// web seeds (requires web-seed-enabled)
seed-require-verified true Only advertise/serve hash-verified pieces when seeding
rpc-enabled false Host opt-in: line RPC control plane
rpc-listen — Bind when RPC enabled (default expect 127.0.0.1:…)
rpc-auth-token — Required when RPC enabled (or env TORRENTDHT_RPC_TOKEN)
rpc-allow-non-loopback false Permit non-loopback RPC binds
rpc-torrent-root download-dir Jail for add_torrent paths
listen-dual-stack true Bind TCP/UDP on dual-stack IPv6 socket when supported (Unix)
end-game-threshold 4 Remaining pieces before end-game mode (0 = off)
dht-persist true Save DHT routing table + stable local NodeId to {download-dir}/.torrentdht/dht-routing.kdl (sealed)
dht-peer-announce-ttl-secs 1800 Expire announced peers from the local DHT peer store after this many seconds
dht-bootstrap (public routers) Optional { node "host" port } list; empty uses Mainline defaults
download-rate-kbps 0 Download cap in KiB/s (0 = unlimited)
upload-rate-kbps 0 Upload cap in KiB/s (0 = unlimited)
announce-ip — Tracker &ip= override (public IPv4)
announce-ipv6 — Tracker &ipv6= override (public IPv6)
tracker-block-private true Refuse private/link-local tracker & peer dials

Secure timeouts (secure { ... })

Key Default Description
connect-timeout-secs 5 TCP connect deadline for mTLS dials
tls-timeout-secs 10 TLS handshake deadline
handshake-timeout-secs 15 Host SecurePolicy handshake deadline

Selective download (download-files { ... })

When patterns are listed, only matching files are fetched (exact basename / path match — not substring ends_with):

download-files {
    file "docs/readme.txt"
    file "video.mkv"
}

CLI: torrentdht download --files readme.txt --files video.mkv …

Completion is wanted-set complete (not necessarily every piece in the torrent).

NAT (nat { ... }, when nat-enabled #true)

Key Default Description
map-tcp true Forward TCP listen-port
map-udp true Forward UDP listen-port (DHT)
upnp true Try UPnP IGD
natpmp true Try NAT-PMP

Mappings are removed when the Client is dropped.

Resilience (resilience { ... })

Key Default Description
heal-interval-secs 15 How often the self-heal watchdog runs
stale-peer-secs 45 Trigger heal when no active peers for this long
stalled-download-secs 120 Trigger heal when no piece completes for this long
peer-ban-secs 300 Temporary ban after repeated peer failures
checkpoint-interval-secs 30 How often to persist resume.kdl
max-peer-failures 5 Failures before a peer is temporarily banned
run-retry-initial-secs 3 Initial backoff for Session::run restart loop
run-retry-max-secs 60 Maximum backoff for Session::run restart loop

Integrity (integrity { ... })

Key Default Description
verify-blocks true CRC32c verify on flush and disk read
ecc true Reed-Solomon parity sidecars per piece
ecc-data-shards 4 RS data shards
ecc-parity-shards 1 RS parity shards (recover 1 missing shard)
encrypt-sidecars true ChaCha20-Poly1305 AEAD for .torrentdht/ metadata
sign-torrents false Sign .torrent info dict with Ed25519
create-image true Build ECC manifest + parity sidecars on create
require-signed-torrents false (default) / true (commercial) Reject unsigned .torrent files on load
trusted-signing-key — Hex Ed25519 public key(s); required when require-signed-torrents
encryption-key-file — Optional 32-byte key file
passphrase — Optional; handshake / access secret is preferred when both set

Torrent payload files stay plaintext (required by the BitTorrent wire protocol). Sidecar manifest and parity data under {download-dir}/{info-hash}/.torrentdht/ are authenticated (AEAD when encrypt on; HMAC when encrypt off) and require a configured access secret / handshake key.

Verify content

After create (with create-image #true), sidecars land under {download-dir}/{info-hash}/.torrentdht/. Validate with:

cargo run -- verify release.torrent --content ./original-input
# or, when content matches the download layout:
cargo run -- verify release.torrent

verify and magnet honor require-signed-torrents / trusted keys from config.

Build

cargo build --release
cargo test

Usage

# Create a .torrent from a file or directory
cargo run -- create --announce "http://tracker.example/announce" -o release.torrent ./content

# Sign with Ed25519 (32-byte seed file)
cargo run -- create --sign-key ~/.torrentdht/signing.key ./content

# Print magnet link (hex + base32 btih, all trackers) — v1 only
cargo run -- magnet release.torrent

# Download from .torrent (hybrid tracker + DHT by default)
cargo run -- download release.torrent

# Download from magnet URI (ut_metadata after mTLS to peers)
cargo run -- download 'magnet:?xt=urn:btih:...'

# Tracker-only or DHT-only discovery
cargo run -- download --tracker-only release.torrent
cargo run -- download --dht-only release.torrent

# Static peer (useful for LAN or testing; bypasses DialGate)
cargo run -- download --peer 127.0.0.1:6882 release.torrent

# Seed a completed download
cargo run -- seed release.torrent

# Download multiple unrelated torrents on one Client (shared DHT + listen port)
cargo run -- run -c config/default.kdl a.torrent b.torrent 'magnet:?xt=urn:btih:...'

# Show resume state for everything under download-dir
cargo run -- status -c config/default.kdl

# Long-running secure client (enable RPC in config when needed)
cargo run -- daemon -c config/default.kdl
# With RPC enabled: first line `auth <token>`, then status | progress <hash> | add_torrent <path> | add_magnet <uri> | stop <hash> | remove <hash> | ping
# `status` returns live sessions (content/state/checkpoint paths + bytes) and an `on_disk` resume scan.

# Create hybrid torrent (v1 SHA1 pieces + v2 file tree + piece layers; multi-file uses BEP 47 pads)
cargo run -- create --v2 ./content
# Pure v2 (btmh magnets only)
cargo run -- create --v2-only ./content

# Custom config
cargo run -- download -c config/default.kdl release.torrent

Note: The legacy two-positional form torrentdht <config.kdl> <torrent> was removed. Use torrentdht download -c <config.kdl> <torrent|magnet> (or seed / run / daemon).

Peer discovery

Mode Config / flag Trackers DHT
Hybrid (default) peer-discovery "hybrid" yes yes
Tracker only --tracker-only or "tracker" yes no
DHT only --dht-only or "dht" no yes

Trackers support http://, https:// (rustls + webpki-roots), and udp:// (BEP 15). Multi-tracker .torrent files use announce-list when present. Redirects are not followed; private addresses are blocked when tracker-block-private is on.

Multi-torrent Client

Use Config::for_commercial_host(dir, port) (or config/commercial.kdl) when embedding. Set content and ledger roots explicitly for the parent app:

let cfg = Config::for_commercial_host("./data/downloads", 6881)
    .with_state_dir("./data/torrentdht-state")
    .with_trusted_signing_keys(vec![/* 32-byte Ed25519 public keys */])
    .with_loopback_rpc("127.0.0.1:7600", "a-commercial-grade-rpc-token-32b");

Commercial code defaults require signed torrents — call .with_trusted_signing_keys(...) before open (empty allowlist fails closed). The sample config/commercial.kdl keeps require-signed-torrents #false until keys are supplied (documented in-file).

let cfg = Config::for_commercial_host("./data", 6881)
    .with_trusted_signing_keys(vec![/* 32-byte Ed25519 public keys */])
    .with_loopback_rpc("127.0.0.1:7600", "a-commercial-grade-rpc-token-32b");
let client = Client::open_secure_with_handshake(cfg, policy, tls, handshake_key).await?;

Optional features (RPC, PEX, webseed/FTP, NAT) stay off until the host enables them. RPC requires a ≥32-char token and loopback bind by default. swarm-transport "utp" requires swarm-security "mse".

Session::from_torrent / from_magnet create a dedicated mTLS Client (single-torrent convenience). Multi-torrent hosts should use one shared Client and add_torrent / add_magnet. Prefer session.run().await for durable download (resume from disk + checkpoint). Host UIs can observe progress via Session::subscribe_progress / current_progress, and mTLS handoffs via Client::subscribe_handoff_notices (fan-out; does not consume the TLS transport). Use Client::stop_torrent / remove_torrent (also via RPC stop / remove) for cooperative cancel.

Publishing

Dual-licensed MIT OR Apache-2.0 (LICENSE-MIT, LICENSE-APACHE). Package metadata targets crates.io; dry-run with cargo publish --dry-run before a release.

Tests

Unit tests plus real localhost end-to-end tests over mTLS (no mocks):

  • Single seed → leech download (mTLS)
  • One seed Client serving two torrents on one port → one leech Client downloading both
  • Inbound peer routing on the shared secure listener
  • Resume / sealed resume / DHT persist / web seed helper / hybrid --v2 create
cargo test
cargo clippy --all-targets -- -D warnings

Optional internet smoke: cargo test --test smoke_dht -- --ignored (also weekly via .github/workflows/smoke-dht.yml).