Crate dig_download 

dig-download — the node-side multi-source download orchestrator for the DIG Node peer network

dig-download answers “get me this content, fast and verified.” Given a ContentId (store / root / capsule / resource) it runs the normative L7 multi-source flow: locate the holders in the DHT, confirm them with dig.getAvailability, fan different byte ranges across different holders simultaneously (dig.fetchRange over dig-nat mux streams), verify each range independently against the capsule’s chain-anchored merkle root, rebalance around slow / dropped / bad sources, and reassemble the verified bytes in order into the node’s store — with pause + resume that never re-fetches an already-verified range. It is the node engine that supersedes the retired browser-side dig-download-utility.

The public surface

• Downloader — built once from injected dependencies, then downloaded against many content ids. Returns a DownloadHandle (progress event stream + pause / resume / cancel + join).
• Trait boundaries (the injection seams — real impls over dig-dht/dig-nat, or the in-memory [testkit]):
  • ProviderLocator — “which peers hold this?” (DhtProviderLocator over dig-dht).
  • RangeTransport — fetch a range / availability from a peer (NatRangeTransport over dig-nat).
  • Sink — where verified bytes land (FileSink stages to <target>.download.tmp and atomically finalizes; dig-node supplies a store-backed sink).
  • StateStore — persist per-range resume progress (InMemoryStateStore / FileStateStore).
  • Verifier / ProofVerifier — per-range + chain-anchored integrity (MerkleVerifier; dig-node injects the digstore proof verifier to bind to the on-chain root).
• onion — onion mode: a transfer carried back through the hops that carried the ask, as an OnionRangeTransport over an injected OnionChannel (the layered transport is dig-onion’s). Delivery changes; trust does not — an onion-delivered byte faces the same per-range and chain-anchored checks as a directly fetched one. Includes the byte-denominated StreamRelayConfig that bounds what a hop spends carrying someone else’s transfer, and which is OFF by default.
• gc — reap stale .download.tmp staging files, never a live/paused-resumable one (ActiveDownloads + TmpGc; run Downloader::gc on an interval like dig-dht’s provider gc()).

Integrity model (L7 §9)

Two checks, two moments. Per range, immediately: the returned bytes cover whole chunk(s) whose lengths match the resource’s chunk_lens, and the declared generation root matches — a truncated / mis-sized / wrong-generation source is caught the instant its range arrives and the range is re-fetched elsewhere. Whole resource, at completion: resource_leaf = SHA-256(concatenated chunk ciphertexts) is the leaf committed under the chain-anchored root (via an injected ProofVerifier). Whichever mix of peers served the ranges, they all verify against the same on-chain root.

Implementers’ note — wiring dig-download into dig-node

dig-node owns the runtime context the trait boundaries abstract, and constructs a Downloader from it:

1. Locator — build a dig_dht::DhtService (its dig-nat transport + bootstrap peers from the relay introducer / gossip pool), wrap it in DhtProviderLocator::new, Arc it.
2. Transport — build a NatRangeTransport::new from the node’s dig_nat::NodeCert (its CA-signed mTLS identity, minted by dig-tls’s NodeCert::load_or_generate) + dig_nat::NatConfig + network_id; it dials providers over the NAT-traversal ladder and runs dig.getAvailability / dig.fetchRange.
3. Verifier — MerkleVerifier::with_proof_verifier with the digstore merkle-proof verifier (the store crate owns the proof byte format) so the whole-resource check binds to the chain-anchored root. This is the ONLY production constructor: there is no fail-open default, so a node cannot accidentally run without the on-chain binding. (The explicitly-named, #[doc(hidden)] MerkleVerifier::insecure_structural_only enforces per-range + structural integrity only and is for tests / deliberate opt-in.)
4. Sink — per download, a FileSink::new(final_path) (stages to <final_path>.download.tmp, atomically renames on finalize), OR a digstore-backed Sink that writes the capsule/resource ciphertext into the store and finalizes on install.
5. State store — a FileStateStore under the download/cache dir (survives restarts).
6. Construct + drive — Downloader::new(locator, transport, verifier, state_store, config), then let handle = downloader.download(content_id, sink, opts); and drive it: handle.next_event() for progress, handle.pause()/resume()/cancel(), handle.join().await for the result. On startup and on an interval, call downloader.gc(download_dir, ttl) to reap abandoned staging files (downloader.active_downloads() protects live/paused ones).

A content-want handler thus becomes: derive the ContentId, pick a sink, download(...), and surface progress — the crate does discovery, multi-source fan-out, verification, retry, and resume.

Re-exports

pub use addr::candidate_socket;

pub use addr::dial_candidates;

pub use addr::AddrError;

pub use addr::MAX_DIAL_CANDIDATES;

pub use error::hex64_or_sentinel;

pub use error::sanitize_untrusted_text;

pub use error::DownloadError;

pub use error::VerifyError;

pub use error::MAX_ERROR_CONTEXT_CHARS;

pub use error::MAX_ERROR_REASON_CHARS;

pub use gc::ActiveDownloads;

pub use gc::GcConfig;

pub use gc::TmpGc;

pub use locate::DhtProviderLocator;

pub use locate::ProviderLocator;

pub use module::module_content_id;

pub use module::module_download_key;

pub use module::ModuleAnchor;

pub use module::ModuleAnchorVerifier;

pub use module::ModuleDownloadConfig;

pub use module::ModuleDownloader;

pub use module::ModuleReader;

pub use module::ModuleTransport;

pub use module::DEFAULT_MAX_MODULE_SIZE;

pub use module::MAX_DESCRIPTOR_ATTEMPTS;

pub use module::MAX_MODULE_CHUNK_COUNT;

pub use onion::decide_relay_stream;

pub use onion::HopPath;

pub use onion::HopPathError;

pub use onion::InboundStream;

pub use onion::OnionChannel;

pub use onion::OnionRangeTransport;

pub use onion::StreamRelayConfig;

pub use onion::StreamRelayDecision;

pub use onion::StreamRelayRefusal;

pub use onion::DEFAULT_MAX_BYTES_PER_STREAM;

pub use onion::DEFAULT_RELAY_BYTES_PER_WINDOW;

pub use onion::MAX_HOP_PATH;

pub use orchestrator::download_key;

pub use orchestrator::DownloadConfig;

pub use orchestrator::DownloadHandle;

pub use orchestrator::DownloadOptions;

pub use orchestrator::Downloader;

pub use orchestrator::DEFAULT_RANGE_TIMEOUT;

pub use orchestrator::DEFAULT_REFRESH_INTERVAL;

pub use plan::plan_ranges;

pub use plan::ChunkLayout;

pub use plan::Range;

pub use plan::RangeState;

pub use progress::DownloadEvent;

pub use progress::DownloadProgress;

pub use progress::DownloadState;

pub use progress::FileStateStore;

pub use progress::InMemoryStateStore;

pub use progress::StateStore;

pub use queue::DownloadQueue;

pub use queue::QueuedHandle;

pub use queue::DEFAULT_MAX_ACTIVE_DOWNLOADS;

pub use read_ladder::local_urls;

pub use read_ladder::override_source;

pub use read_ladder::resolve_node;

pub use read_ladder::CachedResolver;

pub use read_ladder::HealthProbe;

pub use read_ladder::LocalRung;

pub use read_ladder::OverrideInputs;

pub use read_ladder::OverrideSource;

pub use read_ladder::ResolvedNode;

pub use read_ladder::ResolvedTier;

pub use read_ladder::TransportMode;

pub use read_ladder::DEFAULT_PROBE_TIMEOUT;

pub use select::CandidateRef;

pub use select::NullSelector;

pub use select::RangeOutcome;

pub use select::RangeResult;

pub use select::SelectPlan;

pub use select::SelectRequest;

pub use select::SourceSelector;

pub use sink::staging_path_for;

pub use sink::FileSink;

pub use sink::InMemorySink;

pub use sink::Sink;

pub use sink::STATE_SUFFIX;

pub use sink::TMP_SUFFIX;

pub use source::assemble_range_stream;

pub use source::drain_trailer_bounded;

pub use source::FetchedRange;

pub use source::NatRangeTransport;

pub use source::RangeMeta;

pub use source::RangeTransport;

pub use source::SourceHealth;

pub use source::SourceTracker;

pub use throttle::FcfsRateLimiter;

pub use verify::MerkleVerifier;

pub use verify::ProofVerifier;

pub use verify::ResourceCommitment;

pub use verify::ResourceHasher;

pub use verify::Verifier;

Modules

addr

Provider-candidate address resolution — the ONE place a DHT CandidateAddr becomes a dialable SocketAddr, and the ONE place a candidate is rendered as text.

error

DownloadError — the crate’s top-level error, and VerifyError — why a fetched range or a reassembled resource failed integrity.

gc

Garbage-collection of stale .download.tmp staging files.

locate

ProviderLocator — “which peers hold this content?” — and the real dig-dht-backed locator.

module

ModuleDownloader — the whole-.dig-module peer pull (the reshare leg, #1576).

onion

Onion mode — carrying a transfer back through the same layered hops that carried the ask (#30).

orchestrator

Downloader + DownloadHandle — the public entry point and the concurrent scheduler that turns “get me this content” into verified bytes in the node’s store.

plan

Range planning: turn a resource’s chunk layout into the chunk-aligned byte ranges a download fans across providers, and track each range’s scheduling state.

progress

Progress reporting + resume state.

queue

DownloadQueue — a bounded, first-come-first-serve queue over a Downloader (#1435 req. 1): capsule downloads are QUEUED and scheduled a few at a time, not all fired at once.

read_ladder

resolve_node — the client→node connection-order ladder (CLAUDE.md §5.3, dig-node/SPEC.md §2.2).

select

SourceSelector — the selection seam: dig-download delegates “which of these candidate peers should serve this content, and in what order?” to an injected brain, and reports the real measured outcome of every range fetch back to it.

sink

Sink — where verified bytes land: the node’s store-write path.

source

RangeTransport — fetch one byte range (or an availability answer) from one provider — plus per-source health tracking and the real dig-nat-backed implementation.

throttle

FcfsRateLimiter — a first-come-first-serve token-bucket rate limiter for the outbound serve path (#1435 req. 2): when this node serves capsule bytes to requesting peers, it caps the rate so it never overwhelms a single peer or its own uplink.

verify

Per-range + whole-resource integrity — L7 §9 “per-range integrity”.

Structs

AvailabilityItem

One item in a dig.getAvailability batch — a resource key at store, root, or capsule/resource granularity (inferred from which fields are present, per the L7 spec §9): store_id only → has_store; + root → has_root (the capsule store_id:root); + retrieval_key → has_resource. Hashes are 64-hex.

AvailabilityResponse

The peer’s answer to an AvailabilityRequest: one AvailabilityAnswer per queried item, positionally aligned with the request’s items.

ModuleInfo

Result for dig.getModuleInfo — the transfer descriptor of a whole .dig module.

ProviderRecord

The DHT’s stored value: peer provider_peer_id holds the content whose key is content_key, reachable at addresses, until expires_at.

RangeRequest

A byte-range request (dig.fetchRange, L7 spec §9) written at the start of a range-scoped stream. Identifies a resource (store_id + retrieval_key [+ root]) or a whole capsule (capsule: true, identified by store_id [+ root]) and the [offset, offset+length) range.

Enums

ContentId

A DIG content identifier at store / root(capsule) / resource granularity — the key a provider record is stored under and a lookup asks for.

Constants

DEFAULT_LOCAL_NODE_PORT

A node’s default loopback read port (dig-node/SPEC.md §1.1, canonical 9778). Re-exported from the dig-constants SSOT (DIG_NODE_PORT). The default localhost port a client uses to reach the local DIG node.

DIG_LOCAL_HOST

The installed local node’s hosts-file registration (installer-managed). Re-exported from the dig-constants SSOT. The mDNS/local hostname the installed DIG node registers.

RPC_DIG_NET

The public gateway base URL — FINAL fallback tier, never the primary/hard-coded endpoint (CLAUDE.md §5.3). Re-exported from the dig-constants SSOT. The public DIG read gateway.