dig-peer-selector 0.10.0

Self-optimizing peer-selection middleware for the DIG Node — the pure decision + learning layer between dig-download and the discovery layers (dig-dht / dig-gossip / dig-nat). Maintains a dynamic peer registry, learns each peer's throughput/RTT/reliability + saturation point + a relayed penalty from the REAL measured outcome of every transfer (measured-only, non-gameable), and ranks a candidate subset per content request to minimize P99 latency without thundering-herd. No user-facing knobs; re-uses the transport-verified peer_id = SHA-256(TLS SPKI DER).
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145

//! # dig-peer-selector — the self-optimizing peer-selection middleware of the DIG Node
//!
//! The selector is a pure **decision + learning** layer. It sits between [`dig-download`] (the
//! executor that fetches bytes over [`dig-nat`] mTLS) and the peer-discovery layers ([`dig-dht`],
//! [`dig-gossip`]/`dig-pex`, [`dig-nat`]), and answers one question — *"of these candidate peers,
//! which subset should serve this content, and in what order?"* — learning the answer from the
//! **real, measured outcome** of every transfer it influenced. It has **no user-facing
//! configuration**: every tradeoff (saturation point, relayed penalty, decay) is self-tuned from
//! observed data.
//!
//! This crate is the authoritative implementation of `SPEC.md` (version `1`). The SPEC is the
//! contract; this documentation summarizes it — read `SPEC.md` for the normative statements.
//!
//! ## The closed loop
//!
//! 1. The node calls [`dig_dht::DhtService::find_providers`] and maps the [`ProviderRecord`]s into
//!    [`Candidate`]s, then asks the selector to [`select`](PeerSelector::select) the best subset for a
//!    [`ContentRequest`].
//! 2. `dig-download` executes the multi-source byte-range transfer over `dig-nat` mTLS mux streams.
//! 3. Every per-range / per-request [`TransferOutcome`] streams back via
//!    [`record_outcome`](PeerSelector::record_outcome), updating the models **in real time** so the
//!    next `select` — and a mid-transfer [`rebalance`](PeerSelector::rebalance) — is smarter.
//!
//! ## What is learned (measured-only, non-gameable)
//!
//! A peer's quality is refined EXCLUSIVELY from measured outcomes (SPEC §3, §9.2): there is no input
//! path by which a peer raises its own score, and **observed capacity always overrides advertised**
//! (SPEC §9.3). Throughput/RTT are recency-weighted estimators whose decay is derived from each
//! peer's observed volatility — no baked constant (SPEC §3.2, §4.3). The scorer learns a per-class
//! **saturation point** (anti-thundering-herd, SPEC §4.1) and an adaptive **relayed penalty**
//! (SPEC §4.2), and orients toward **minimizing P99 request latency** (SPEC §4.4).
//!
//! ## Boundaries (what the selector is NOT)
//!
//! It never queries the DHT, runs the gossip pool, opens a socket/TLS session/mux stream, or
//! fetches/verifies/persists bytes — those belong to `dig-dht`, `dig-gossip`, `dig-nat`, and
//! `dig-download` respectively (SPEC §1.2). The selector only *reads their outputs* or *drives their
//! choices*. It re-uses the transport-verified [`PeerId`] (`= SHA-256(TLS SPKI DER)`) and the
//! `dig-dht` content/candidate types verbatim — it defines no parallel identity (SPEC §5, §11).
//!
//! ## Implementers' note — how `dig-node` embeds the selector (P3 integration, digstore)
//!
//! The selector is the **source-selection seam** between `dig-dht` and `dig-download`. `dig-node`
//! wires it as follows (the wiring lives in the node, NOT this crate — SPEC §6.1, §7.5):
//!
//! 1. **Construct** one [`PeerSelector`] per node: `PeerSelector::new(SelectorConfig::default())`.
//! 2. **Feed the registry** continuously:
//!    - subscribe to `dig_gossip::GossipHandle::subscribe_pool_events()` and forward each event —
//!      convert `dig_gossip::PoolEvent` → [`PoolEvent`] with a trivial 1:1 field map (identical
//!      shapes; see [`pool_event`]) — into [`on_pool_event`](PeerSelector::on_pool_event);
//!    - on each established `dig-nat` connection, call
//!      [`on_connection_class`](PeerSelector::on_connection_class) with its
//!      [`dig_nat::TraversalKind`];
//!    - optionally seed from a `connected_pool_peers()` snapshot at startup via
//!      [`upsert_candidate`](PeerSelector::upsert_candidate).
//! 3. **Per content want**, call `find_providers(&content)`, map each [`ProviderRecord`] into a
//!    [`Candidate`] (via [`Candidate::from_provider_record`]), and call
//!    [`select`](PeerSelector::select). Use each [`SelectedPeer`]'s `max_concurrency` as the per-peer
//!    concurrent-range cap in `dig-download` (replacing its built-in `pick_source` heuristic).
//! 4. **Translate the `dig-download` `DownloadEvent` stream into outcomes as it flows** (SPEC §6.2):
//!    map `RangeCompleted` → a `Range` [`TransferOutcome`] with `result: Success`; `RangeFailed` →
//!    `Failure { reason }` (an integrity/verify failure → [`FailureReason::VerificationFailed`]);
//!    optionally `Completed` → a `Request` outcome for whole-request P99 learning. Call
//!    [`record_outcome`](PeerSelector::record_outcome) for each. A `Paused` is NOT a failure — do not
//!    record one (SPEC §6.4).
//! 5. **On a dropped source / relocate**, call [`rebalance`](PeerSelector::rebalance) with the still-
//!    active peers and the still-needed ranges to get a replacement subset (SPEC §5.5). On resume,
//!    `select`/`rebalance` only the ranges NOT in `DownloadState::done_ranges` (SPEC §6.4).
//!
//! ## DigPeer hand-off — the connect step, handed back
//!
//! The selector returns *ranked peer identities*, and it still opens no socket. It does, however,
//! hand back everything needed to open one: [`dial_plan`](PeerSelector::dial_plan) pairs each
//! [`SelectedPeer`] with the [`PeerTarget`] that reaches it, built from the addresses the registry
//! learned for that peer. Pass the target straight to `dig_peer::DigPeer::connect` (or
//! `connect_with_runtime` to compose the full NAT ladder) — the target carries the `peer_id`, which
//! the handshake PINS, so a different CA-valid peer cannot answer in place of the chosen one (#1283).
//!
//! Consumers MUST NOT re-derive this mapping from their own candidate lists: a second implementation
//! of the addressing is how two crates silently drift apart on which address they dial. For the same
//! reason the candidate ordering itself is inherited from [`dig_dht::dial_candidates`] — kind filter,
//! `host:port` dedup, bound and reserved IPv4 fallback slot included — rather than restated here.
//!
//! ```ignore
//! let selection = selector.select(&request, &candidates);
//! for (chosen, target) in selector.dial_plan(&selection, "mainnet") {
//!     let peer = dig_peer::DigPeer::connect(&target, &node_cert, &nat_config).await?;
//!     // ... fetch up to `chosen.max_concurrency` ranges from `peer`
//! }
//! ```
//!
//! Because `TransferOutcome`/`Selection` are defined here structurally, this crate does NOT depend on
//! `dig-download` — avoiding a dependency cycle; the event→outcome mapping lives in the node adapter
//! (SPEC §11).
//!
//! [`dig-download`]: https://github.com/DIG-Network/dig-download
//! [`dig-dht`]: https://github.com/DIG-Network/dig-dht
//! [`dig-nat`]: https://github.com/DIG-Network/dig-nat
//! [`dig-gossip`]: https://github.com/DIG-Network/dig-gossip
//! [`ProviderRecord`]: dig_dht::ProviderRecord

#![forbid(unsafe_code)]
#![warn(missing_docs)]

pub mod config;
pub mod dial;
pub mod engine;
pub mod observe;
pub mod pool_event;
pub mod quality;
pub mod registry;
pub mod scoring;
pub mod types;

// ---- The frozen public surface (SPEC §11) --------------------------------------------------------

pub use config::{ClockSource, SelectorConfig, DEFAULT_REGISTRY_CAPACITY};
pub use dial::peer_target;
pub use engine::PeerSelector;
pub use observe::{peer_id_hex, PeerSnapshot, SelectorSnapshot};
pub use pool_event::{PoolEvent, PoolRemovalReason};
pub use quality::{Estimate, PeerQuality, Reliability};
pub use registry::{FeedResult, PeerEntry, DISPATCH_TTL_SECS};
pub use scoring::{PeerClass, RelayModel, SaturationModel};
pub use types::{
    Candidate, ContentRequest, FailureReason, OutcomeKind, OutcomeResult, Provenance,
    RangePlanDelta, SelectedPeer, Selection, TransferOutcome,
};

// ---- Re-used from the sibling crates (NOT redefined — SPEC §5, §11) ------------------------------

/// The bound on how many candidate addresses one dial target carries, re-used from `dig-dht` — the
/// crate that owns the dial-candidate ordering this crate inherits (SPEC §5.8). Never re-declared
/// here: a copied constant is a rival implementation waiting to drift.
pub use dig_dht::MAX_DIAL_CANDIDATES;
/// The candidate/content types re-used from `dig-dht` (SPEC §7.1): what is fetched + how a provider
/// is addressed.
pub use dig_dht::{AddressKind, CandidateAddr, ContentId, ProviderRecord};
/// The transport-verified peer identity (`peer_id = SHA-256(TLS SPKI DER)`), re-used from `dig-nat`.
pub use dig_nat::PeerId;
/// The `dig-nat` connection-class ladder the selector reads observationally (SPEC §7.3).
pub use dig_nat::TraversalKind;
/// The dial target handed back for each selected peer, re-used from `dig-peer` (SPEC §5.8) — the
/// `peer_id`-pinned input to `dig_peer::DigPeer::connect`.
pub use dig_peer::PeerTarget;