pub struct PendingReports { /* private fields */ }Expand description
Coalescing per-PV pending-timestamp map shared between every shard worker (incl. their spawn_blocking late-success closures) and the single global flush owner.
Why a shared map instead of an mpsc channel? With a channel,
a slow flush owner causes the buffer to fill and try_send
drops happen — for a PV that goes silent right after a dropped
report, the registry’s last_event is permanently
under-committed. With a coalescing map keyed by PV, every
shard call is an entry().and_modify().or_insert() that
always succeeds; concurrent updates from different shards
never lose data because the map is bounded by PV count
(typical: thousands), not by sample rate (typical: 100k/s).
Coalescing semantics. A successful append for (pv, ts)
upserts the entry to max(current, ts). A stale late-success
report (e.g. from a spawn_blocking task that finished after
shard already wrote a newer sample) does NOT clobber a newer
committed value.
Implementations§
Source§impl PendingReports
impl PendingReports
pub fn new() -> Self
Sourcepub fn report(&self, pv: &str, ts: SystemTime)
pub fn report(&self, pv: &str, ts: SystemTime)
Coalescing report. If pv already has a newer timestamp,
no-op. Always succeeds — this is the channel-replacement
invariant that makes silent-PV under-commit impossible.
Sourcepub fn snapshot(&self) -> HashMap<String, SystemTime>
pub fn snapshot(&self) -> HashMap<String, SystemTime>
Snapshot the entire map for a flush cycle. Returns a owned HashMap; the shared map stays intact so concurrent shards can keep reporting during the flush.
Sourcepub fn remove_committed(&self, committed: &HashMap<String, SystemTime>)
pub fn remove_committed(&self, committed: &HashMap<String, SystemTime>)
Remove entries whose CURRENT value still equals what we committed. If a concurrent shard advanced an entry to a newer ts after the snapshot, leave it alone — the next flush will pick it up.
Sourcepub fn remove_failed(&self, failed: &[String])
pub fn remove_failed(&self, failed: &[String])
Unconditionally remove every failed PV’s entry from the pending map, regardless of its current timestamp.
Why unconditional. The storage’s loss queue carries
only PV names — it cannot tell us whether a concurrent
shard’s post-snapshot report was for bytes that ARE on
disk (e.g., a fresh writer opened after eviction) versus
bytes that are also gone. The matching remove_committed
path can leave a post-snapshot entry behind, and the next
clean flush would commit it — turning a “PV’s bytes were
lost” report into a last_event lie.
The trade-off: a brand-new sample whose bytes truly DID
reach disk after the loss event gets dropped from this
commit cycle. Future samples re-populate pending and the
registry catches up. Under-commit is the safe direction;
over-commit is never acceptable.