Struct MessageDeduplicator 

pub struct MessageDeduplicator { /* private fields */ }

Two-layer message deduplicator: Bloom filter + bounded LRU cache.

See module documentation for usage.

Implementations

impl MessageDeduplicator

pub fn new(config: DedupConfig) -> Self

Create a new deduplicator with the given configuration.

pub fn check_and_insert(&mut self, msg_id: &MsgId, now: u64) -> bool

Check whether msg_id has been seen before and, if not, record it.

Returns true if the message is a duplicate (was already seen), false if it is new (first occurrence).

now is the caller-supplied timestamp in milliseconds (e.g. from std::time::SystemTime or a monotonic clock).

pub fn is_duplicate_bloom(&self, msg_id: &MsgId) -> bool

Check the bloom filter alone (probabilistic — may have false positives).

Returns true if the bloom filter believes msg_id was seen before.

pub fn insert_bloom(&mut self, msg_id: &MsgId)

Set the bloom filter bits for msg_id.

pub fn bloom_bit_positions(msg_id: &MsgId, count: u32) -> Vec<usize>

Compute count bloom bit positions for msg_id using Kirsch-Mitzenmacher double hashing over FNV-1a.

Uses two independent FNV-1a 64-bit hashes (h1, h2) and derives positions as (h1 + i * h2) mod usize::MAX. The modulo against the actual bit-array size is applied by the caller.

pub fn make_msg_id(data: &[u8]) -> MsgId

Construct a MsgId from arbitrary data.

Produces a 32-byte fingerprint using four independent FNV-1a 64-bit hashes (seeded with 0, 1, 2, 3) concatenated into a [u8; 32]. This is deterministic: the same data always yields the same MsgId.

pub fn purge_expired(&mut self, now: u64) -> usize

Remove all cache entries whose first_seen timestamp is older than now - window_ms. Returns the number of entries purged.

Note: the bloom filter is not cleared by this call (bloom filters do not support deletion). Only the exact cache is pruned.

pub fn evict_oldest(&mut self) -> bool

Evict the oldest (least recently inserted) entry from the cache.

Returns true if an entry was evicted, false if the cache was empty.

pub fn seen_count(&self) -> usize

Return the number of unique entries currently in the exact cache.

pub fn stats(&self) -> &MsgDedupStats

Return a reference to the accumulated statistics.

pub fn reset(&mut self)

Reset the deduplicator to a clean initial state (clears bloom, cache, insertion order, and stats). Configuration is preserved.