Struct Deduplicator 

pub struct Deduplicator { /* private fields */ }

Deduplicator for tracking seen messages and preventing duplicates.

Supports two modes:

• HashMap mode: exact tracking with FIFO eviction (original behavior)
• Bloom filter mode: O(1) memory with probabilistic duplicate detection

Implementations

impl Deduplicator

pub fn new() -> Self

Creates a new deduplicator with default configuration.

pub fn with_config(config: DeduplicatorConfig) -> Self

Creates a new deduplicator with custom configuration.

pub fn reconfigure(&mut self, config: DeduplicatorConfig)

Applies a new configuration in place, carrying the tracked set across.

Rebuilding from scratch on a runtime configuration change forgets every id held, which was equivalent to a restart while the set lived in memory only and is a loss now that it is persisted: the ids restored at launch would be dropped, and the next export would overwrite the record with a near-empty set — so the restart-time duplicate the record exists to recognise would be processed after all. The exact-mode entries therefore survive, re-bounded by the new configuration: ids past the new retention window are dropped, and past the new cap the newest are kept, with each entry’s exportable flag intact so a carried-over Self::mark_seen_local id still stays out of Self::export_seen.

A change of mode carries nothing: a bloom filter has no ids to hand over and cannot take ids in.

pub fn is_exact(&self) -> bool

Returns true when deduplication is exact (HashMap mode): a positive Self::is_duplicate answer is then authoritative. In bloom mode (false) a positive answer may be a false positive (~1% with default settings), so callers must not treat a hit as proof of a prior send.

pub fn is_duplicate(&self, message_id: &MessageId) -> bool

Checks if a message has been seen before.

Arguments

• message_id - The message ID to check

Returns

Returns true if the message has been seen, false otherwise.

Note: In bloom filter mode, this may return false positives (claiming a message is a duplicate when it isn’t). The false positive rate is typically below 1% with default settings.

pub fn is_duplicate_mut(&mut self, message_id: &MessageId) -> bool

Checks if a message has been seen before (mutable version for bloom rotation and LRU access tracking).

pub fn mark_seen(&mut self, message_id: MessageId) -> bool

Marks a message as seen.

Arguments

• message_id - The message ID to mark as seen

Returns

Returns true if this is a new message, false if it was already seen.

pub fn mark_seen_local(&mut self, message_id: MessageId) -> bool

Marks a message as seen for the life of this process only.

Identical to Self::mark_seen for Self::is_duplicate, eviction and retention, but the id is left out of Self::export_seen. For a sender marking its own outgoing frame: the mark exists to stop a relayed echo of that frame re-entering the receive path, which a restart does not change, and exporting it would spend the persisted record’s cap on ids no peer will ever push at this node. A later Self::mark_seen of an id already tracked here is a no-op and does not make it exportable.

Returns

Returns true if the message was newly marked, false if it was already tracked.

pub fn unmark_seen(&mut self, message_id: &MessageId) -> bool

Removes a message from the seen set so a future copy is processed fresh instead of being treated as a duplicate.

Intended for receivers that reject a message after marking it seen (e.g. an encryption-policy or security-gate rejection): forgetting the id makes a replay re-enter the same gate — and be rejected again without a delivery ACK — instead of hitting the duplicate re-ACK path, which would confirm to an injector that the target is online.

Returns

Returns true if the message was tracked and has been removed. Bloom filters do not support removal, so in bloom filter mode this is a no-op returning false: a replayed rejected message is treated as a duplicate until its filter window rotates out.

pub fn export_seen(&self, max: usize) -> Vec<SeenId>

Exports up to max tracked ids, newest first, for persistence.

Exact (HashMap) mode only: a bloom filter has no ids to export, so bloom mode returns an empty vector and the persisted set simply stays empty. Newest first so that a cap smaller than the tracked set keeps the ids most likely to be replayed — the recent ones. Ids tracked with Self::mark_seen_local are left out.

pub fn import_seen(&mut self, entries: Vec<SeenId>, now: DateTime<Utc>) -> usize

Re-admits ids exported by Self::export_seen on a previous run.

Skips ids already tracked (the live set wins), ids whose retention window has closed as of now, and stops at max_tracked_messages so a restore can never push the set over its cap or evict live entries. Entries are admitted in the order given, so a caller that wants the newest to survive a tight cap passes them newest first — which is the order export_seen produces. Returns how many were admitted. A no-op in bloom mode, which has nothing to import into.

pub fn cleanup_expired(&mut self) -> usize

Removes expired entries that exceed the retention time.

Returns

Returns the number of entries removed.

Note: In bloom filter mode, expiration is handled automatically through filter rotation, so this returns 0.

pub fn tracked_count(&self) -> usize

Gets the number of messages currently being tracked.

Note: In bloom filter mode, this returns an estimate based on insertion count.

pub fn clear(&mut self)

Clears all tracked messages.

pub fn is_tracked(&self, message_id: &MessageId) -> bool

Checks if a message ID is being tracked (seen).

pub fn is_bloom_filter_mode(&self) -> bool

Returns whether bloom filter mode is active.

pub fn stats(&self) -> DeduplicatorStats

Gets statistics about the deduplicator.

pub fn estimated_memory_bytes(&self) -> usize

Gets the estimated memory usage in bytes.

Trait Implementations

impl Default for Deduplicator

fn default() -> Self

Returns the “default value” for a type.