Module deduplicator 

Message deduplication using rotating bloom filters for O(1) memory.

This module implements a scalable deduplication system using bloom filters that can handle millions of messages with constant memory usage.

The rotating bloom filter approach uses multiple time-windowed filters:

• Current filter: accepts new message IDs
• Previous filters: used for lookups, rotated out after retention period

Structs

Deduplicator

Deduplicator for tracking seen messages and preventing duplicates.

DeduplicatorConfig

Configuration for deduplication.

DeduplicatorStats

Statistics about the deduplicator.

SeenId

One id of the exact-mode seen set, as exported for persistence and imported on the next launch. seen_at_ms is wall-clock milliseconds so the retention window can be re-applied on import.

Enums

DeduplicatorMode

The mode of operation for the deduplicator.