Module message_dedup 

Message deduplication using a two-layer approach.

Provides efficient duplicate detection for gossip/pubsub messages via:

• Layer 1 – Bloom filter: Fast probabilistic rejection using a bit array with Kirsch-Mitzenmacher double hashing (FNV-1a based). If the bloom filter reports “not seen”, the message is definitely new.
• Layer 2 – Bounded LRU cache: Exact deduplication backed by a HashMap<[u8;32], DedupEntry> with an insertion-order Vec for LRU eviction once capacity is reached.

Usage

use ipfrs_network::message_dedup::{MessageDeduplicator, DedupConfig, MsgId};

let config = DedupConfig {
    bloom_bits: 1 << 16,
    bloom_hash_count: 7,
    cache_capacity: 4096,
    window_ms: 60_000,
};
let mut dedup = MessageDeduplicator::new(config);

let id = MessageDeduplicator::make_msg_id(b"hello world");
assert!(!dedup.check_and_insert(&id, 1_000));  // new
assert!( dedup.check_and_insert(&id, 1_001));  // duplicate

Structs

DedupConfig

Configuration for MessageDeduplicator.

DedupEntry

A single entry in the exact-dedup LRU cache.

MessageDeduplicator

Two-layer message deduplicator: Bloom filter + bounded LRU cache.

MsgDedupStats

Accumulated statistics for a MessageDeduplicator.

MsgId

A 32-byte message fingerprint used for deduplication.