Skip to main content

Module store

sanitize_engine

Module store 

Source
Expand description

Thread-safe, concurrent one-way replacement store.

§Concurrency Model

The store uses dashmap::DashMap — a concurrent hash map with shard-level locking (default 64 shards). This gives us:

  • Lock-free reads for lookups of already-mapped values.
  • Shard-level write locks that are held only while inserting a new entry. With 64 shards and 8–16 threads, the probability of two threads contending on the same shard is very low.
  • Atomic get-or-insert via the entry() API, which prevents TOCTOU races and guarantees first-writer-wins semantics.

The forward map is keyed by (Category, original) → sanitized. Replacements are one-way only — there is no reverse map, no mapping file, and no restore capability.

§Memory Characteristics

At 10M unique values with average key length 20 bytes and average value length 30 bytes:

  • Forward map: 10M × (20 + 30 + ~120 DashMap overhead) ≈ 1.7 GB
  • Total: ~1.7 GB — acceptable for server workloads.

An optional capacity_limit can be set to prevent unbounded growth.

Structs§

MappingStore
Thread-safe concurrent one-way replacement store.