Skip to main content

Module dict

Module dict 

Source
Expand description

Per-document string-interning dict with refcounted ownership.

A Dict gives every distinct byte sequence a stable, dict-owned, NUL-terminated pointer. Looking up the same bytes twice returns the same pointer; consumers may rely on pointer equality to test “is this the same name?” without a strcmp.

§Refcount model

Each Dict carries an internal atomic refcount, mirroring libxml2’s xmlDict ownership convention. This lets multiple independent consumers — a parser context, a built document, the C-ABI consumer’s own handle — co-own the same dict without coordination beyond xmlDictReference / xmlDictFree.

  • Dict::new_refcounted returns a freshly heap-allocated dict at refcount 1 (one outstanding reference, owned by the caller).
  • Dict::add_ref bumps the count for an additional borrow.
  • Dict::release decrements; the last release drops the dict and frees every interned string.

Atomic ordering uses Release on decrement and an Acquire fence on the last release — the standard pattern (cf. std::sync::Arc).

§Why interning instead of arena allocation

Element / attribute / namespace names in XML repeat heavily. An HTML page has thousands of <p> / <a> / <div> tags; an OSM dump has millions of <node> records. Bumpalo-arena allocation is fast but stores each occurrence as a separate copy — the same name string ends up at thousands of distinct heap addresses, defeating pointer equality and wasting space.

The dict trades a hashmap lookup per unique name for one allocation per unique name and constant-cost equality checks across all occurrences. For docs with high name repetition the total bytes-stored often drops by 10×.

Content strings (text node bodies, attribute values) are NOT interned — they’re typically unique per occurrence and would waste memory hashing things that never collide. They stay in the document’s bumpalo arena.

§Performance notes

  • The hashmap uses std’s default RandomState (SipHash). For XML names (typically ≤16 bytes) that’s a few-ns hash; the more expensive path is the hashmap probe + alloc on miss. A fast non-DoS-resistant hasher (FxHash / ahash) would shave ~20-30% off the hash cost — worth doing if profiling identifies it.
  • Storage uses one Box<[u8]> per unique name. An alternative would back the strings with a bumpalo-style arena owned by the dict itself, paying only one allocation per insert and freeing everything wholesale on Dict drop. The cost of the doubled allocation is amortised across all occurrences of the name, which is usually many — switch only if profiling demands.
  • The hot path (lookup of an already-interned name) is one hash
    • one byte comparison. No allocation. Designed to be cheap enough to call on every element / attribute name during parse.

Structs§

Dict
A refcounted per-document string interner.