Module shared_string_arena 

SharedStringArena - append-only position-independent string pool backed by an MMF.

Why this exists

Variable-length strings can’t be stored inline in fixed-size slots (SharedHashMap, SharedVec, etc.) without padding waste or truncation. The natural cross-process solution is a shared byte arena: every process maps the same file at (potentially) different base addresses, and string references are position-independent (offset, len) pairs. Adding mmap_base + offset in any process resolves to the same bytes.

Layout

+---------------------------+
| ArenaHeader (64B)         |
|   magic, capacity_bytes   |
|   used_bytes: AtomicU64   |
+---------------------------+
| bytes[0 .. capacity]      |
+---------------------------+

Protocol

intern(s):

1. offset = used_bytes.fetch_add(len).
2. If offset + len > capacity, rollback with fetch_sub(len) and return Full. (Note: the rollback is best-effort; if two threads race-overflow simultaneously, both fetch_subs leave the counter deterministic without “losing” bytes.)
3. Memcpy s.bytes() into arena[offset..offset+len].
4. Return StringRef { offset, len }.

get(r):

• Bounds-check r.offset + r.len <= used_bytes (sanity), then return &arena[r.offset..r.offset+r.len] as a &str.

Concurrency

Concurrent interners get distinct slices via fetch_add. Once the bytes are written, they are never moved (append-only). A reader holding a StringRef can always resolve it correctly, provided their get happens AFTER the interner returned the ref (which is the natural happens-before edge: the interner does the write, then makes the ref visible to the reader).

Deduplication

Not provided here. For dedup, layer a SharedHashMap<u64 hash, StringRef> over the arena and consult it before each intern.

No deletion

Append-only. The whole arena is reclaimed via clear (callers must ensure no concurrent readers); fine-grained deletion requires a free-list / compaction protocol that defeats the point of an arena.

Structs

ArenaHeader

SharedStringArena

StringRef

Position-independent reference to a string in a SharedStringArena. Encoded as a u64 of OFFSET_BITS offset and LEN_BITS length, for stable cross-process passing: the same u64 resolves to the same bytes in every process that maps the arena.

Enums

ArenaError

Constants

ARENA_MAGIC

Format tag written last when a region is initialized, and required to match on open. The trailing byte is the layout generation: A1 carried a StringRef packed as offset:u32|len:u32, A2 packs it 40:24. A region written by one generation resolves every ref wrongly under the other, so the tag differs and the older region is refused rather than misread.

ARENA_MAGIC_V1

The A1 tag, retained so an old-format region is recognised and named in the refusal instead of reported as unrecognised bytes.

LEN_BITS

Bits given to the string length, and the resulting ceiling on one interned string.

MAX_LEN

Largest interned string: 16 MiB - 1.

MAX_OFFSET

Largest addressable byte offset: 1 TiB - 1.

OFFSET_BITS

Bits of a packed StringRef given to the byte offset, and the resulting ceiling on arena capacity.

Functions

arena_file_size