Module rowmap 

Memory-mapped, process-shareable per-message meta map keyed by stable row_id. Resolves FTS/vector _rowids to (session_id, message_id) and hydrates hit meta (role, project, source_agent, timestamp, search_text) in memory, with a take_rows miss-fallback for rows appended since the build. Also carries a session_id -> message count aggregate.

session_id/project/source_agent/role are dictionary-encoded (each distinct value stored once, referenced by u32 index); search_text is block-compressed ([BLOCK_ROWS] rows per zstd block). On the real 2M-message corpus that takes the map from ~655 MB (flat) to ~270 MB.

Built via temp + atomic rename and mmap’d read-only, so N pond processes on the box share one physical copy in the OS page cache and a restart re-opens instantly. Stable row ids (enable_stable_row_ids) keep a built map valid across compaction; it only rebuilds when the dataset version advances.

Layout: Header | [Record; count] | [SessionEntry] | [DictEntry; project] | [DictEntry; agent] | [DictEntry; role] | [BlockEntry] | blob. Records are sorted by row_id (binary search); a row’s block is record_index / BLOCK_ROWS. The blob holds the compressed search_text blocks, then per-row {32-byte header + message_id}, then per-session session_id bytes, then the dict value bytes. Each session entry also carries its max message timestamp, the watermark the pond sync skip oracle compares against the source.

Structs

ChainPaths

The on-disk LSM chain for a store: the highest-version base plus every delta layered above it, ascending.

RowMeta

Borrowed view of one row’s meta. The dictionary-encoded fields borrow the mmap; search_text is owned (decompressed from its block).

RowMetaEntry

Owned input row for RowMetaMap::build.

RowMetaMap

An open, memory-mapped row meta map. lookup, lookup_meta, and lookup_count are lock-free and reentrant.

RowMetaSet

An LSM chain of immutable segment maps (base + ascending deltas) viewed as one logical map. Rows are partitioned across segments by row_id (append is disjoint; compaction rebuilds the base), so key/meta lookups take the newest hit and counts sum across segments.

Functions

discover_chain

Discover the chain under cache_dir for store_key: the highest-version base (-v{V}) plus every delta (-d{V}) above it, ascending. None if no base exists yet.