Skip to main content

Module mmap_index

Module mmap_index 

Source
Expand description

Flat binary mmap format for sparse-vector posting lists.

Layout (every struct is #[repr(C)] for a stable layout):

[FileHeader]                    16 bytes
[DimHeader × num_dims]          16 bytes × N
[PostingEntry × total_entries]  16 bytes × M

Each entry carries a max_next_weight ceiling. Files written here store the inclusive ceiling of the wand module (tail_max: the maximum weight over the entry and everything after it); readers fold max(weight, max_next_weight), so a file whose ceiling excludes the entry itself reads identically.

Structs§

MmapPostingData
Mmap’d posting data — read-only view of sparse.mmap.
PostingEntry
On-disk posting entry.

Functions§

search_mmap
Top-limit search straight from the mapping, without loading anything into RAM. dim_map translates the query’s token ids into the file’s dimension indices.
search_mmap_allowed
search_mmap restricted to allowed ids (see run_search_allowed).
write_mmap_file
Write the flat binary mmap format from in-RAM posting lists, one per remapped dimension.