Struct SparseIndex 

pub struct SparseIndex {
    pub total_docs: u32,
    pub total_vectors: u32,
    /* private fields */
}

Sparse vector index for a field: lazy loading with mmap

Only stores the offset table in memory. Posting lists are loaded on-demand during queries using mmap for memory efficiency.

Fields

total_docs: u32

Total document count in this segment (for IDF computation)

total_vectors: u32

Total sparse vectors in this segment (for multi-valued IDF)

Implementations

impl SparseIndex

pub fn new( handle: LazyFileHandle, offsets: Vec<(u64, u32)>, total_docs: u32, total_vectors: u32, ) -> Self

Create a new lazy sparse index

pub async fn get_posting( &self, dim_id: u32, ) -> Result<Option<Arc<BlockSparsePostingList>>>

Get posting list for a dimension (async, loads on-demand)

pub fn has_dimension(&self, dim_id: u32) -> bool

Check if dimension exists (without loading)

pub fn num_dimensions(&self) -> usize

Get the number of dimensions in the index

pub fn clear_cache(&self)

Clear the cache (useful for memory pressure)

pub fn cache_size(&self) -> usize

Get cache size (number of cached posting lists)

pub fn active_dimensions(&self) -> impl Iterator<Item = u32> + '_

Iterate over all active dimension IDs (dimensions that have posting lists) This is synchronous and doesn’t load the posting lists

pub fn get_cached(&self, dim_id: u32) -> Option<Arc<BlockSparsePostingList>>

Get cached posting list if available (for synchronous access) Returns None if not cached (doesn’t load from disk)

pub fn idf(&self, dim_id: u32) -> f32

Compute IDF (inverse document frequency) for a dimension Uses cached posting if available, otherwise returns 0

pub fn idf_weights(&self, dim_ids: &[u32]) -> Vec<f32>

Get IDF weights for multiple dimensions (uses cache only)

pub fn get_posting_blocking( &self, dim_id: u32, ) -> Result<Option<Arc<BlockSparsePostingList>>>

Get posting list synchronously (blocking) - for merger use only Prefer async get_posting() for query-time access

Trait Implementations

impl Clone for SparseIndex

fn clone(&self) -> SparseIndex

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.