Struct AsyncSegmentReader 

pub struct AsyncSegmentReader { /* private fields */ }

Async segment reader with lazy loading

• Term dictionary: only index loaded, blocks loaded on-demand
• Postings: loaded on-demand per term via HTTP range requests
• Document store: only index loaded, blocks loaded on-demand via HTTP range requests

Implementations

impl AsyncSegmentReader

pub async fn open<D: Directory>( dir: &D, segment_id: SegmentId, schema: Arc<Schema>, doc_id_offset: DocId, cache_blocks: usize, ) -> Result<Self>

Open a segment with lazy loading

pub fn meta(&self) -> &SegmentMeta

pub fn num_docs(&self) -> u32

pub fn avg_field_len(&self, field: Field) -> f32

Get average field length for BM25F scoring

pub fn doc_id_offset(&self) -> DocId

pub fn schema(&self) -> &Schema

pub fn sparse_indexes(&self) -> &FxHashMap<u32, SparseIndex>

Get sparse indexes for all fields

pub fn vector_indexes(&self) -> &FxHashMap<u32, VectorIndex>

Get vector indexes for all fields

pub fn term_dict_stats(&self) -> SSTableStats

Get term dictionary stats for debugging

pub async fn get_postings( &self, field: Field, term: &[u8], ) -> Result<Option<BlockPostingList>>

Get posting list for a term (async - loads on demand)

For small posting lists (1-3 docs), the data is inlined in the term dictionary and no additional I/O is needed. For larger lists, reads from .post file.

pub async fn doc(&self, local_doc_id: DocId) -> Result<Option<Document>>

Get document by local doc_id (async - loads on demand)

pub async fn prefetch_terms( &self, field: Field, start_term: &[u8], end_term: &[u8], ) -> Result<()>

Prefetch term dictionary blocks for a key range

pub fn store_has_dict(&self) -> bool

Check if store uses dictionary compression (incompatible with raw merging)

pub fn store_raw_blocks(&self) -> Vec<RawStoreBlock>

Get raw store blocks for optimized merging

pub fn store_data_slice(&self) -> &LazyFileSlice

Get store data slice for raw block access

pub async fn all_terms(&self) -> Result<Vec<(Vec<u8>, TermInfo)>>

Get all terms from this segment (for merge)

pub async fn all_terms_with_stats(&self) -> Result<Vec<(Field, String, u32)>>

Get all terms with parsed field and term string (for statistics aggregation)

Returns (field, term_string, doc_freq) for each term in the dictionary. Skips terms that aren’t valid UTF-8.

pub fn term_dict_iter(&self) -> AsyncSSTableIterator<'_, TermInfo>

Get streaming iterator over term dictionary (for memory-efficient merge)

pub async fn read_postings(&self, offset: u64, len: u32) -> Result<Vec<u8>>

Read raw posting bytes at offset

pub fn search_dense_vector( &self, field: Field, query: &[f32], k: usize, rerank_factor: usize, combiner: MultiValueCombiner, ) -> Result<Vec<(DocId, f32)>>

Search dense vectors using RaBitQ

Returns (doc_id, score) pairs sorted by score (descending). The doc_ids are adjusted by doc_id_offset for this segment. If mrl_dim is configured, the query vector is automatically trimmed. For multi-valued documents, scores are combined using the specified combiner.

pub fn has_dense_vector_index(&self, field: Field) -> bool

Check if this segment has a dense vector index for the given field

pub fn get_dense_vector_index(&self, field: Field) -> Option<Arc<RaBitQIndex>>

Get the dense vector index for a field (if available)

pub fn get_ivf_vector_index(&self, field: Field) -> Option<Arc<IVFRaBitQIndex>>

Get the IVF vector index for a field (if available)

pub fn get_scann_vector_index( &self, field: Field, ) -> Option<(Arc<IVFPQIndex>, Arc<PQCodebook>)>

Get the ScaNN vector index for a field (if available)

pub fn get_vector_index(&self, field: Field) -> Option<&VectorIndex>

Get the vector index type for a field

pub async fn search_sparse_vector( &self, field: Field, vector: &[(u32, f32)], limit: usize, combiner: MultiValueCombiner, ) -> Result<Vec<(u32, f32)>>

Search for similar sparse vectors using dedicated sparse posting lists

Uses shared WandExecutor with SparseTermScorer for efficient top-k retrieval. Optimizations (via WandExecutor):

1. MaxScore pruning: Dimensions sorted by max contribution
2. Block-Max WAND: Skips blocks where max contribution < threshold
3. Top-K heap: Efficient score collection

Returns (doc_id, score) pairs sorted by score descending.

pub async fn get_positions( &self, field: Field, term: &[u8], ) -> Result<Option<PositionPostingList>>

Get positions for a term (for phrase queries)

Position offsets are now embedded in TermInfo, so we first look up the term to get its TermInfo, then use position_info() to get the offset.

pub fn has_positions(&self, field: Field) -> bool

Check if positions are available for a field