Struct SparseVectorQuery 

pub struct SparseVectorQuery {
    pub field: Field,
    pub vector: Vec<(u32, f32)>,
    pub combiner: MultiValueCombiner,
    pub heap_factor: f32,
    pub weight_threshold: f32,
    pub max_query_dims: Option<usize>,
    pub pruning: Option<f32>,
    pub min_query_dims: usize,
    pub over_fetch_factor: f32,
    pub lsp_gamma: Option<usize>,
    /* private fields */
}

Sparse vector query for similarity search

Fields

field: Field

Field containing the sparse vectors

vector: Vec<(u32, f32)>

Query vector as (dimension_id, weight) pairs

combiner: MultiValueCombiner

How to combine scores for multi-valued documents

heap_factor: f32

Approximate search factor (1.0 = exact, lower values = faster but approximate) Controls MaxScore pruning aggressiveness in block-max scoring

weight_threshold: f32

Minimum abs(weight) for query dimensions (0.0 = no filtering) Dimensions below this threshold are dropped from candidate generation. BMP still uses the bounded full query when scoring visited candidates.

max_query_dims: Option<usize>

Maximum candidate-generation dimensions (None = implementation cap). Keeps only the top-k dimensions by abs(weight); BMP final scoring uses up to MAX_QUERY_TERMS dimensions from the full query.

pruning: Option<f32>

Fraction of query dimensions to keep (0.0-1.0), same semantics as indexing-time pruning: sort by abs(weight) descending, keep top fraction. BMP applies it to candidate generation and scores visited candidates with the bounded full query. None or 1.0 = no pruning.

min_query_dims: usize

Minimum number of query dimensions before pruning and weight_threshold filtering are applied. Protects short queries from losing signal. Default: 4. Set to 0 to always apply.

over_fetch_factor: f32

Multiplier on executor limit for ordinal deduplication (1.0 = no over-fetch)

lsp_gamma: Option<usize>

LSP/0 γ. None is depth-derived; Some(0) is exhaustive.

Implementations

impl SparseVectorQuery

pub fn new(field: Field, vector: Vec<(u32, f32)>) -> Self

Create a new sparse vector query

Default combiner is LogSumExp { temperature: 0.7 } — a softmax-weighted smooth maximum. A document’s score follows its strongest ordinals; ordinal count contributes nothing on its own, so many-chunk documents cannot outrank a focused strong match.

pub fn with_combiner(self, combiner: MultiValueCombiner) -> Self

Set the multi-value score combiner

pub fn with_over_fetch_factor(self, factor: f32) -> Self

Set executor over-fetch factor for multi-valued fields. After MaxScore execution, ordinal combining may reduce result count; this multiplier compensates by fetching more from the executor. (1.0 = no over-fetch, 2.0 = fetch 2x then combine down)

pub fn with_heap_factor(self, heap_factor: f32) -> Self

Set the heap factor for approximate search

Controls the trade-off between speed and recall:

• 1.0 = exact search (default)
• 0.8-0.9 = ~20-40% faster with minimal recall loss
• Lower values = more aggressive pruning, faster but lower recall

pub fn with_weight_threshold(self, threshold: f32) -> Self

Set minimum weight threshold for query dimensions Dimensions with abs(weight) below this are dropped before search.

pub fn with_max_query_dims(self, max_dims: usize) -> Self

Set maximum number of query dimensions (top-k by weight)

pub fn with_pruning(self, fraction: f32) -> Self

Set pruning fraction (0.0-1.0): keep top fraction of query dims by weight. Same semantics as indexing-time pruning.

pub fn with_min_query_dims(self, min_dims: usize) -> Self

Set minimum query dimensions before pruning/filtering are applied. Queries with fewer dimensions than this skip weight_threshold and pruning.

pub fn with_lsp_gamma(self, gamma: usize) -> Self

Select at most the top-γ superblocks by SBMax using LSP/0. Zero retains exhaustive SBMax-ordered traversal.

pub fn from_indices_weights( field: Field, indices: Vec<u32>, weights: Vec<f32>, ) -> Self

Create from separate indices and weights vectors

pub fn from_text( field: Field, text: &str, tokenizer_name: &str, weighting: QueryWeighting, sparse_index: Option<&SparseIndex>, ) -> Result<Self>

Create from raw text using a HuggingFace tokenizer (single segment)

This method tokenizes the text and creates a sparse vector query. For multi-segment indexes, use from_text_with_stats instead.

Arguments

• field - The sparse vector field to search
• text - Raw text to tokenize
• tokenizer_name - HuggingFace tokenizer path (e.g., “bert-base-uncased”)
• weighting - Weighting strategy for tokens
• sparse_index - Optional sparse index for IDF lookup (required for IDF weighting)

pub fn from_text_with_stats( field: Field, text: &str, tokenizer: &HfTokenizer, weighting: QueryWeighting, global_stats: Option<&GlobalStats>, ) -> Result<Self>

Create from raw text using global statistics (multi-segment)

This is the recommended method for multi-segment indexes as it uses aggregated IDF values across all segments for consistent ranking.

Arguments

• field - The sparse vector field to search
• text - Raw text to tokenize
• tokenizer - Pre-loaded HuggingFace tokenizer
• weighting - Weighting strategy for tokens
• global_stats - Global statistics for IDF computation

pub fn from_text_with_tokenizer_bytes( field: Field, text: &str, tokenizer_bytes: &[u8], weighting: QueryWeighting, global_stats: Option<&GlobalStats>, ) -> Result<Self>

Create from raw text, loading tokenizer from index directory

This method supports the index:// prefix for tokenizer paths, loading tokenizer.json from the index directory.

Arguments

• field - The sparse vector field to search
• text - Raw text to tokenize
• tokenizer_bytes - Tokenizer JSON bytes (pre-loaded from directory)
• weighting - Weighting strategy for tokens
• global_stats - Global statistics for IDF computation

Trait Implementations

impl Clone for SparseVectorQuery

fn clone(&self) -> SparseVectorQuery

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SparseVectorQuery

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for SparseVectorQuery

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Query for SparseVectorQuery

fn scorer<'a>( &self, reader: &'a SegmentReader, limit: usize, ) -> ScorerFuture<'a>

Create a scorer for this query against a single segment (async)

fn scorer_with_options<'a>( &self, reader: &'a SegmentReader, limit: usize, options: ScorerOptions, ) -> ScorerFuture<'a>

Create a scorer with collector-specific construction options. Query implementations that can avoid optional position data should override this; the default preserves existing behavior.

fn scorer_sync<'a>( &self, reader: &'a SegmentReader, limit: usize, ) -> Result<Box<dyn Scorer + 'a>>

Create a scorer synchronously (mmap/RAM only).

fn scorer_sync_with_options<'a>( &self, reader: &'a SegmentReader, limit: usize, options: ScorerOptions, ) -> Result<Box<dyn Scorer + 'a>>

Synchronous counterpart to Query::scorer_with_options.

fn count_estimate<'a>(&self, _reader: &'a SegmentReader) -> CountFuture<'a>

Estimated number of matching documents in a segment (async)

fn decompose(&self) -> QueryDecomposition

Decompose this query for MaxScore optimization.

fn is_filter(&self) -> bool

True if this query is a pure filter (always scores 1.0, no positions). Used by the planner to convert non-selective MUST filters into predicates.

fn as_doc_predicate<'a>( &self, _reader: &'a SegmentReader, ) -> Option<DocPredicate<'a>>

For filter queries: return a cheap per-doc predicate against a segment. The predicate does O(1) work per doc (e.g., fast-field lookup).

fn as_doc_bitset(&self, _reader: &SegmentReader) -> Option<DocBitset>

Build a compact bitset of matching doc_ids for this query.

fn bitset_cardinality_estimate(&self, _reader: &SegmentReader) -> Option<u64>

Cheap estimate of how many docs this filter clause matches in the segment. Used by the boolean planner to order MUST/MUST_NOT evaluation: the narrowest clause is materialized first and wider clauses refine it with per-doc probes instead of being fully materialized. None = unknown (treated as matching everything).