Struct SparseVectorConfig 

pub struct SparseVectorConfig {
    pub index_size: IndexSize,
    pub weight_quantization: WeightQuantization,
    pub weight_threshold: f32,
    pub block_size: usize,
    pub posting_list_pruning: Option<f32>,
    pub query_config: Option<SparseQueryConfig>,
}

Configuration for sparse vector storage

Research-validated optimizations for learned sparse retrieval (SPLADE, uniCOIL, etc.):

• Weight threshold (0.01-0.05): Removes ~30-50% of postings with minimal nDCG impact
• Posting list pruning (0.1): Keeps top 10% per dimension, 50-70% index reduction, <1% nDCG loss
• Query pruning (top 10-20 dims): 30-50% latency reduction, <2% nDCG loss
• UInt8 quantization: 4x compression, 1-2% nDCG loss (optimal trade-off)

Fields

index_size: IndexSize

Size of dimension/term indices

weight_quantization: WeightQuantization

Quantization for weights (see WeightQuantization docs for trade-offs)

weight_threshold: f32

Minimum weight threshold - weights below this value are not indexed

Research recommendation (Guo et al., 2022; SPLADE v2):

• 0.01-0.05 for SPLADE models removes ~30-50% of postings
• Minimal impact on nDCG@10 (<1% loss)
• Major reduction in index size and query latency

block_size: usize

Block size for posting lists (must be power of 2, default 128 for SIMD) Larger blocks = better compression, smaller blocks = faster seeks

posting_list_pruning: Option<f32>

Static pruning: fraction of postings to keep per inverted list (SEISMIC-style) Lists are sorted by weight descending and truncated to top fraction.

Research recommendation (SPLADE v2, Formal et al., 2021):

• None = keep all postings (default, exact)
• Some(0.1) = keep top 10% of postings per dimension
  • 50-70% index size reduction
  • <1% nDCG@10 loss
  • Exploits “concentration of importance” in learned representations

Applied only during initial segment build, not during merge.

query_config: Option<SparseQueryConfig>

Query-time configuration (tokenizer, weighting)

Implementations

impl SparseVectorConfig

pub fn splade() -> Self

SPLADE-optimized config with research-validated defaults

Optimized for SPLADE, uniCOIL, and similar learned sparse retrieval models. Based on research findings from:

• Pati (2025): UInt8 quantization = 4x compression, 1-2% nDCG loss
• Formal et al. (2021): SPLADE v2 posting list pruning
• Qiao et al. (2023): Query dimension pruning and approximate search
• Guo et al. (2022): Weight thresholding for efficiency

Expected performance vs. full precision baseline:

• Index size: ~15-25% of original (combined effect of all optimizations)
• Query latency: 40-60% faster
• Effectiveness: 2-4% nDCG@10 loss (typically acceptable for production)

Vocabulary: ~30K dimensions (fits in u16)

pub fn compact() -> Self

Compact config: Maximum compression (experimental)

Uses aggressive UInt4 quantization for smallest possible index size. Expected trade-offs:

• Index size: ~10-15% of Float32 baseline
• Effectiveness: ~3-5% nDCG@10 loss

Recommended for: Memory-constrained environments, cache-heavy workloads

pub fn full_precision() -> Self

Full precision config: No compression, baseline effectiveness

Use for: Research baselines, when effectiveness is critical

pub fn conservative() -> Self

Conservative config: Mild optimizations, minimal effectiveness loss

Balances compression and effectiveness with conservative defaults. Expected trade-offs:

• Index size: ~40-50% of Float32 baseline
• Query latency: ~20-30% faster
• Effectiveness: <1% nDCG@10 loss

Recommended for: Production deployments prioritizing effectiveness

pub fn with_weight_threshold(self, threshold: f32) -> Self

Set weight threshold (builder pattern)

pub fn with_pruning(self, fraction: f32) -> Self

Set posting list pruning fraction (builder pattern) e.g., 0.1 = keep top 10% of postings per dimension

pub fn bytes_per_entry(&self) -> f32

Bytes per entry (index + weight)

pub fn to_byte(&self) -> u8

Serialize config to a single byte

pub fn from_byte(b: u8) -> Option<Self>

Deserialize config from a single byte Note: weight_threshold, block_size and query_config are not serialized in the byte

pub fn with_block_size(self, size: usize) -> Self

Set block size (builder pattern) Must be power of 2, recommended: 64, 128, 256

pub fn with_query_config(self, config: SparseQueryConfig) -> Self

Set query configuration (builder pattern)

Trait Implementations

impl Clone for SparseVectorConfig

fn clone(&self) -> SparseVectorConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SparseVectorConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SparseVectorConfig

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for SparseVectorConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl PartialEq for SparseVectorConfig

fn eq(&self, other: &SparseVectorConfig) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Serialize for SparseVectorConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for SparseVectorConfig