Crate sporse 

Sparse vector index for learned sparse retrieval.

Indexes sparse vectors using an inverted index with Block-Max WAND traversal for exact top-k inner product search. The vectors can be lexical (SPLADE-style vocabulary-term weights) or latent (the composite codes the sae module learns); the index treats both the same way.

Quick start

use sporse::{SparseVec, SporseIndex};

let mut index = SporseIndex::new();

// Insert documents as sparse vectors
index.insert(0, &SparseVec::new(vec![(0, 1.0), (3, 2.5), (7, 0.8)]));
index.insert(1, &SparseVec::new(vec![(1, 3.0), (3, 1.0)]));
index.insert(2, &SparseVec::new(vec![(0, 0.5), (7, 2.0)]));

// Build the index (computes block-max metadata)
index.build();

// Search: returns (doc_id, score) pairs, highest score first
let query = SparseVec::new(vec![(0, 1.0), (3, 1.0)]);
let results = index.search(&query, 2);
assert_eq!(results[0].0, 0); // doc 0 scores 1.0*1.0 + 2.5*1.0 = 3.5

Modules

sae

Composite-code sparse-autoencoder codes (CCSA) and their index integration. Composite-code sparse-autoencoder codes (CCSA; Lassance, Formal, and Clinchant 2022, arXiv:2204.07023): the encoding and index-integration half.

Structs

RawImpactQuantizer

Quantization policy for writing SparseVec weights as raw impacts.

SparseVec

A sparse vector: sorted list of (dimension, weight) pairs.

SporseIndex

Inverted index for sparse vector retrieval using Block-Max WAND.

Enums

RawImpactError

Errors returned when converting sparse vectors to quantized raw impacts.