Struct SparseHandle 

pub struct SparseHandle { /* private fields */ }

Implementations

impl SparseHandle

pub fn create(path: &str) -> Result<Self, String>

Create a new empty sparse index at the given path.

pub fn open(path: &str) -> Result<Self, String>

Open an existing sparse index. Tries new mmap format first, falls back to legacy bincode.

pub fn create_with_store( store: Arc<dyn BlobStore>, index_name: &str, cache_base: &Path, ) -> Result<Self, String>

Create a new empty sparse index backed by a BlobStore.

cache_base is the root directory for mmap caches. Inside it, a unique subdirectory {pid}/{index_name}_{seq} is created automatically. Source of truth is the store.

pub fn open_with_store( store: Arc<dyn BlobStore>, index_name: &str, cache_base: &Path, ) -> Result<Self, String>

Open an existing sparse index from a BlobStore.

cache_base is the root directory for mmap caches. Blobs are materialized from the store into {cache_base}/{pid}/{index_name}_{seq}/, then mmap’d.

pub fn insert(&self, node_id: u64, vector: &SparseVector) -> Result<(), String>

pub fn remove(&self, node_id: u64) -> Result<bool, String>

pub fn search(&self, query: &SparseVector, limit: usize) -> Vec<(u64, f32)>

pub fn search_filtered( &self, query: &SparseVector, limit: usize, allowed_ids: &[u64], ) -> Vec<(u64, f32)>

Top-limit records among allowed_ids only.

A sparse score is a plain dot product with no corpus statistics, so this is exactly Self::search intersected with the set: the same documents in the same order, with the same scores (to a few units in the last place — the two paths add a document’s lanes in a different order). Pinned by tests/test_filter_truth.rs.

Hand over sorted, unique ids when you can: the set is then read where it is, and the filter costs between ×0.15 (a very selective set, which is faster than searching everything) and ×1.3 of an unfiltered search at any size — 540 000 ids answer in 0.22 ms where they took 6.0 ms before (tests/bench_filter_selectivity.rs). An unsorted set is copied, sorted and deduplicated at every query.

pub fn compact(&self) -> Result<(), String>

Merge every segment into one, applying the tombstones — the walk over sorted token tables described in crate::segments. What it buys: one mapping to search instead of N, WAND pruning over the whole index again, and the deleted documents’ bytes back.

Commits are cheap because they append; this is where that is paid, once, when the caller decides. Nothing is lost if it is interrupted: the manifest is only rewritten once the merged segment is on disk.

pub fn num_segments(&self) -> usize

How many segments the index is made of — what a compaction policy watches, and what a search pays per query dimension.

pub fn len(&self) -> usize

pub fn is_empty(&self) -> bool

pub fn commit_inner(&self) -> Result<(), String>

Write index to disk in the new mmap format, then re-mmap. If store-backed, also persists to BlobStore. Write what is in RAM as a new segment and point the manifest at it. What was already committed is not touched: the cost of a commit is the cost of the delta, where it used to be the cost of the whole index (tests/bench_commit_cost.rs).

An index written before segments is converted here, once: its whole content becomes segment zero — the full write it did at every commit anyway — and meta.json appears next to it.

Trait Implementations

impl Drop for SparseHandle

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.