Struct Storage 

pub struct Storage {
    pub vectors: MmapVectorStore,
    pub wal: WriteAheadLog,
    pub metadata: MetadataStore,
    pub index: AnyIndex,
    pub sparse: SparseIndex,
    pub hybrid: HybridEngine,
    pub planner: QueryPlanner,
    pub config: CollectionConfig,
    /* private fields */
}

Fields

vectors: MmapVectorStore

wal: WriteAheadLog

metadata: MetadataStore

index: AnyIndex

sparse: SparseIndex

hybrid: HybridEngine

planner: QueryPlanner

config: CollectionConfig

Implementations

impl Storage

pub fn create(data_dir: &Path, config: &CollectionConfig) -> Result<Self>

pub fn open(data_dir: &Path, collection_name: &str) -> Result<Self>

pub fn upsert(&mut self, record: VectorRecord) -> Result<UpsertResult>

pub fn bulk_upsert(&mut self, records: Vec<VectorRecord>) -> Result<usize>

Bulk-insert new records efficiently: append all vectors, write the WAL in one batched fsync, insert metadata in one transaction, then build the dense (and sparse) index once — instead of the per-insert index rebuilds upsert triggers. Intended for loading a fresh/large collection.

Assumes the ids are new (uses append semantics). A final checkpoint truncates the WAL, since the data is durable in the vector store, the metadata db, and the saved index.

pub fn delete(&mut self, id: &VectorId) -> Result<()>

pub fn search(&self, query: &Vector, k: usize) -> Result<Vec<SearchResult>>

Search the dense HNSW index and hydrate results with metadata payloads.

pub fn search_sparse(&self, query: &str, k: usize) -> Result<Vec<SearchResult>>

Full-text BM25 search using the sparse index.

pub fn execute_sql(&self, sql: &str) -> Result<Vec<SearchResult>>

Parse and execute a SQL string against this collection.

The SQL must include a VECTOR_SIM(col, [...]) op threshold predicate. Optional scalar equality conditions (AND field = 'value') are applied as post-scan metadata filters.

pub fn execute_search( &self, request: &SearchRequest, ) -> Result<Vec<SearchResult>>

Route a SearchRequest through the query planner and execute the resulting physical plan.

This is the primary search entry point for callers that want the full planner pipeline (cost model, hybrid decision, candidate oversampling).

pub fn search_dense( &self, query_vector: &Vector, k: usize, ) -> Result<Vec<SearchResult>>

Dense HNSW search with full metadata hydration.

Equivalent to search but makes the intent explicit and is used by callers that distinguish dense vs hybrid paths.

pub fn search_dense_filtered( &self, query: &Vector, k: usize, filter: &Value, ) -> Result<Vec<SearchResult>>

Dense k-NN search restricted to records whose payload matches filter, with predicate pushdown.

When the filter is selective (the allowed set is small relative to the collection), candidates are generated from the filter: an exact, full-precision brute-force scan over only the matching vectors. This never scores the excluded majority, so a 95%-eliminating filter costs roughly 5% of the work — and is exact, unlike oversample-then-discard.

When the filter is weakly selective, it falls back to an ANN search with a post-filter (few candidates are dropped, so recall stays high).

pub fn search_hybrid( &self, query_vector: Option<&Vector>, query_text: Option<&str>, k: usize, alpha: f32, strategy: FusionStrategy, ) -> Result<Vec<SearchResult>>

Two-stage hybrid retrieval pipeline.

At least one of query_vector / query_text must be provided.

Stage 1 — Dense: fetch k * 5 HNSW candidates (or empty if no vector). Stage 2 — Sparse: re-score dense candidates with BM25 (or full-corpus search when no dense candidates exist). Stage 3 — Fusion: combine via strategy and alpha. Stage 4 — Hydrate: populate payload and text from metadata.

pub fn rebuild_index(&mut self) -> Result<()>

Rebuild the dense index from the current metadata + mmap store. Also rebuilds the sparse index from stored text payloads.

pub fn save_indexes(&self) -> Result<()>

Persist both the dense index and the sparse index to disk.

pub fn recover_from_wal(&mut self) -> Result<usize>

pub fn checkpoint(&mut self) -> Result<()>

pub fn stats(&self) -> Result<IndexStats>