Struct SearchConfig 

pub struct SearchConfig {
    pub top_k: usize,
    pub ef_search: usize,
    pub pruning_threshold: f32,
    pub rerank_factor: Option<usize>,
    pub hybrid: Option<HybridConfig>,
    pub score_fn: Option<ScoreFn>,
    pub partition_filter: Option<String>,
    pub column_filter: Option<ColumnFilter>,
}

Fields

top_k: usize

ef_search: usize

pruning_threshold: f32

Maximum distance from query to file centroid edge for a file to be searched. Files where distance(query, centroid) - radius > pruning_threshold are skipped. Set to f32::INFINITY to disable pruning (scan all files).

rerank_factor: Option<usize>

When Some(factor), fetch top_k * factor candidates from the HNSW index and rerank them using exact F32 distances before truncating to top_k. Corrects the approximation error introduced by PQ-compressed HNSW distances. None (default) disables reranking.

hybrid: Option<HybridConfig>

Hybrid BM25+vector search configuration.

When set, the pipeline loads global IDF stats from the table’s BM25 stats file, fetches a larger candidate pool from HNSW (candidate_pool or 10 * top_k), scores each candidate with BM25 against query_text, then fuses vector distance and BM25 score via RRF (default) or linear combination.

The BM25 stats file (metadata/ailake_bm25_stats.bin) is populated automatically by TableWriter when bm25_text_column is configured. If absent, pure vector distances are used (BM25 scores default to 0).

score_fn: Option<ScoreFn>

Optional scoring function for hybrid ranking.

When set, the search pipeline reads the Parquet row for each HNSW candidate and calls score_fn(distance, &single_row_batch). The returned value replaces distance in SearchResult and determines final ranking (lower = better).

If rerank_factor is also set, score_fn receives the exact (non-approximated) distance from the reranking step.

Use ScoreFn::new(|d, row| ...) to construct. See ScoreFn docs for an example using hybrid_score with episodic memory columns.

partition_filter: Option<String>

Partition filter: only search files whose DataFileEntry::partition_value matches this string. None searches all files (no partition pruning). Set to agent_id in Agent.recall() for per-agent isolated search.

column_filter: Option<ColumnFilter>

Column-level predicate pushed down to the Parquet read of each surviving file, via AilakeFileReader::read_parquet_filtered (row-group statistics skip + exact Arrow RowFilter — see ailake-parquet::ParquetVectorReader). Applies only where a Parquet read already happens (flat-scan fallback, reranking, hybrid, score_fn, or equality-delete row checks) — it does not gate which files survive centroid/partition pruning, and it isn’t applied against columns only materialized later by SchemaFiller (schema-evolution defaults). None disables pushdown (default).

Implementations

impl SearchConfig

pub fn with_column_filter(self, filter: ColumnFilter) -> Self

pub fn with_pruning(self, threshold: f32) -> Self

pub fn with_reranking(self, factor: usize) -> Self

pub fn with_score_fn( self, f: impl Fn(f32, &RecordBatch) -> f32 + Send + Sync + 'static, ) -> Self

pub fn with_hybrid(self, cfg: HybridConfig) -> Self

Trait Implementations

impl Clone for SearchConfig

fn clone(&self) -> SearchConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SearchConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SearchConfig

fn default() -> Self

Returns the “default value” for a type.