Skip to main content

Module hybrid

Module hybrid 

Source
Expand description

Hybrid search: the keyword arm, and its fusion with the vector arm (§5.9).

Dense vectors and keyword matching fail in opposite directions. An embedding finds a paraphrase and misses an exact identifier it never saw in training; BM25 finds the identifier and misses the paraphrase entirely. Reciprocal Rank Fusion combines them without either needing to know the other’s score scale, which is the property that makes it usable here: cosine distance and BM25 are not comparable numbers, and any scheme that adds them is inventing a conversion nobody measured. RRF adds ranks, which are comparable by construction.

Before this existed, reciprocal_rank_fusion was a pure function over two rank lists with nothing in the crate producing the keyword half and no FTS5 table in the schema — §9 budgeted hybrid search at ≤50 ms for a path that could not run. The fusion function is unchanged in substance; what is new is everything that feeds it.

Structs§

HybridHit
One fused result, with the evidence for its position.
HybridSearch
A hybrid search over one model’s vectors and the concept-text index (§5.9).

Constants§

RRF_K
The k in 1/(k + rank), from the paper and from §5.9.

Functions§

escape_fts5_query
Turn arbitrary user text into an FTS5 MATCH expression that cannot be a syntax error and cannot mean something the user did not write.
keyword_search
Keyword search over concept text, best match first (§5.9).