Hybrid search and ranking with deterministic scoring for khive.

This crate provides:

• HNSW vector search with DeterministicScore output
• BM25 keyword search for exact matches
• Reciprocal Rank Fusion (RRF) for hybrid search
• Graph traversal for relationship-aware retrieval

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                      khive-retrieval                             │
│  ┌───────────┐  ┌───────────┐  ┌───────────┐  ┌───────────┐    │
│  │   hnsw/   │  │   bm25/   │  │  graph/   │  │  fusion/  │    │
│  │ (vector)  │  │ (keyword) │  │(traversal)│  │   (RRF)   │    │
│  └───────────┘  └───────────┘  └───────────┘  └───────────┘    │
│                       │                                          │
│                       ▼                                          │
│               ┌───────────────┐                                  │
│               │    hybrid/    │                                  │
│               │   (unified)   │                                  │
│               └───────────────┘                                  │
│                                                                  │
│  Inputs: Query + optional embedding + optional start nodes       │
│  Outputs: Vec<(Id, DeterministicScore)>                         │
└─────────────────────────────────────────────────────────────────┘

Design Principles

Deterministic Scoring (ADR-002)

All scores use DeterministicScore from khive-score for:

• Cross-platform identical rankings (x86_64, ARM64, WASM)
• Ord implementation (sortable, usable in BTreeSet)
• Hash implementation (cacheable)

Index Management (ADR-003)

• HNSW: Hierarchical Navigable Small World graphs for ANN search
• BM25: Okapi BM25 for keyword relevance
• Both support incremental updates with periodic rebuild

Graph Traversal (ADR-004)

• BFS for level-by-level exploration
• DFS for deep path exploration
• Bidirectional BFS for shortest path

ID Types and Bridging

Each retrieval module uses a different ID type:

The [fusion::fuse] function is generic over the ID type, so hybrid search that combines results from different modules requires a common representation. Bridging strategies:

1. String-based: Convert all IDs to String before fusion.
2. DocumentId-based: Convert EmbeddingId to DocumentId via DocumentId::new(embedding_id.to_string()).
3. Application-level mapping: Maintain a bidirectional lookup table between ID types in the application layer.

See [DocumentId] for details on the newtype and conversion traits.

Quick Start

use khive_retrieval::{VectorSearch, KeywordSearch, HybridSearcher, Query, HybridConfig};

// Implement granular traits independently:
// - VectorSearch for embedding-based search (HNSW)
// - KeywordSearch for text-based search (BM25)
// - HybridSearcher for combined search (requires both)
// - Reranker for post-retrieval reranking (standalone)

// Example: keyword-only search
let results = searcher.keyword_search("distributed systems", 10).await?;

// Example: hybrid search (vector + keyword with fusion)
let query = Query::hybrid("distributed systems", embedding_vec);
let config = HybridConfig::new(10);
let results = searcher.hybrid_search(&query, &config).await?;

for (id, score) in results {
    println!("{}: {}", id, score);
}