Struct MaxSimReranker 

pub struct MaxSimReranker { /* private fields */ }

MaxSim between the query’s token matrix and each candidate’s document, read from the source of the candidate’s corpus.

Every source must carry the same representation and score semantics, so scores from different corpora share one scale; the reranker requires its query feature to carry that representation, so vectors from a different encoder are refused before anything is fetched. Each rerank takes one view of every source, and a candidate its source’s view does not hold is unscored.

Implementations

impl MaxSimReranker

pub fn new( sources: impl IntoIterator<Item = Arc<dyn MultiVectorSource>>, feature: impl Into<String>, ) -> Result<Self>

Sources must be for distinct corpora.

Examples found in repository

examples/stored_maxsim.rs (line 96)

fn main() -> Result<()> {
    let representation = Representation::new("toy-encoder", "1", DIMENSION, true)?;
    let directory = std::env::temp_dir().join(format!("lateweave-example-{}", std::process::id()));
    let path = directory.join("vectors");

    // The indexing side: one token per word, the axis standing in for what
    // a real encoder produces.
    let mut writer =
        VectorStoreWriter::create(&path, Encoding::Float32, "fruit", representation.clone())?;
    let embeddings = [unit(0), unit(2), unit(1), unit(2), unit(0), unit(3)].concat();
    writer.append(["a", "b", "c"], &embeddings, DIMENSION, &[2, 2, 2], None)?;
    writer.commit()?;

    // The serving side.
    let gatherer = TermGatherer {
        requires: BTreeMap::new(),
        documents: vec![("a", "red apple"), ("b", "green apple"), ("c", "red car")],
        indexed_at: SystemTime::now(),
    };
    let store = Arc::new(VectorStore::open(&path)?) as Arc<dyn MultiVectorSource>;
    let reranker = MaxSimReranker::new([store], DEFAULT_FEATURE)?;
    let pipeline = SearchPipeline::new(Arc::new(gatherer), Some(Arc::new(reranker)));

    let query = Query::new("apple").with_feature(
        DEFAULT_FEATURE,
        Feature::lazy(representation, || {
            TokenMatrix::new(unit(1).to_vec(), DIMENSION)
        }),
    );
    let request = SearchRequest::new(10, 2).with_budget(ResourceBudget::new(4096, 64, Some(1))?);

    let print = |label: &str| -> Result<()> {
        let result = pipeline.search(&query, &request)?;
        println!("{label}:");
        for document in &result.documents {
            println!(
                "  #{} {:?} score {}",
                document.rank, document.key, document.score
            );
        }
        println!("  {:?}", result.diagnostics);
        Ok(())
    };
    print("before the delete")?;

    // The indexing side deletes "b" from the vectors; the gatherer still
    // finds it, and the pipeline drops it.
    writer.delete(["b"]);
    writer.commit()?;
    print("after the delete")?;

    std::fs::remove_dir_all(directory)?;
    Ok(())
}

pub fn feature(&self) -> &str

Trait Implementations

impl Reranker for MaxSimReranker

fn requires(&self) -> &Requirements

fn score_semantics(&self) -> &str

fn rerank( &self, query: &Query, candidates: &[Candidate], budget: &ResourceBudget, ) -> Result<Scored>

fn name(&self) -> &str

Reported in search diagnostics.