Crate lateweave 

Composable candidate generation and optional late-interaction reranking.

Query -> CandidateGenerator -> [Reranker] -> deterministic top-k

A Query is raw text plus named Features, each stamped with the Representation that produced it. Stages name documents by DocumentKey, the corpus and ID the system of record gives them, and each reads its own indexes, which writers anywhere move independently: every stage result says how fresh it is, and SearchPipeline ranks only the documents every stage holds and reports the oldest commit it read. MaxSimReranker scores MultiVectorSources, one per corpus, such as a VectorStore, which reads what a VectorStoreWriter or any other writer of its format commits.

Structs

Candidate

One gathered document; provenance names what produced it.

DocumentKey

A document as every stage names it: the corpus it belongs to and the ID the system of record gives it. Engines map keys to their own internal positions privately, so they never have to agree on one.

Feature

One query representation stamped with the encoder that produced it.

Gathered

What a gatherer found, and how fresh the index it searched was: every write committed to that index before as_of is reflected in it.

MaxSimReranker

MaxSim between the query’s token matrix and each candidate’s document, read from the source of the candidate’s corpus.

PackedDocuments

Token vectors of documents: a row-major float32 [sum(lengths), dimension] matrix holding each document’s tokens contiguously, in lengths order.

Query

Raw text plus the named features stages may consume.

RankedDocument

Representation

Identity of the encoder that produced a vector feature.

ResourceBudget

Bounded execution is caller policy; each reranker maps it onto its own representation.

Scored

scores[i] scores candidate i, or is None when the reranker’s index does not hold that document: not written to it yet, or already deleted. Every write committed to that index before as_of is reflected in it.

SearchDiagnostics

SearchPipeline

A gatherer, optionally followed by a reranker.

SearchRequest

Per-search parameters.

SearchResult

SearchTimings

StoreView

One commit of a VectorStore. It stays readable after later commits.

Subset

The documents a search is restricted to, by corpus. A corpus it does not name contributes no documents; an ID an index does not hold is ignored. Cloning shares each corpus’s restriction rather than copying its IDs.

TokenMatrix

A row-major float32 [tokens, dimension] matrix.

VectorStore

Reads a vector store, following what its writer commits.

VectorStoreWriter

Stages appends, deletes, and compactions of one store, and publishes them together on commit.

Enums

Encoding

Error

RankingError

Restriction

Which documents of one corpus a search may return.

Constants

DEFAULT_FEATURE

The feature name a MaxSimReranker reads unless told otherwise.

MANIFEST_FILE

STORE_FORMAT

The format every manifest this version reads declares.

Traits

CandidateGenerator

First stage: selects candidate documents.

FeatureValue

Whatever an encoder produced for the query text: a token matrix, a dense vector, a sparse weighting. Stages downcast to the type they consume.

MultiVectorSource

Where a MaxSim reranker reads one corpus’s document vectors from.

Reranker

Second stage: one qualified score, or None, for every candidate it is given, in candidate order, read from one consistent state of each index it scores. A candidate from a corpus the reranker cannot score at all is an error, not None. Gather scores never influence a reranked result.

VectorView

One consistent state of a MultiVectorSource: the vectors a document ID names never change for the life of the view.

Functions

maxsim_scores

MaxSim of one query against packed documents.

Type Aliases

Requirements

Feature name to the representation a stage was built for.

Result