Skip to main content

Module embed

Module embed 

Source
Expand description

The dense projection as a kept child process (packset-embed), one per direction. An absent encoder is a supported state: every failure falls back to the lexical scorers. A vector is derivable from the text, never the store.

Constants§

BIN_VARS
Environment variables naming the encoder.
RERANK_DEPTH
How deep the second stage reads: the deepest cut-off the locomo table scores.

Functions§

apply_rerank
Reorder the head of a ranking by cross-encoder scores. scores must cover RERANK_DEPTH.min(hits.len()); the tail keeps its order; ties are stable.
binary
The encoder binary this seat would run: PACKSET_EMBED, else beside the writer, else on PATH.
encode
encode_document
Encode one atom’s text.
encode_late
Encode one text three ways from one pass: a vector per token, the pooled vector, and learned term weights. Read by the retrieval benchmark only.
encode_query
Encode one query.
encode_sparse
Learned term weights from SPLADE (doi:10.1145/3404835.3463098), a model trained for the weights, as opposed to the sparse head encode_late returns beside a dense vector. Read by the benchmark, not the writer.
requested
Whether one request runs the stage: the query flag over the host default.
rerank
Cross-encoder scores for every candidate against the question (doi:10.48550/arXiv.1901.04085), in the caller’s order. Not the panel’s rerank, which is diversification.
rerank_hits
Reorder the top of a ranking by a cross-encoder; None when there is no working reranker, so the caller keeps the ranking and says so.
wanted
Whether the live search path runs the second stage by default. Off unless asked; the same spellings /v1/search?rerank= accepts.
warm_queries
Start every query encoder now, side by side, so the first agents to ask at once do not each pay a model load. Each probe holds one slot while it runs, which is what makes the pool spread rather than stack.

Type Aliases§

Sparse
Learned term weights: which vocabulary entries a text activates, and how much. Ascending by index, so two of them intersect in one pass.