Skip to main content

Module ranking

Module ranking 

Source
Expand description

Cross-provider ranking policy (SPEC.md §6.6, F10).

score is provider-local and ordinal: it orders one provider’s frames against one query, and nothing in the protocol makes two providers’ numbers commensurable. A host still has to put the frames in some order — a prompt is a sequence, and a budget is spent front-first — so every host is making a cross-provider ranking decision whether or not it admits to one. F10’s rule is that the decision is the host’s policy, named as such.

RankingStrategy is where that policy lives. super::compose_for_prompt keeps ranking by raw score (ScoreDescending) so an existing host’s output does not move; super::compose_for_prompt_with takes any strategy, and two that need no configuration ship here:

  • RoundRobinByRank — every provider’s best frame, then every provider’s second, and so on. Uses only within-provider rank, where the ordering is meaningful.
  • PerProviderQuota — the same idea in blocks of k: each provider’s top k, then each provider’s next k.

§Why raw score starves a provider

Consider a semantic-search provider that reports cosine similarity in the 0.80.95 band and a lexical provider that reports a normalized BM25 rank topping out near 0.4. Both are honest about their own frames. Rank the union by raw score under a budget that fits four frames and the lexical provider contributes nothing — not because its evidence is worse, but because its retriever’s number is smaller. The prompt’s evidence set was decided by an implementation detail of someone else’s scorer. starvation_* in this module’s tests is that scenario, run.

§Determinism

Ranking must be a pure function of the input set: two runs over the same frames must produce the same order, or a host’s prompt stops being reproducible and its provider prompt cache stops hitting (super’s module docs). Every strategy here derives its provider ordering from a BTreeMap and breaks every remaining tie on the canonical FrameId, so no ordering ever depends on hash iteration or arrival order.

Structs§

PerProviderQuota
Give every provider its top k before any provider gets its k + 1th: each provider’s best k frames, then each provider’s next k, and so on.
RoundRobinByRank
Interleave providers by within-provider rank: every provider’s best frame first, then every provider’s second-best, and so on.
ScoreDescending
Rank the whole mixed set by raw score, descending — the reference host’s documented default (SPEC.md §6.6), and what super::order_by_value has always done.

Traits§

RankingStrategy
A host’s policy for ordering frames drawn from more than one provider — the choice SPEC.md §6.6 (F10) says a host owns and must name.

Functions§

is_ranking_permutation
Whether order is exactly the indices 0..n, each once — the RankingStrategy contract, as an assertion a strategy’s own tests can make.
rank_with
Apply a RankingStrategy, returning the frames best-first.