Expand description
The calibration surface — the single documented home for the constants that shape ranking and
confidence. Moving any value here (or an inline boost listed in the REGISTRY below) moves the
frozen eval baselines, so every change is a reviewed diff against evals/baseline.json plus, in
the ideal, a re-freeze justification.
Why this module exists (D4): the 2026-07-04 gate incident showed that scoring constants
scattered as inline literals across a 1200-line retrieval.rs are the “constants nobody
remembers choosing” — a change to one, hidden inside a refactor, silently drifted the baselines.
The band thresholds and the ambiguity ratio live here as named constants; the remaining inline
boosts are catalogued in the REGISTRY so a reviewer can see the whole surface in one place, with
the test that pins each.
§REGISTRY — the full calibration surface (value · effect · pinning test)
Housed here (named constants):
BAND_STRONG_MIN= 45 — score ≥ this is Strong. Test:bands_are_calibrated_to_score_distribution.BAND_WEAK_MIN= 25 — score ≥ this (and < strong) is Weak; below is Fallback. Same test.AMBIGUITY_RATIO= 0.7 — runner-up ≥ ratio × top demotes a Strong top to Ambiguous. Test:calibrate_demotes_strong_top_when_runner_up_is_within_ambiguity_ratio.
Still inline in retrieval.rs / retrieval/scoring.rs (candidates for future extraction; each
documented at its use site):
- BM25F params (k1=1.2, b=0.75, weights [5,8,2,6,4,3]) — now the injected
crate::retrieval::RankerConfigDefault. Test:ranker_config_is_injected_not_read_from_env. - exact whole-query boost (+80) / partial substring (+6). Test: se-corpus
exact-lookupgolden. - intent boost (+20 × intents; generic-code gate → +6/0). Tests:
generic_code_intent_does_not_make_body_only_helper_strong+code-intentgolden. - name boost (+20, kind-gated for code). Covered via ground tests + the monotonicity pin.
- relation-context (+12 × hits) / directional relation (+48 + 8 × endpoint) / code-action
(+28 × hits + kind 8–16 + fn 18). Tests:
directional_relation_*,code_action_*. matched_identityadditive tiebreak (+1/term). Covered via ground tests.- route penalty for body-only scatter (× coverage, floor 0.25). Test:
route_score_penalizes_partial_body_only_mentions. - anchor arms:
id_coverage ≥ 0.4+ name-grade +known_coverage ≥ 0.6. Tests:mostly_unknown_query_does_not_anchor_partial_name_match,anchored_typo_match_*. - canonical dominance (parent → best_child + 1). Test:
canonical_dominance_lifts_a_parent_*. apply_floor(drop trailing Fallback when a confident hit exists). Test:apply_floor_drops_trailing_fallback_only_when_a_confident_hit_exists.- band monotonicity invariant (a garbage term never raises a band). Test:
adding_a_garbage_term_never_raises_a_hits_band.
Constants§
- AMBIGUITY_
RATIO - How close a runner-up must be, as a fraction of the top score, to demote a Strong top to Ambiguous — so a coin-flip query stops reporting false confidence.
- BAND_
STRONG_ MIN - Score at or above which a hit bands Strong. Calibrated to the measured distribution: clean retrieval accuracy is 100% down to 45 and garbage tops out ~32, so 45 is discriminating yet keeps Strong high-accuracy and never labels garbage Strong.
- BAND_
WEAK_ MIN - Score at or above which a hit bands Weak (below Strong); under this it is Fallback.
- DISTINCTIVE_
NAME_ IDF - IDF at or above which an EXACT name-field match counts as a distinctive identity anchor — a
query term rare enough that naming a node with it is unambiguous. Fixes band conservatism on
common-vocabulary domains: “why did my CARBONARA turn into scrambled eggs” now anchors on
carbonara (idf ≈ 1.54 here) instead of being capped to Weak by the common co-terms, while “egg”
(idf ≈ 1.3, shared by many recipes) stays below the bar and cannot anchor. Empirically 1.5 is the
separation point: a distinctive dish/symbol name in a small cross-referenced corpus reads as
df≈3-4(recipe + its own section + a technique that lists it), so its idf sits ~1.5, while genuinely common terms fall under it. Gated additionally byknown_coverage ≥ 0.6, so an out-of-vocabulary garbage query cannot anchor on its one real term. Verified no-regression across all eval suites; only lifts bands that were honestly under-confident.