Expand description
Token-budget selection over ranked context candidates (Wave 14B/15.1).
Four deterministic, no-panic selectors:
select_with_budget: greedy value/token knapsack overContextItems with a soft budget + hard maximum; required items are always kept (the caller compresses them when they alone exceed the hard max).select_in_order: theoptimizer-off variant — rank-order cut, no value/token reordering.mmr_diversify: Maximal Marginal Relevance — penalizes candidates similar to already-selected ones so one component/owner group cannot crowd out the rest.enforce_quotas: TOKEN-aware per-kind caps (fraction × available tokens, adapted to the pool’s running average token cost) over a ranked list, preserving rank order.
Functions§
- enforce_
quotas - Enforce per-kind quotas over a ranked list, preserving rank order —
TOKEN-aware (reviewer item 6): each kind’s cap is a token budget
fraction * available_tokens; a candidate is accepted only while accepting it would not push its kind’s accumulated spend over that cap. Because the cap is a token budget applied over actual per- candidate costs, the effective entry allowance adapts to the selected pool’s running average token cost (expensive candidates exhaust a kind’s allocation faster, cheap ones admit more entries). A candidate that would exceed its group cap is skipped — the next candidate of another group takes its place, so the remaining groups rebalance. - mmr_
diversify - Maximal Marginal Relevance selection: repeatedly pick the candidate
maximizing
lambda * score - (1 - lambda) * max_similarity(selected), up tobudgetitems.lambdais clamped to [0, 1]; 0.5 is the default (seemmr_diversify_default). Deterministic: ties keep the earlier candidate inrankedorder. - mmr_
diversify_ default mmr_diversifywith the spec’s defaultlambda = 0.5.- select_
in_ order - Budget selection in list order (the
optimizer-off ablation ofselect_with_budget): walk items in importance order, keep required items unconditionally, and accept pool items while the total token spend stays within both the softbudgetand thehard_maxceiling. No value/token reordering — a candidate’s rank order decides. - select_
with_ budget - Select indices into
itemswithin a tokenbudget. Required items come first and are never dropped (even when they alone exceed the budget — required context is never silently cut); the remaining budget is filled greedily by value/token, highest first. Deterministic: ties break on index order.