Expand description
Cache-safe, cross-provider reasoning-effort control (#834).
Operators pin a single reasoning-effort level (proxy.effort) that lean-ctx
translates to each provider’s native parameter. Unlike per-turn “effort
routing” — which changes the effort between turns of one conversation and
thereby invalidates the provider prompt cache (OpenAI lists “changes to
reasoning effort” as a cache-invalidation cause; Anthropic breaks message
cache breakpoints on thinking-mode/config changes) — this value is a
constant: identical on every request, so the cached prefix stays
byte-stable (#448/#498) and only the model’s reasoning depth changes.
Safety rules, enforced by every applier:
- Opt-in: the caller only invokes an applier when
proxy.effortis set; off is a strict no-op that preserves the byte-unchanged meter-only path. - Never override the client: an effort the client set explicitly is left untouched, so the request keeps the client’s own cache key.
- Never enable reasoning the client didn’t ask for: the Anthropic applier
only dials an existing adaptive request, so it never adds thinking tokens
(or a 400) where the client wanted none. OpenAI reasoning models always
reason, so setting the level only ever caps/redirects existing reasoning.
The Gemini applier excludes 2.5 flash-lite (thinking off by default) for
the same reason, and never sends both
thinkingLevelandthinkingBudget. - Model-gated: models that would reject the parameter are skipped, so the
feature can never turn a working request into a 400. Gemini’s generation is
read from the URL path (
thinkingLevelon 3.x,thinkingBudgeton 2.5). - Deterministic: the rewrite is a pure function of
(document, level), so identical requests stay byte-identical across turns.
Structs§
- Effort
Stats - Point-in-time view of the effort control for
/status: the active level (so an operator can confirmproxy.effortis live) plus how many requests have been steered per provider. Pair the counters with the per-modelreasoning_tokensthe usage meter already records to see the realized output-token savings.
Functions§
- apply_
anthropic - Set
output_config.efforton an Anthropic request, but only when the client already requested adaptive thinking (thinking.type == "adaptive"). That guard means lean-ctx never enables thinking the client didn’t ask for (no surprise reasoning cost) and never sends adaptive config to a model that rejects it (the client already proved the model supports it). No-op when the client already setoutput_config.effort. - apply_
google - Set the Gemini thinking control that matches
model’s generation on agenerateContentrequest (thinkingLevelfor 3.x,thinkingBudgetfor 2.5).modelcomes from the request URL path — Gemini carries it there, not in the body — threaded in by the Google handler. No-op when the model is unknown/excluded, when the client already pinned either thinking field (never override, and never end up with both → 400), or on an unexpected body shape. - apply_
openai_ chat - Set
reasoning_efforton an OpenAI Chat Completions request. No-op when the client already set it or the model is non-reasoning. Returns whether the document changed. - apply_
openai_ responses - Set
reasoning.efforton an OpenAI Responses request (nested object). No-op when the client already pinnedreasoning.effortor the model is non-reasoning. Any otherreasoning.*fields (e.g.summary) are preserved. - openai_
supports_ effort - Whether an OpenAI model accepts a reasoning-effort parameter. Reasoning
models (the o-series and the gpt-5/gpt-6 families, including any
codexbuild) do; the non-reasoninggpt-4*/gpt-3*models and the*-chat-latestnon-reasoning variants reject it with a 400, so they are excluded. A vendor-prefixed name from an OpenAI-compatible gateway (openai/gpt-5.5,openrouter/openai/o3) is reduced to its bare model segment first. - snapshot
- Snapshot the counters together with the currently resolved effort level
(
None→"off").