Skip to main content

Module verbosity

lean_ctx::proxy

Module verbosity 

Source
Expand description

Cache-safe wire verbosity steer (#895 Track B).

Opt-in (proxy.verbosity_steer). When on, the proxy appends a single constant “be concise” instruction to the last user turn of each request. Output-shaping for non-rules-aware clients: lean-ctx already steers verbosity through rules injection for editors that load rules, but raw API clients (and the holdout’s treatment arm) need it on the wire.

Cache-safety, by construction:

  • The steer is constant (identical bytes every turn), so it never adds per-turn entropy.
  • It is appended after all existing content of the last user turn — i.e. strictly after the last cache_control breakpoint — and never modifies a cache_control-anchored block. The provider’s cached prefix (everything up to the last breakpoint) stays byte-identical across turns (#448/#498); only the always-reprocessed tail grows by the constant suffix.
  • For array content the suffix is a new trailing text element; existing blocks (including cache anchors) are left untouched. For plain-string content (OpenAI prefix-cached; Anthropic strings carry no cache_control) the suffix is concatenated.
  • Idempotent: if the steer is already present it is not appended again.

Constants§

STEER
The constant verbosity instruction. Kept short and directive; its byte stability is what makes the steer prompt-cache-safe.

Functions§

apply_anthropic
Append the steer to the last user message of an Anthropic /v1/messages body. No-op (returns false) when there is no user turn.
apply_google
Append the steer to the last user turn of a Gemini generateContent body (contents[].parts[]).
apply_openai_chat
Append the steer to the last user message of an OpenAI Chat Completions body. OpenAI prefix-caches automatically, so appending to the newest turn never disturbs a previously cached prefix.
apply_openai_responses
Append the steer to the last user item of an OpenAI Responses body. input is either a plain string or an array of role/content items.
steered_count
Total requests that received a wire verbosity steer (telemetry).