Skip to main content

Module context_overhead

Module context_overhead 

Source
Expand description

Honest accounting of the fixed per-turn context lean-ctx injects (GitHub #361).

Three components ride every request and — on a provider WITHOUT prompt caching — are re-billed on every turn:

  • the exposed MCP tool schemas (description + input schema of each tool),
  • the MCP server instructions block, and
  • the rules block lean-ctx writes into the host’s instruction file (CLAUDE.md / AGENTS.md).

lean-ctx gain measures compression on lean-ctx-touched reads — its denominator is lean-ctx traffic, not the provider bill. On a phase-isolated / non-caching workload (separate process per phase, no provider cache) the cached-re-read lever has no surface, so the headline can read net-positive while the bill moved net-negative. Surfacing this overhead — and stating the denominator — keeps the meter honest.

Net bill impact ≈ gross_saved_tokens − total_tokens() × turns.

Structs§

ContextOverhead
A measured breakdown, in tokens, of the per-turn context lean-ctx adds.

Functions§

net_of_injection
Pure net-of-injection reconciliation: the total injection tax (overhead_per_turn × turns) and the signed net savings after subtracting it. Lives here — the home of injection accounting — so both lean-ctx gain and the verified savings ledger/ROI reconcile against the same math. The net is signed because on a non-caching rail a short run can legitimately go net-negative until savings outgrow the per-turn injection (#361, #685).
observed_turns
Provider turns (requests) the proxy actually observed carrying the injected prefix. The proxy is the only component that sees every provider turn, so its persisted request count is the honest multiplier for the per-turn injection tax. 0 when the proxy is not in the request path — we never guess turns we did not see, so net_of_injection then collapses to the gross savings.
tool_tokens
Description + input-schema tokens for one tool definition — exactly the two fields a client re-sends in every request’s tool list.