Skip to main content

Module context_overhead

Module context_overhead 

Source
Expand description

Honest accounting of the fixed per-turn context lean-ctx injects (GitHub #361).

Three components ride every request and — on a provider WITHOUT prompt caching — are re-billed on every turn:

  • the exposed MCP tool schemas (description + input schema of each tool),
  • the MCP server instructions block, and
  • the rules block lean-ctx writes into the host’s instruction file (CLAUDE.md / AGENTS.md).

lean-ctx gain measures compression on lean-ctx-touched reads — its denominator is lean-ctx traffic, not the provider bill. On a phase-isolated / non-caching workload (separate process per phase, no provider cache) the cached-re-read lever has no surface, so the headline can read net-positive while the bill moved net-negative. Surfacing this overhead — and stating the denominator — keeps the meter honest.

Net bill impact ≈ gross_saved_tokens − total_tokens() × turns.

Structs§

ContextOverhead
A measured breakdown, in tokens, of the per-turn context lean-ctx adds.

Constants§

NATIVE_BASELINE_TOKENS_PER_TURN
Estimated per-turn overhead of native IDE tools (Read, Grep, Shell, Glob, Write, StrReplace). lean-ctx stands in for them only where they are denied (Replace mode); see native_baseline_tokens_per_turn.

Functions§

native_baseline_tokens_per_turn
The native prefix lean-ctx’s own prefix replaces in the active hook mode. Only Replace denies the native tools; in Hybrid and MCP mode they stay in the host’s tool list and lean-ctx’s prefix adds to theirs, so nothing is subtracted.
net_of_injection
Net-of-injection reconciliation with baseline + cache corrections (#1104).
observed_turns
Provider turns (requests) the proxy actually observed carrying the injected prefix. The proxy is the only component that sees every provider turn, so its persisted request count is the honest multiplier for the per-turn injection tax. 0 when the proxy is not in the request path — we never guess turns we did not see, so net_of_injection then collapses to the gross savings.
prefix_cost_factor
What one prefix token costs per turn, in base-input tokens: hits read at CACHE_READ_MULTIPLIER, misses write at CACHE_WRITE_MULTIPLIER. --no-cache-adjust prices a provider without caching (1.0, #1104).
proactive_injected_tokens
Total proactive context tokens appended by this process.
provider_cache_hit_rate
Share of turns whose prefix the provider serves from its cache. An estimate (default 75%), set with dashboard_cache_hit_rate.
record_proactive_injection
Record tokens appended as proactive context so savings reports can account for the dynamic response-side injection separately from fixed overhead.
set_no_cache_adjust
tool_tokens
Description + input-schema tokens for one tool definition — exactly the two fields a client re-sends in every request’s tool list.