Skip to main content

Module postprocess

Module postprocess 

Source
Expand description

Gateway output post-processor (deeper addon integration).

The single seam where lean-ctx’s own context-engineering is applied to downstream MCP tool output. Invoked once from super::proxy, right after crate::core::addons::runtime::scrub_output has removed secrets — so everything here operates on already-sanitized text.

Three independent, config-gated layers (all default off → pure pass-through):

  • L1 compress (compress, #1093): format-aware shrink of the returned text to a token budget.
  • L2 handle/spill (spill, #1094): oversized output → content- addressed archive + a ctx_expand handle instead of the full blob.
  • L3 index (index, #1095): side-channel consolidation into BM25 / property graph / knowledge so the output is searchable later.

Determinism (#498): L1/L2 are pure functions of (content, budget); L3 is a background side-channel that never touches the returned string.

Modules§

compress
L1 output compression (#1093): shrink downstream tool output to a token budget before it reaches the model.
index
L3 consolidation (#1095): side-channel — feed downstream output through the same pipeline as provider data so it becomes searchable (BM25), linked (property-graph cross-source edges from file references), and remembered (knowledge). Runs on a background thread and never touches the text returned to the model, so it cannot perturb output determinism (#498).
spill
L2 handle/spill (#1094): when downstream output is larger than the budget, store the verbatim blob in the content-addressed archive and hand the model a compact summary plus a ctx_expand retrieval handle instead of the full payload. This generalizes what Repomix (outputId) and Headroom (CCR) each do for themselves to every addon, through one lean-ctx retrieval path.

Functions§

process
Apply the configured output post-processing to one scrubbed downstream result, returning the (possibly transformed) text for the model.