Module compression 

LLM-based context compression middleware.

Long tool-heavy conversations balloon the message list that gets sent to the LLM on every turn, which slows each call down (and, past the window, fails). This middleware observes the per-LLM-call message list in Middleware::on_pre_llm and, once it exceeds CompressionConfig::trigger_tokens, summarises the earlier portion of the conversation into a single compact message.

Design notes:

• It only mutates the per-call message copy (PreLlmCtx.messages), never the stored session history — the JSONL turn log keeps full fidelity.
• The cut between “old” and “recent” messages is tool-pairing safe: it never separates an Assistant{tool_calls} message from its Tool results, so the resulting message list stays valid for OpenAI-compatible APIs.
• Summaries are cached per (session, transcript) so repeated LLM calls within one turn (after each tool result) don’t re-pay the summarization LLM call.
• If summarization fails, the old block is dropped entirely rather than failing the turn — the block is the oldest context, so losing it is the graceful fallback.

Structs

CompressionConfig

Tuning knobs for SummarizingMiddleware.

SummarizingMiddleware

Compresses the earlier part of a long conversation via LLM summarization.