Skip to main content

Module compaction

Module compaction 

Source
Expand description

Conversation-history compaction.

Rust equivalent of (a self-contained subset of) upstream _compaction.py (see UPSTREAM_DRIFT.md §9). Upstream’s _compaction.py is a large (1500+ line), annotation-driven system: it groups messages into logical spans (system / user / assistant-text / tool-call), stamps grouping and token-count metadata onto Message.additional_properties, and never deletes history — it flags messages _excluded and lets the client filter them out when building the payload sent to the model. It ships seven strategies (Truncation, SlidingWindow, SelectiveToolCall, ToolResult, LLM-backed Summarization, TokenBudgetComposed, ContextWindow) plus a CompactionProvider(ContextProvider) that wires a strategy into the client’s get_response loop.

This module delivers that surface in two layers. The message-list layer: the Tokenizer and CompactionStrategy abstractions and the non-LLM strategies Truncation, SlidingWindow, TokenBudget, SelectiveToolResult / ToolResultCompactionStrategy and ContextWindowCompactionStrategy, which return a reduced list rather than annotating messages in place. The group layer (index): the CompactionMessageIndex group model, CompactionTriggers, PipelineCompactionStrategy, TriggeredCompactionStrategy and the LLM-backed SummarizationCompactionStrategy, over the asynchronous AsyncCompactionStrategy every synchronous strategy also implements. CompactionProvider runs either kind before each run.

Compaction never errors on content: given any message list it returns a (possibly unchanged) retained subset that satisfies the strategy’s constraint.

Modules§

index
The message-group model, triggers, composition and LLM summarization.

Structs§

ApproxTokenizer
A dependency-free default tokenizer using a ~4-characters-per-token heuristic. Mirrors upstream’s CharacterEstimatorTokenizer.
CompactionMessageGroup
An atomic run of messages a strategy keeps or drops as a whole.
CompactionMessageIndex
A conversation split into CompactionMessageGroups. See the module docs.
CompactionProvider
A ContextProvider that compacts the accumulated message list — typically the run’s history, once a HistoryProvider has prepended it in before_run — down to fit a CompactionStrategy’s constraint before it reaches the model. Rust equivalent of (a subset of) upstream’s CompactionProvider (see module docs and UPSTREAM_DRIFT.md §9).
CompactionTriggers
Ready-made CompactionTriggers (upstream CompactionTriggers), all measured over the included groups.
ContextWindowCompactionStrategy
Token-budget compaction from a model’s context window (upstream ContextWindowCompactionStrategy): the input budget is the window less the output reservation; once the conversation passes tool_eviction_threshold of it, older tool results are collapsed (the last keep_last_tool_call_groups kept), and if it then still passes truncation_threshold, the oldest messages are dropped down to the eviction level.
PipelineCompactionStrategy
Runs strategies in order, each on the previous one’s output (upstream PipelineCompactionStrategy).
SelectiveToolResult
Replace the payload of Content::FunctionResult (tool-result) content in all but the last keep_last messages that carry tool results — they are the bulkiest and least useful once stale. Text and other content is left intact.
SlidingWindow
Keep leading system message(s) + the last window non-system messages. Mirrors upstream’s SlidingWindow strategy.
SummarizationCompactionStrategy
Summarizes the oldest groups with a chat client once the conversation grows past target_count + threshold non-system messages, keeping about target_count of the newest (upstream SummarizationStrategy).
TokenBudget
Keep leading system message(s), then walk from the newest message backward accumulating token counts, keeping messages until adding the next would exceed max_tokens. Returns the kept messages in original order. Mirrors upstream’s ContextWindow/token-budget strategy.
ToolResultCompactionStrategy
SelectiveToolResult with a custom stand-in (upstream ToolResultCompactionStrategy’s ToolCallFormatter): all but the last keep_last tool results have their payload replaced by what the formatter returns, keeping each exchange paired.
TriggeredCompactionStrategy
Runs a strategy only when its trigger fires (the trigger gating of upstream’s strategy base class).
Truncation
Keep the most recent max_messages, always preserving any leading system message(s) at the front. Mirrors upstream’s Truncation strategy.

Enums§

CompactionGroupKind
What a CompactionMessageGroup holds.

Constants§

COMPACTION_SUMMARY_KEY
The Message::additional_properties key marking a summary message SummarizationCompactionStrategy wrote, with the ids of the messages it replaced ({"summary_of_message_ids": [...]}).
DEFAULT_SUMMARIZATION_PROMPT
The default summarization instruction, as upstream.
DEFAULT_SUMMARY_INPUT_TOKEN_BUDGET
The default token budget for a summarization request.
OMITTED_TOOL_RESULT
Stand-in payload left in place of a tool result this strategy compacts.

Traits§

AsyncCompactionStrategy
The asynchronous form of CompactionStrategy, for strategies that call out (summarization). Every CompactionStrategy is one.
CompactionStrategy
A strategy that reduces a message list to fit some constraint.
Tokenizer
Counts tokens for a piece of text. Rust equivalent of upstream TokenizerProtocol.

Functions§

compact
Compact messages with strategy and tokenizer.
count_message_tokens
Sum the token counts of a message’s text content (text and reasoning content items) using tokenizer.

Type Aliases§

CompactionTrigger
Decides, from a conversation’s CompactionMessageIndex, whether a strategy should run.
ToolResultFormatter
Formats the stand-in for a compacted tool result: called with the name of the tool that produced it (when its call is in the conversation) and the result.