Expand description
Conversation-history compaction.
Rust equivalent of (a self-contained subset of) upstream _compaction.py
(see UPSTREAM_DRIFT.md §9). Upstream’s _compaction.py is a large
(1500+ line), annotation-driven system: it groups messages into logical
spans (system / user / assistant-text / tool-call), stamps grouping and
token-count metadata onto Message.additional_properties, and never
deletes history — it flags messages _excluded and lets the client filter
them out when building the payload sent to the model. It ships seven
strategies (Truncation, SlidingWindow, SelectiveToolCall,
ToolResult, LLM-backed Summarization, TokenBudgetComposed,
ContextWindow) plus a CompactionProvider(ContextProvider) that wires a
strategy into the client’s get_response loop.
This module delivers that surface in two layers. The message-list
layer: the Tokenizer and CompactionStrategy abstractions and the
non-LLM strategies Truncation, SlidingWindow, TokenBudget,
SelectiveToolResult / ToolResultCompactionStrategy and
ContextWindowCompactionStrategy, which return a reduced list rather
than annotating messages in place. The group layer (index): the
CompactionMessageIndex group model, CompactionTriggers,
PipelineCompactionStrategy, TriggeredCompactionStrategy and the
LLM-backed SummarizationCompactionStrategy, over the asynchronous
AsyncCompactionStrategy every synchronous strategy also implements.
CompactionProvider runs either kind before each run.
Compaction never errors on content: given any message list it returns a (possibly unchanged) retained subset that satisfies the strategy’s constraint.
Modules§
- index
- The message-group model, triggers, composition and LLM summarization.
Structs§
- Approx
Tokenizer - A dependency-free default tokenizer using a ~4-characters-per-token
heuristic. Mirrors upstream’s
CharacterEstimatorTokenizer. - Compaction
Message Group - An atomic run of messages a strategy keeps or drops as a whole.
- Compaction
Message Index - A conversation split into
CompactionMessageGroups. See the module docs. - Compaction
Provider - A
ContextProviderthat compacts the accumulated message list — typically the run’s history, once aHistoryProviderhas prepended it inbefore_run— down to fit aCompactionStrategy’s constraint before it reaches the model. Rust equivalent of (a subset of) upstream’sCompactionProvider(see module docs andUPSTREAM_DRIFT.md§9). - Compaction
Triggers - Ready-made
CompactionTriggers (upstreamCompactionTriggers), all measured over the included groups. - Context
Window Compaction Strategy - Token-budget compaction from a model’s context window (upstream
ContextWindowCompactionStrategy): the input budget is the window less the output reservation; once the conversation passestool_eviction_thresholdof it, older tool results are collapsed (the lastkeep_last_tool_call_groupskept), and if it then still passestruncation_threshold, the oldest messages are dropped down to the eviction level. - Pipeline
Compaction Strategy - Runs strategies in order, each on the previous one’s output (upstream
PipelineCompactionStrategy). - Selective
Tool Result - Replace the payload of
Content::FunctionResult(tool-result) content in all but the lastkeep_lastmessages that carry tool results — they are the bulkiest and least useful once stale. Text and other content is left intact. - Sliding
Window - Keep leading system message(s) + the last
windownon-system messages. Mirrors upstream’sSlidingWindowstrategy. - Summarization
Compaction Strategy - Summarizes the oldest groups with a chat client once the conversation
grows past
target_count + thresholdnon-system messages, keeping abouttarget_countof the newest (upstreamSummarizationStrategy). - Token
Budget - Keep leading system message(s), then walk from the newest message
backward accumulating token counts, keeping messages until adding the
next would exceed
max_tokens. Returns the kept messages in original order. Mirrors upstream’sContextWindow/token-budget strategy. - Tool
Result Compaction Strategy SelectiveToolResultwith a custom stand-in (upstreamToolResultCompactionStrategy’sToolCallFormatter): all but the lastkeep_lasttool results have their payload replaced by what the formatter returns, keeping each exchange paired.- Triggered
Compaction Strategy - Runs a strategy only when its trigger fires (the trigger gating of upstream’s strategy base class).
- Truncation
- Keep the most recent
max_messages, always preserving any leading system message(s) at the front. Mirrors upstream’sTruncationstrategy.
Enums§
- Compaction
Group Kind - What a
CompactionMessageGroupholds.
Constants§
- COMPACTION_
SUMMARY_ KEY - The
Message::additional_propertieskey marking a summary messageSummarizationCompactionStrategywrote, with the ids of the messages it replaced ({"summary_of_message_ids": [...]}). - DEFAULT_
SUMMARIZATION_ PROMPT - The default summarization instruction, as upstream.
- DEFAULT_
SUMMARY_ INPUT_ TOKEN_ BUDGET - The default token budget for a summarization request.
- OMITTED_
TOOL_ RESULT - Stand-in payload left in place of a tool result this strategy compacts.
Traits§
- Async
Compaction Strategy - The asynchronous form of
CompactionStrategy, for strategies that call out (summarization). EveryCompactionStrategyis one. - Compaction
Strategy - A strategy that reduces a message list to fit some constraint.
- Tokenizer
- Counts tokens for a piece of text. Rust equivalent of upstream
TokenizerProtocol.
Functions§
- compact
- Compact
messageswithstrategyandtokenizer. - count_
message_ tokens - Sum the token counts of a message’s text content (text and reasoning
content items) using
tokenizer.
Type Aliases§
- Compaction
Trigger - Decides, from a conversation’s
CompactionMessageIndex, whether a strategy should run. - Tool
Result Formatter - Formats the stand-in for a compacted tool result: called with the name of the tool that produced it (when its call is in the conversation) and the result.