Module compaction 

Conversation-history compaction.

Rust equivalent of (a self-contained subset of) upstream _compaction.py (see UPSTREAM_DRIFT.md §9). Upstream’s _compaction.py is a large (1500+ line), annotation-driven system: it groups messages into logical spans (system / user / assistant-text / tool-call), stamps grouping and token-count metadata onto Message.additional_properties, and never deletes history — it flags messages _excluded and lets the client filter them out when building the payload sent to the model. It ships seven strategies (Truncation, SlidingWindow, SelectiveToolCall, ToolResult, LLM-backed Summarization, TokenBudgetComposed, ContextWindow) plus a CompactionProvider(ContextProvider) that wires a strategy into the client’s get_response loop.

This module intentionally delivers a smaller, dependency-free surface: the Tokenizer and CompactionStrategy abstractions upstream defines, plus four concrete, non-LLM strategies that mirror upstream’s Truncation, SlidingWindow, ContextWindow/TokenBudget, and ToolResult (renamed SelectiveToolResult here to avoid confusion with Content::FunctionResult… “tool result” is the plain-English name). Compaction here works by returning a reduced list rather than annotating messages in place — simpler, and sufficient for the strategies included. Wiring a strategy into the client’s get_response loop (upstream’s CompactionProvider) is intentionally out of scope for this change; see UPSTREAM_DRIFT.md §9.

Compaction never errors on content: given any message list it returns a (possibly unchanged) retained subset that satisfies the strategy’s constraint.

Structs

ApproxTokenizer

A dependency-free default tokenizer using a ~4-characters-per-token heuristic. Mirrors upstream’s CharacterEstimatorTokenizer.

CompactionProvider

A ContextProvider that compacts the accumulated message list — typically the run’s history, once a HistoryProvider has prepended it in before_run — down to fit a CompactionStrategy’s constraint before it reaches the model. Rust equivalent of (a subset of) upstream’s CompactionProvider (see module docs and UPSTREAM_DRIFT.md §9).

SelectiveToolResult

Drop Content::FunctionResult (tool-result) content from all but the last keep_last messages that carry tool results — they are the bulkiest and least useful once stale. Text and other content is left intact. Messages that become empty after stripping are dropped entirely. Mirrors upstream’s ToolResult strategy.

SlidingWindow

Keep leading system message(s) + the last window non-system messages. Mirrors upstream’s SlidingWindow strategy.

TokenBudget

Keep leading system message(s), then walk from the newest message backward accumulating token counts, keeping messages until adding the next would exceed max_tokens. Returns the kept messages in original order. Mirrors upstream’s ContextWindow/token-budget strategy.

Truncation

Keep the most recent max_messages, always preserving any leading system message(s) at the front. Mirrors upstream’s Truncation strategy.

Traits

CompactionStrategy

A strategy that reduces a message list to fit some constraint.

Tokenizer

Counts tokens for a piece of text. Rust equivalent of upstream TokenizerProtocol.

Functions

compact

Convenience free function: compact messages with strategy and tokenizer.

count_message_tokens

Sum the token counts of a message’s text content (text and reasoning content items) using tokenizer.