Struct LlmContextCompactor 

pub struct LlmContextCompactor<P: LlmProvider + ?Sized, H: AgentHooks = DefaultHooks> { /* private fields */ }

LLM-based context compactor.

Uses the LLM itself to summarize older messages into a compact form.

Budgets

The compactor performs no budget evaluation of its own: it may issue up to two summarization LLM calls per compaction (the second only when the first summary was truncated, retried with a doubled token budget) before the agent loop’s next UsageLimits boundary check runs. Every call’s usage — including failed attempts — is reported via CompactionResult::llm_usage / FailedCompaction::llm_usage and folded by the loop immediately after compaction, so the overshoot is bounded and consistent with the loop’s boundary-check semantics.

P is ?Sized so callers can hold an Arc<dyn LlmProvider> — useful when the provider is resolved dynamically per-thread (e.g. inside agent-server’s daemon worker, where the same compactor type wraps whichever concrete provider the host’s resolver picks). Concrete-type users (Arc<AnthropicProvider>, etc.) still work unchanged.

Implementations

impl<P: LlmProvider + ?Sized> LlmContextCompactor<P>

pub fn new(provider: Arc<P>, config: CompactionConfig) -> Self

Create a new LLM context compactor.

pub fn with_defaults(provider: Arc<P>) -> Self

Create with default configuration.

impl<P: LlmProvider + ?Sized, H: AgentHooks> LlmContextCompactor<P, H>

pub fn with_guardrail_hooks<H2: AgentHooks>( self, hooks: Arc<H2>, ) -> LlmContextCompactor<P, H2>

Apply the run’s guardrail hooks to every summarization LLM call.

pre_llm_request runs before the call (Proceed/Modify apply; Block aborts the compaction attempt with an error) and on_llm_response runs on the produced summary (Accept applies; Block and RetryWithFeedback abort the compaction attempt — the compactor never retries a rejected summary, so a deterministically-rejecting hook cannot start a paid retry loop here). An aborted compaction surfaces as a compact_history error; the agent loop then continues with the uncompacted history (threshold trigger) or fails the recovery (overflow trigger).

pub const fn config(&self) -> &CompactionConfig

Get the configuration.

pub fn with_prompts( self, system_prompt: impl Into<String>, summary_prompt_prefix: impl Into<String>, summary_prompt_suffix: impl Into<String>, ) -> Self

Override the prompts used for LLM-based summarization.

Trait Implementations

impl<P: LlmProvider + ?Sized, H: AgentHooks> ContextCompactor for LlmContextCompactor<P, H>

fn compact<'life0, 'life1, 'async_trait>( &'life0 self, messages: &'life1 [Message], ) -> Pin<Box<dyn Future<Output = Result<String>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Compact a list of messages into a summary.

fn estimate_tokens(&self, messages: &[Message]) -> usize

Estimate tokens for a message list.

fn needs_compaction(&self, messages: &[Message]) -> bool

Check if compaction is needed.

fn compact_history<'life0, 'async_trait>( &'life0 self, messages: Vec<Message>, ) -> Pin<Box<dyn Future<Output = Result<CompactionResult>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Perform full compaction, returning new message history.

fn compact_history_with_usage<'life0, 'async_trait>( &'life0 self, messages: Vec<Message>, ) -> Pin<Box<dyn Future<Output = Result<CompactionResult, FailedCompaction>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Like compact_history, but a failure additionally reports the provider-billed usage of any summarization calls already made, so callers can account billed-but-wasted spend.