Struct CompactionService 

pub struct CompactionService { /* private fields */ }

Service for LLM-driven conversational context compaction.

When conversation history approaches a model’s context window, this service summarises older messages via a smaller/cheaper “compaction” LLM call. The resulting anchored summary preserves semantic continuity while freeing tokens for the main provider.

Supports both proactive compaction (before each turn) and reactive compaction (after a provider context-overflow error). A circuit breaker gates repeated failures to avoid cascading costs.

Implementations

impl CompactionService

pub fn new(providers: ProviderRegistry, config: CompactionConfig) -> Self

Creates a new compaction service.

pub fn config(&self) -> &CompactionConfig

Returns the compaction configuration.

pub async fn compact_if_needed( &self, messages: &[MessageRecord], model_context: u32, max_output: u32, store: &dyn SessionStore, session_id: Uuid, ) -> RuntimeResult<Option<CompactionResult>>

Proactive compaction: called before each provider turn to check whether the conversation will overflow the context window.

Returns Ok(None) when compaction is not needed. Returns Ok(Some(result)) when compaction was performed successfully.

Errors

Returns RuntimeError when the compaction LLM call or storage fails.

pub async fn compact_after_overflow( &self, messages: &[MessageRecord], model_context: u32, max_output: u32, store: &dyn SessionStore, session_id: Uuid, ) -> RuntimeResult<CompactionResult>

Reactive compaction: called after the provider returns a context overflow error. This always performs compaction (does not check is_overflow first).

Uses model_context to compute a safe crate::policy::CompactionConfig::keep_tokens override so the compaction prompt itself does not exceed the compaction model’s context window.

Errors

Returns RuntimeError when the compaction LLM call or storage fails.