Struct SummarizeStrategy 

pub struct SummarizeStrategy { /* private fields */ }

Summarization trim strategy: over-budget old messages → summarizer → one System summary message.

Behavior

• Keep the most recent rounds verbatim: keep complete rounds from the tail forward until the budget is approached (the budget first deducts a reservation for the summary output, see with_summary_max_tokens); at least the most recent round is kept (kept even when a single round exceeds the budget; nothing more can be compacted);
• Incremental compaction: when over budget again, the previous summary message goes to the summarizer as input (prior summary), producing an integrated new summary — the summary is never compacted repeatedly (semantic drift), and information already compacted is not lost;
• Materialization: replace: true, the compaction result is written back to storage with zero recomputation afterwards;
• Failure degradation: when the summarizer call fails, degrade to window dropping (projection — storage keeps the original text, retried on the next over-budget retrieval); the conversation is not interrupted.

The summary message uses the System role and sits at the front of the sequence (a summary is not conversation content but history compaction; the System role also avoids consecutive User messages after compaction). The summary message’s tokens count toward the budget and obey the window trim rules like any other message.

Comparison

Compared to the default WindowDrop: window dropping is a lossless projection with cheap per-turn recomputation, suited to ad-hoc inspection; this strategy trades one summarizer call for a longer compactable span, suited to long conversations that must not exceed limits, at the cost of the summarizer call (cost + latency) and information loss.

Example

Inject the summarization strategy (using FakeProvider in place of a real summarizer to demonstrate the full flow):

use std::sync::Arc;
use molo::memory::{Memory, SummarizeStrategy, WindowMemory};
use molo::{FakeProvider, FakeReply, Message};

#[tokio::main]
async fn main() -> Result<(), molo::memory::MemoryError> {
    let fake = Arc::new(FakeProvider::new([FakeReply::Text("Key points from earlier rounds".into())]));
    let mut memory = WindowMemory::new(30)
        .with_strategy(Arc::new(SummarizeStrategy::new(fake)));
    for i in 1..=4 {
        memory.record(Message::user(format!("Question from round {i}"))).await?;
        memory.record(Message::assistant(format!("Answer from round {i}"))).await?;
    }

    // 44 tokens total > budget 30: compacted to [summary, most recent round].
    let context = memory.context().await?;
    assert_eq!(context.len(), 3);
    assert!(matches!(context[0], Message::System(_)));
    assert_eq!(context[1], Message::user("Question from round 4"));
    Ok(())
}

Implementations

impl SummarizeStrategy

pub fn new(provider: impl Provider + 'static) -> Self

Creates a summarization strategy with the default prompt and a default summary output cap (1024 tokens).

pub fn with_prompt(self, prompt: impl Into<String>) -> Self

Replaces the default summarization prompt (instruction + requirements; the history text is appended by the strategy).

pub fn with_summary_max_tokens(self, max_tokens: u32) -> Self

Sets the token cap for the summary output (default 1024).

The value doubles as the budget reservation for kept rounds: the kept rounds’ token total stays within “budget − summary cap”, leaving room for the summary output.

Trait Implementations

impl Debug for SummarizeStrategy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl TrimStrategy for SummarizeStrategy

fn trim<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, messages: &'life1 [Message], budget: &'life2 Budget, counter: &'life3 dyn TokenCounter, ) -> Pin<Box<dyn Future<Output = Result<TrimResult, MemoryError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Trims a message sequence against a budget and counter, returning the trim result (semantics in TrimResult).

fn trim_with_counts<'life0, 'life1, 'life2, 'life3, 'life4, 'async_trait>( &'life0 self, messages: &'life1 [Message], counts: &'life2 [usize], budget: &'life3 Budget, counter: &'life4 dyn TokenCounter, ) -> Pin<Box<dyn Future<Output = Result<TrimResult, MemoryError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait, 'life4: 'async_trait,

Trims with per-message token counts; the default implementation delegates to trim, so custom strategies need not know about it. WindowDrop overrides it to amortized O(1) — counts are cached by Memory, avoiding per-message recounts inside the strategy (and saving IO with remote counters).