Skip to main content

compact_with_llm

Function compact_with_llm 

Source
pub async fn compact_with_llm(
    messages: &mut Vec<Message>,
    llm: &dyn Provider,
    model: &str,
    cancel: CancellationToken,
) -> Option<usize>
Expand description

Perform full LLM-based compaction of the conversation history.

Splits the message history into two parts: messages to summarize (older) and messages to keep (recent). Calls the LLM to generate a summary, then replaces the old messages with:

  1. A compact boundary marker
  2. A summary message (as a user message with is_compact_summary=true)
  3. The kept recent messages

Returns the number of messages removed, or None if compaction failed.