Struct SummarizationMiddleware 

pub struct SummarizationMiddleware { /* private fields */ }

Automatically summarizes conversation history when it exceeds a token limit.

Uses a configurable token counter to estimate message sizes. When the total exceeds max_tokens, older messages (excluding the system prompt) are summarized into a single summary message using the provided ChatModel.

Implementations

impl SummarizationMiddleware

pub fn new( model: Arc<dyn ChatModel>, max_tokens: usize, token_counter: impl Fn(&Message) -> usize + Send + Sync + 'static, ) -> Self

Create a new summarization middleware.

• model — The model to use for generating summaries.
• max_tokens — When total tokens exceed this, summarize older messages.
• token_counter — Function that estimates the token count for a message.

Trait Implementations

impl AgentMiddleware for SummarizationMiddleware

fn before_model<'life0, 'life1, 'async_trait>( &'life0 self, request: &'life1 mut ModelRequest, ) -> Pin<Box<dyn Future<Output = Result<(), SynapticError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Called before each model invocation. Can modify the request.

fn before_agent<'life0, 'life1, 'async_trait>( &'life0 self, _messages: &'life1 mut Vec<Message>, ) -> Pin<Box<dyn Future<Output = Result<(), SynapticError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Called once when the agent starts executing.

fn after_agent<'life0, 'life1, 'async_trait>( &'life0 self, _messages: &'life1 mut Vec<Message>, ) -> Pin<Box<dyn Future<Output = Result<(), SynapticError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Called once when the agent finishes executing.

fn after_model<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, _request: &'life1 ModelRequest, _response: &'life2 mut ModelResponse, ) -> Pin<Box<dyn Future<Output = Result<(), SynapticError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Called after each model invocation. Can modify the response.

fn wrap_model_call<'life0, 'life1, 'async_trait>( &'life0 self, request: ModelRequest, next: &'life1 dyn ModelCaller, ) -> Pin<Box<dyn Future<Output = Result<ModelResponse, SynapticError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Wraps the model call. Override to intercept or replace the model invocation.

fn wrap_tool_call<'life0, 'life1, 'async_trait>( &'life0 self, request: ToolCallRequest, next: &'life1 dyn ToolCaller, ) -> Pin<Box<dyn Future<Output = Result<Value, SynapticError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Wraps a tool call. Override to intercept or replace tool execution.