Struct DeepSummarizationMiddleware 

pub struct DeepSummarizationMiddleware { /* private fields */ }

Middleware that auto-summarizes conversation history when approaching the token limit.

Before each model call, estimates the token count. If it exceeds max_input_tokens * threshold_fraction, older messages are summarized by the model and the full history is offloaded to a file in the backend.

Implementations

impl DeepSummarizationMiddleware

pub fn new( backend: Arc<dyn Backend>, model: Arc<dyn ChatModel>, max_input_tokens: usize, threshold_fraction: f64, ) -> Self

Trait Implementations

impl AgentMiddleware for DeepSummarizationMiddleware

fn before_model<'life0, 'life1, 'async_trait>( &'life0 self, request: &'life1 mut ModelRequest, ) -> Pin<Box<dyn Future<Output = Result<(), SynapticError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Called before each model invocation. Can modify the request.

fn before_agent<'life0, 'life1, 'async_trait>( &'life0 self, _messages: &'life1 mut Vec<Message>, ) -> Pin<Box<dyn Future<Output = Result<(), SynapticError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Called once when the agent starts executing.

fn after_agent<'life0, 'life1, 'async_trait>( &'life0 self, _messages: &'life1 mut Vec<Message>, ) -> Pin<Box<dyn Future<Output = Result<(), SynapticError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Called once when the agent finishes executing.

fn after_model<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, _request: &'life1 ModelRequest, _response: &'life2 mut ModelResponse, ) -> Pin<Box<dyn Future<Output = Result<(), SynapticError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, Self: 'async_trait,

Called after each model invocation. Can modify the response.

fn wrap_model_call<'life0, 'life1, 'async_trait>( &'life0 self, request: ModelRequest, next: &'life1 dyn ModelCaller, ) -> Pin<Box<dyn Future<Output = Result<ModelResponse, SynapticError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Wraps the model call. Override to intercept or replace the model invocation.

fn wrap_tool_call<'life0, 'life1, 'async_trait>( &'life0 self, request: ToolCallRequest, next: &'life1 dyn ToolCaller, ) -> Pin<Box<dyn Future<Output = Result<Value, SynapticError>> + Send + 'async_trait>>

where 'life0: 'async_trait, 'life1: 'async_trait, Self: 'async_trait,

Wraps a tool call. Override to intercept or replace tool execution.