pub struct SummarizationCompactionStrategy { /* private fields */ }Expand description
Summarizes the oldest groups with a chat client once the conversation
grows past target_count + threshold non-system messages, keeping about
target_count of the newest (upstream SummarizationStrategy).
Security: the summary replaces the original messages and is trusted as an assistant message from then on, so a compromised summarizer could plant instructions that persist (indirect prompt injection). Only point it at a model you trust as much as the primary one.
A failed or empty summarization leaves the conversation unchanged.
Implementations§
Source§impl SummarizationCompactionStrategy
impl SummarizationCompactionStrategy
Sourcepub fn new(client: Arc<dyn ChatClient>) -> SummarizationCompactionStrategy
pub fn new(client: Arc<dyn ChatClient>) -> SummarizationCompactionStrategy
Summarize with client, keeping 4 messages and allowing 2 more
before summarizing, as upstream.
Sourcepub fn with_target_count(
self,
target_count: usize,
) -> Result<SummarizationCompactionStrategy, Error>
pub fn with_target_count( self, target_count: usize, ) -> Result<SummarizationCompactionStrategy, Error>
Keep about target_count non-system messages (at least 1).
Sourcepub fn with_threshold(self, threshold: usize) -> SummarizationCompactionStrategy
pub fn with_threshold(self, threshold: usize) -> SummarizationCompactionStrategy
Allow threshold messages beyond the target before summarizing.
Sourcepub fn with_prompt(
self,
prompt: impl Into<String>,
) -> SummarizationCompactionStrategy
pub fn with_prompt( self, prompt: impl Into<String>, ) -> SummarizationCompactionStrategy
The summarization instruction.
Sourcepub fn with_max_summary_input_tokens(
self,
tokens: Option<usize>,
) -> Result<SummarizationCompactionStrategy, Error>
pub fn with_max_summary_input_tokens( self, tokens: Option<usize>, ) -> Result<SummarizationCompactionStrategy, Error>
Cap the summarization request at tokens (prompt plus transcript;
whole groups are taken until the next would exceed it), or lift the
cap with None.
Trait Implementations§
Source§impl AsyncCompactionStrategy for SummarizationCompactionStrategy
impl AsyncCompactionStrategy for SummarizationCompactionStrategy
Source§fn compact_async<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
messages: &'life1 [Message],
tokenizer: &'life2 dyn Tokenizer,
) -> Pin<Box<dyn Future<Output = Vec<Message>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
SummarizationCompactionStrategy: 'async_trait,
fn compact_async<'life0, 'life1, 'life2, 'async_trait>(
&'life0 self,
messages: &'life1 [Message],
tokenizer: &'life2 dyn Tokenizer,
) -> Pin<Box<dyn Future<Output = Vec<Message>> + Send + 'async_trait>>where
'life0: 'async_trait,
'life1: 'async_trait,
'life2: 'async_trait,
SummarizationCompactionStrategy: 'async_trait,
Source§impl Clone for SummarizationCompactionStrategy
impl Clone for SummarizationCompactionStrategy
Source§fn clone(&self) -> SummarizationCompactionStrategy
fn clone(&self) -> SummarizationCompactionStrategy
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read more