pub struct SlidingWindowStrategy { /* private fields */ }Expand description
Keeps system messages plus the last window_size non-system messages.
Triggers compaction when the estimated token count exceeds max_tokens.
§Example
use neuron_context::SlidingWindowStrategy;
let strategy = SlidingWindowStrategy::new(10, 100_000);Implementations§
Source§impl SlidingWindowStrategy
impl SlidingWindowStrategy
Sourcepub fn new(window_size: usize, max_tokens: usize) -> Self
pub fn new(window_size: usize, max_tokens: usize) -> Self
Creates a new SlidingWindowStrategy.
§Arguments
window_size— maximum number of non-system messages to retainmax_tokens— token threshold above which compaction is triggered
Sourcepub fn with_counter(
window_size: usize,
max_tokens: usize,
counter: TokenCounter,
) -> Self
pub fn with_counter( window_size: usize, max_tokens: usize, counter: TokenCounter, ) -> Self
Creates a new SlidingWindowStrategy with a custom TokenCounter.
Trait Implementations§
Source§impl ContextStrategy for SlidingWindowStrategy
impl ContextStrategy for SlidingWindowStrategy
Source§fn should_compact(&self, messages: &[Message], token_count: usize) -> bool
fn should_compact(&self, messages: &[Message], token_count: usize) -> bool
Whether compaction should be triggered given the current messages and token count.
Source§fn compact(
&self,
messages: Vec<Message>,
) -> impl Future<Output = Result<Vec<Message>, ContextError>> + WasmCompatSend
fn compact( &self, messages: Vec<Message>, ) -> impl Future<Output = Result<Vec<Message>, ContextError>> + WasmCompatSend
Compact the message list to reduce token usage.
Source§fn token_estimate(&self, messages: &[Message]) -> usize
fn token_estimate(&self, messages: &[Message]) -> usize
Estimate the token count for a list of messages.
Auto Trait Implementations§
impl Freeze for SlidingWindowStrategy
impl RefUnwindSafe for SlidingWindowStrategy
impl Send for SlidingWindowStrategy
impl Sync for SlidingWindowStrategy
impl Unpin for SlidingWindowStrategy
impl UnsafeUnpin for SlidingWindowStrategy
impl UnwindSafe for SlidingWindowStrategy
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more