Struct WindowMemory 

pub struct WindowMemory { /* private fields */ }

Window memory: stores all messages; context() trims to the budget on retrieval.

• Within budget: returns a full clone as-is;
• Over budget: calls the TrimStrategy — the default WindowDrop drops the earliest messages by round (projection, lossless); an injected heavy strategy can declare replace: true to materialize (write back to storage, zero recomputation afterwards).

Counting is lazy: after a materialized write-back nothing is counted until the first async call (record / context) recomputes in one pass — TokenCounter may involve remote IO, and the synchronous path never touches it. Materialization happens inside context on &self, using an internal std::sync::Mutex for interior mutability; the critical section contains no await, so single-threaded runtimes are not blocked.

Comparison

Compared to InMemoryMemory: the latter stores all messages verbatim and never trims, suiting small conversations without budget control; use this type when context size must be controlled (the model’s context window is finite).

Panics

Panics only when the internal lock is poisoned (a panic occurred while holding it); never in normal use.

Examples

Basic usage:

use molo::memory::{Memory, WindowMemory};

let mut memory = WindowMemory::new(100);
memory.record(molo::message::Message::user("hello")).await?;
assert_eq!(memory.context().await?.len(), 1);

Trims by round when over budget, keeping at least the most recent round:

use molo::memory::{Memory, WindowMemory};

let mut memory = WindowMemory::new(3);
memory.record(molo::message::Message::user("u1")).await?;
memory.record(molo::message::Message::assistant("a1")).await?;
memory.record(molo::message::Message::user("u2")).await?;
memory.record(molo::message::Message::assistant("a2")).await?;

// 4 tokens total, over the 3-token budget: the earliest round is dropped.
let context = memory.context().await?;
assert_eq!(context.len(), 2);
assert_eq!(context[0], molo::message::Message::user("u2"));

Implementations

impl WindowMemory

pub fn new(max_tokens: usize) -> Self

Creates an empty session with a context budget of max_tokens tokens.

Counting and trimming default to CharTokenCounter and WindowDrop; replace them with with_token_counter and with_strategy respectively.

pub fn with_max_rounds(self, max_rounds: usize) -> Self

Adds a “keep at most the recent N rounds” limit (unlimited by default; the smaller window wins when set together with the token budget).

pub fn with_token_counter(self, counter: Box<dyn TokenCounter>) -> Self

Replaces the default heuristic counting with exact counting (matching your own model).

Swapping changes the counting convention and invalidates previously counted results: the first async call (record / context) recomputes with the new counter.

pub fn with_strategy(self, strategy: Arc<dyn TrimStrategy>) -> Self

Injects a custom trim strategy (summarization, LLM compaction, etc.); defaults to WindowDrop.

Example

A summarization strategy: replaces everything but the most recent round with a summary message and materializes it (replace: true):

use std::sync::Arc;
use molo::memory::{
    Budget, Memory, MemoryError, TokenCounter, TrimResult, TrimStrategy, WindowMemory,
};
use molo::Message;

#[derive(Default)]
struct Summarize;

#[molo::async_trait]
impl TrimStrategy for Summarize {
    async fn trim(
        &self,
        messages: &[Message],
        _budget: &Budget,
        _counter: &dyn TokenCounter,
    ) -> Result<TrimResult, MemoryError> {
        // Keep the most recent round; replace earlier messages with a
        // summary message.
        // The summary uses the System role so no consecutive User
        // messages appear after replacement (role-alternation
        // constraint).
        let pos = messages
            .iter()
            .rposition(|m| matches!(m, Message::User(_)))
            .unwrap_or(0);
        let mut result = Vec::with_capacity(messages.len() - pos + 1);
        result.push(Message::system("prior summary"));
        result.extend_from_slice(&messages[pos..]);
        Ok(TrimResult { messages: result, replace: true })
    }
}

#[tokio::main]
async fn main() {
    let mut memory = WindowMemory::new(7).with_strategy(Arc::new(Summarize));
    for i in 1..=4 {
        memory.record(Message::user(format!("u{i}"))).await.unwrap();
        memory.record(Message::assistant(format!("a{i}"))).await.unwrap();
    }

    // 8 > 7: compacted to [summary, most recent round].
    let context = memory.context().await.unwrap();
    assert_eq!(context.len(), 3);
    assert!(matches!(context[0], Message::System(_)));
}

pub fn set_max_tokens(&mut self, max_tokens: usize)

Adjusts the token budget at runtime; takes effect on the next context retrieval.

With a projection strategy, raising the budget restores the trimmed history; with a materializing strategy, old messages are gone and a higher budget cannot restore them.

pub fn set_max_rounds(&mut self, max_rounds: Option<usize>)

Adjusts the round limit at runtime (None = unlimited); takes effect on the next context retrieval.

Trait Implementations

impl Debug for WindowMemory

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Memory for WindowMemory

fn record<'life0, 'async_trait>( &'life0 mut self, message: Message, ) -> Pin<Box<dyn Future<Output = Result<(), MemoryError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Records a message, appending it to the end of the session context.

fn record_protected<'life0, 'async_trait>( &'life0 mut self, message: Message, ) -> Pin<Box<dyn Future<Output = Result<(), MemoryError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Records a protected message: implementations must not remove it when trimming.

fn context<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<Vec<Message>, MemoryError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Returns the full message sequence of the current context (in order of occurrence), which the agent hands to the Provider as-is.