agent-context

Backend-agnostic LLM conversation context manager built on the kameo actor model.

Three-Zone + Scratch Memory

Messages are split into three storage zones plus an optional transient scratch layer:

Concatenation order: immutable → compressed → incremental → scratch (if present).

Compression Notification

Use RequestSubscribeCompressed to register a subscriber that can modify the summary and kept messages after compression. Unsubscribe anytime with RequestUnsubscribeCompressed.

Usage

use agent_context::{
    AgentContext, RequestAppend, RequestSend, RequestSendStream, RequestMessages,
    ContextBackend, CommonOpts,
};
use futures::StreamExt;

// 1. Implement ContextBackend for your LLM backend
// 2. Spawn the actor
let ctx = AgentContext::new(backend, vec![backend.system_message("You are a helpful assistant.")]);
let actor_ref = AgentContext::spawn(ctx);

// 3. Non-streaming conversation
actor_ref.ask(RequestAppend { message: user_msg }).await?;
let response = actor_ref.ask(RequestSend { opts: my_opts }).await?;

// 4. Streaming conversation
let mut stream = actor_ref.ask(RequestSendStream { opts: my_opts }).await??;
while let Some(chunk) = stream.next().await { /* e.g. render chunk in UI */ }
let chunks = stream.take_chunks();
let msg = backend.merge_chunks(&chunks).expect("chunk merge failed");
let msgs = backend.to_request_messages(vec![msg])?;
for m in msgs {
    actor_ref.ask(RequestAppend { message: m }).await?;
}

// 5. Read all messages
let all: Vec<_> = actor_ref.ask(RequestMessages).await?;

ContextBackend

The core trait — implement it to support any LLM provider (DeepSeek, Zhipu, OpenAI, etc.):

• Message factories: user_message, system_message, tool_message
• Format conversion: to_system_message, to_request_messages
• Response parsing: extract_messages, merge_chunks
• Model interaction: send, send_stream, estimate_tokens

Each backend's Opts type must embed CommonOpts (model, context_window, max_tokens, auto_compress, scratch) via AsRef<CommonOpts>.

Compression

Automatic compression is triggered before each RequestSend/RequestSendStream when CommonOpts::auto_compress is true and the estimated tokens exceed context_window. Use RequestCompress with CompressStrategy::Summarize for manual compression. Register a compressed subscriber via RequestSubscribeCompressed to customize the result.

License

MIT OR Apache-2.0