ailoop-context 1.0.0-rc.2

Conversation history management and compaction for ailoop
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65

//! Conversation history management for ailoop.
//!
//! Owns the message vector that drives every [`ChatRequest`] and
//! enforces the token-budget contract: [`ContextManager`] tracks the
//! running history, applies a [`CompactionStrategy`] when the wired
//! [`Tokenizer`] reports the budget is exceeded, and persists the pin
//! mask that keeps system-style anchors and live `ToolCall ↔ ToolResult`
//! pairs intact across compactions.
//!
//! Most application code does not construct a `ContextManager` directly
//! — the [`ailoop`](https://docs.rs/ailoop) façade wires one inside
//! [`Conversation`](https://docs.rs/ailoop) with a sensible default
//! budget. Reach for this crate when you need a custom strategy
//! ([`SummarizeStrategy`] vs [`TruncateStrategy`]), a non-default
//! tokenizer (e.g. `ailoop_anthropic::OnlineCalibratedTokenizer`),
//! direct access to `compact_if_needed` from
//! [`advanced::run_chat`](https://docs.rs/ailoop), or your own
//! [`HistoryStore`] backend for persistence.
//!
//! ## Mini-index
//!
//! - [`ContextManager`] + [`ContextManagerBuilder`] — the history
//!   container and its configuration entry point.
//! - [`CompactionStrategy`] — trait implemented by reduction
//!   algorithms; ships with [`TruncateStrategy`] and
//!   [`SummarizeStrategy`].
//! - [`CompactionOutput`], [`CompactionReport`], [`CompactionError`],
//!   [`FromMessagesError`] — the surrounding vocabulary.
//! - [`HistoryStore`] — async persistence trait. [`InMemoryHistoryStore`]
//!   and [`JsonFileHistoryStore`] cover tests and single-file durable
//!   storage; implement the trait for Redis, Postgres, etc.
//! - [`ConversationSnapshot`] — versioned wire format for
//!   [`HistoryStore`] payloads. Validated at deserialize time so a
//!   malformed file fails loudly rather than panicking later.
//! - [`Tokenizer`] / [`CharTokenizer`] — re-exported from
//!   [`ailoop_core`] so [`ContextManagerBuilder::tokenizer`] callers
//!   can stay on this crate's import surface.
//!
//! [`ChatRequest`]: ailoop_core::ChatRequest

#![deny(missing_docs)]

pub mod compaction;
pub mod context_manager;
pub mod errors;
pub mod history_store;
pub mod snapshot;

pub use compaction::{
    CompactionOutput, CompactionStrategy, DEFAULT_SUMMARIZER_PROMPT, SummarizeStrategy,
    TruncateStrategy,
};
pub use context_manager::{CompactionReport, ContextManager, ContextManagerBuilder};
pub use errors::{CompactionError, FromMessagesError};
pub use history_store::{
    HistoryStore, InMemoryHistoryStore, JsonFileHistoryStore, JsonFileHistoryStoreError,
};
pub use snapshot::ConversationSnapshot;

// `Tokenizer` lives in `ailoop-core` (its canonical home alongside
// `Message`). Re-export at the root so callers wiring
// `ContextManagerBuilder::tokenizer` can stay on this crate's import
// surface without dragging `ailoop-core` into their dep tree.
// `CharTokenizer` is the dev/test fallback.
pub use ailoop_core::{CharTokenizer, Tokenizer};