context-compressor

Dynamic token-budget context reducer for LLM conversations, with built-in regex-based sanitizers for secrets (API keys, bearer tokens, env vars, telegram tokens, ...). Prunes old tool results, deduplicates identical tool outputs, strips images outside the active window, and applies a summary handoff when the budget is exceeded.

Designed to be dropped in front of any chat-completion call to keep prompts under a hard token ceiling without losing the active task.

Installation

cargo add context-compressor

Usage

use context_compressor::{mask_token, redact_sensitive_text};

// Redact secrets before logging:
let auth = "Authorization: Bearer mytoken12345678901234567";
assert_eq!(
    redact_sensitive_text(auth, true),
    "Authorization: Bearer mytoke...4567"
);

// Mask a single token for display:
assert_eq!(mask_token("longtoken123456789"), "longto...6789");

For full compression of message arrays, construct a ContextCompressor and call .compress(...).

License

Licensed under the Apache License, Version 2.0. See LICENSE for details.