context-compressor 0.1.0

Dynamic token budget context reducer with regex sanitizers.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

# context-compressor

[![Crates.io](https://img.shields.io/crates/v/context-compressor.svg)](https://crates.io/crates/context-compressor)
[![docs.rs](https://docs.rs/context-compressor/badge.svg)](https://docs.rs/context-compressor)
[![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-blue.svg)](LICENSE)

Dynamic token-budget context reducer for LLM conversations, with built-in regex-based
sanitizers for secrets (API keys, bearer tokens, env vars, telegram tokens, ...). Prunes old
tool results, deduplicates identical tool outputs, strips images outside the active window, and
applies a summary handoff when the budget is exceeded.

Designed to be dropped in front of any chat-completion call to keep prompts under a hard token
ceiling without losing the active task.

## Installation

```sh
cargo add context-compressor
```

## Usage

```rust
use context_compressor::{mask_token, redact_sensitive_text};

// Redact secrets before logging:
let auth = "Authorization: Bearer mytoken12345678901234567";
assert_eq!(
    redact_sensitive_text(auth, true),
    "Authorization: Bearer mytoke...4567"
);

// Mask a single token for display:
assert_eq!(mask_token("longtoken123456789"), "longto...6789");
```

For full compression of message arrays, construct a `ContextCompressor` and call `.compress(...)`.

## License

Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for details.