token-budget-pool 0.1.0

Shared token + dollar budget across concurrent LLM tasks. Thread-safe, returns BudgetExceeded when a record would push past a cap. Zero deps.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

# token-budget-pool

[![crates.io](https://img.shields.io/crates/v/token-budget-pool.svg)](https://crates.io/crates/token-budget-pool)
[![docs.rs](https://img.shields.io/docsrs/token-budget-pool)](https://docs.rs/token-budget-pool)

Shared token + dollar budget across N concurrent LLM tasks. Thread-safe,
rejects records that would push past a cap (totals unchanged on
rejection), zero deps.

## Usage

```rust
use token_budget_pool::{BudgetPool, Caps};
use std::sync::Arc;

let pool = Arc::new(BudgetPool::with_caps(Caps {
    max_input_tokens: Some(10_000),
    max_output_tokens: Some(5_000),
    max_total_tokens: None,
    max_cost_usd: Some(1.0),
}));

// Hand a clone to every task that spends from the budget.
let p = pool.clone();
std::thread::spawn(move || {
    if let Err(e) = p.record(1_000, 500, 0.05) {
        eprintln!("budget exceeded: {e}");
    }
});
```

## License

MIT or Apache-2.0.