Struct AtomicTokenBudget 

pub struct AtomicTokenBudget { /* private fields */ }

Atomic counterpart of AtomicBudget that tracks LLM token spend.

try_reserve_tokens deducts an estimate up front so concurrent prompts can’t all over-commit. record_usage reconciles one TokenReservation against the provider’s reported totals — if the actual usage was smaller than that estimate, the difference is returned to the pool; if larger, the overage is debited from future reservations.

Reservations also refund their estimate on drop, so an error path that abandons the handle without calling record_usage does not permanently leak tokens.

Implementations

impl AtomicTokenBudget

pub fn new(capacity: u64) -> Self

Create a token budget with the given capacity, initially full.

pub fn capacity(&self) -> u64

Total capacity at construction time.

pub fn available(&self) -> u64

Currently available tokens.

Trait Implementations

impl Debug for AtomicTokenBudget

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl TokenBudget for AtomicTokenBudget

fn try_reserve_tokens<'life0, 'async_trait>( &'life0 self, est: u64, ) -> Pin<Box<dyn Future<Output = Result<Option<TokenReservation>, BudgetError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Reserve est prompt+completion tokens optimistically.

fn record_usage<'life0, 'async_trait>( &'life0 self, reservation: TokenReservation, prompt: u64, completion: u64, ) -> Pin<Box<dyn Future<Output = ()> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Record the actual prompt + completion token usage from a finished call. The implementation reconciles the supplied reservation against the observed usage.

fn tokens_consumed<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = u64> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Total tokens consumed since construction (prompt + completion).