Struct ExecutionTokenBudget 

pub struct ExecutionTokenBudget { /* private fields */ }

One execution’s token budget, reserving against its own ceiling and the server-wide one together.

Both reservations are taken together and both unwound if either fails, so a refused dispatch leaves neither counter charged.

Implementations

impl ExecutionTokenBudget

pub fn new(limits: LlmLimits, aggregate: SharedTokenBudget) -> Self

pub fn limits(&self) -> LlmLimits

pub fn used(&self) -> u64

Tokens this execution currently holds against its ceiling.

pub fn held(&self) -> u64

The part of Self::used held for unreported usage.

pub fn reserve(&self, model: &str, tokens: u64) -> Result<(), LlmCallError>

Reserve tokens against this execution and the aggregate, before dispatch.

The caller passes the full reservation — estimated_input_tokens + (output_cap × prompt_count), via Self::reservation_for — because reserving only input enforces the ceiling retroactively: output tokens are typically the expensive half, and a small prompt can legitimately produce a very large completion with nothing standing in the way.

pub fn reservation_for( &self, estimated_input_tokens: u64, prompt_count: u64, output_cap: Option<u64>, ) -> u64

The reservation a dispatch of prompt_count prompts needs: estimated_input_tokens + (output_cap × prompt_count).

output_cap is the resolved per-model outputReserve when the model declares one, and LlmLimits::default_output_cap otherwise. The same cap is passed to the provider as the request’s maxOutputTokens, which makes the reservation an actual upper bound rather than an estimate.

pub fn reconcile(&self, reserved: u64, reported: u64, indeterminate: u64)

Reconcile a completed dispatch down from its reserved estimate to what the provider actually reported.

reported is the summed usage of the elements the provider gave counts for; indeterminate is the reserve belonging to elements it reported nothing for. The indeterminate part is not released — None means indeterminate, not free, since a throttled call may still have been billed — and is instead carried as held reserve, bounded by LlmLimits::max_held_tokens.

pub fn release(&self, tokens: u64)

Release a reservation for prompts that never dispatched — the remainder of a batch abandoned partway.