Struct UsageTracker 

pub struct UsageTracker { /* private fields */ }

Tracks cumulative token usage across multiple LLM calls.

UsageTracker accumulates Usage records from each request and provides context-awareness features for detecting when the conversation is approaching the model’s context limit.

Example

use llm_stack::usage::{Usage, UsageTracker};

let mut tracker = UsageTracker::with_context_limit(128_000);

// Record usage from each LLM call
tracker.record(Usage {
    input_tokens: 1000,
    output_tokens: 500,
    ..Default::default()
});

assert_eq!(tracker.total().input_tokens, 1000);
assert!(!tracker.is_near_limit(0.8)); // Not near 80% yet

Use Cases

• Billing/cost tracking: Aggregate costs across a session
• Budget alerts: Warn when approaching token limits
• Compaction triggers: Signal when context window is nearly full
• Token debugging: Analyze per-call consumption patterns

Implementations

impl UsageTracker

pub fn new() -> Self

Creates a new tracker with no context limit.

pub fn with_context_limit(limit: u64) -> Self

Creates a tracker with a known context window limit.

The limit is used for context_utilization and is_near_limit calculations.

pub fn record(&mut self, usage: Usage)

Records a usage sample from an LLM call.

The usage is added to the running total and stored for per-call analysis.

pub fn total(&self) -> &Usage

Returns the accumulated usage across all recorded calls.

pub fn calls(&self) -> &[Usage]

Returns the usage from each individual call, in order.

pub fn call_count(&self) -> usize

Returns the number of calls recorded.

pub fn context_limit(&self) -> Option<u64>

Returns the context limit, if set.

pub fn set_context_limit(&mut self, limit: u64)

Sets or updates the context limit.

Useful when the model is determined after tracker creation.

pub fn context_utilization(&self) -> Option<f64>

Returns the context utilization as a ratio (0.0 to 1.0+).

Utilization is calculated as total_input_tokens / context_limit. Returns None if no context limit is set.

Note

The value can exceed 1.0 if the total exceeds the limit (which shouldn’t happen in practice but is not enforced).

pub fn is_near_limit(&self, threshold: f64) -> bool

Checks if the context utilization is at or above the given threshold.

Returns false if no context limit is set.

Example

use llm_stack::usage::{Usage, UsageTracker};

let mut tracker = UsageTracker::with_context_limit(100_000);
tracker.record(Usage {
    input_tokens: 85_000,
    output_tokens: 1000,
    ..Default::default()
});

assert!(tracker.is_near_limit(0.8));   // 85% >= 80%
assert!(!tracker.is_near_limit(0.9));  // 85% < 90%

pub fn cost(&self, pricing: &ModelPricing) -> Option<Cost>

Computes the cost of all recorded usage given a pricing table.

Uses the pricing rates (per-million tokens) to calculate cost. Returns None if the cost would overflow.

pub fn reset(&mut self)

Resets the tracker, clearing all recorded usage.

Trait Implementations

impl Clone for UsageTracker

fn clone(&self) -> UsageTracker

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for UsageTracker

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for UsageTracker

fn default() -> Self

Returns the “default value” for a type.