Struct TokenTrackingLLM 

pub struct TokenTrackingLLM<L: BaseChatModel> { /* private fields */ }

LLM wrapper with token statistics

Wraps any BaseChatModel, accumulating prompt / completion token usage automatically, preferring the real usage returned by the LLM, falling back to tiktoken estimates.

Implementations

impl<L: BaseChatModel> TokenTrackingLLM<L>

pub fn new(llm: L, counter: Arc<dyn TokenCounter>) -> Self

Wraps an LLM with a custom counter.

pub fn for_openai(llm: L) -> Result<Self, TokenCounterError>

Wraps with a Tiktoken (cl100k_base) counter

pub async fn chat( &self, messages: Vec<Message>, config: Option<RunnableConfig>, ) -> Result<LLMResult, L::Error>

Calls the LLM and counts tokens

pub async fn get_usage(&self) -> TrackerTokenUsage

Returns the cumulative usage

pub async fn reset(&self)

Resets the statistics

pub async fn estimate_cost(&self, pricing: &ModelPricing) -> f64

Estimates the cost (USD)