Module token_counter 

Token counter and cost tracking

Provides token counting (tiktoken), usage statistics, cost estimation, and a TokenTrackingLLM wrapper.

Re-exports

pub use counter::CharRatioCounter;

pub use counter::TokenCounter;

pub use counter::TrackerTokenUsage;

pub use tiktoken::TiktokenCounter;

pub use tracker::ModelPricing;

pub use tracker::TokenTrackingLLM;

Modules

counter

Token counter trait and usage statistics

tiktoken

Tiktoken counter (OpenAI tokenizer)

tracker

Token-tracking LLM wrapper and cost estimation

Enums

TokenCounterError

Error type for token counting.

Functions

count_tokens

Count tokens in text using the global tiktoken encoder.