Module token 

Structs

TokenEstimatorWrapper

Wrapper for TokenEstimator that implements Debug

Functions

default_token_estimator

Default lightweight token estimator. Uses heuristic based on character types:

Type Aliases

TokenEstimator

Token estimator function type. Callers can inject precise implementations (e.g., using tiktoken) if available.