pub fn estimate_token_count(text: &str) -> usize
Count the number of tokens in text using tiktoken BPE.
text
Returns 0 for empty strings. Falls back to a character-based heuristic if the BPE tokenizer is unavailable.