Module token_count

Module token_count 

Source

Functionsยง

count_file_tokens
Counts the tokens that would be generated for a file
count_tree_tokens
Counts the tokens that would be generated for the entire file tree section
estimate_tokens
Estimates the number of tokens in a text string using a real tokenizer