count_file_tokens

context_builder::token_count

Function count_file_tokens 

Source 
pub fn count_file_tokens(
    base_path: &Path,
    entry: &DirEntry,
    line_numbers: bool,
) -> usize
Expand description

Counts the tokens that would be generated for a file