[−][src]Crate polyglot_tokenizer
Enums
Token |
Functions
get_key_tokens | Tokenize the content and return only the identifiers and symbols from the langauge |
tokenize |
Token |
get_key_tokens | Tokenize the content and return only the identifiers and symbols from the langauge |
tokenize |