[−][src]Crate code_tokenizer
Code Tokenizer
A string/code tokenizer. Transforms a string with a vector of operators into a vector of keywords.
Functions
get_tokens | Creates a vector of tokens from a String and a vector of operators. |
A string/code tokenizer. Transforms a string with a vector of operators into a vector of keywords.
get_tokens | Creates a vector of tokens from a String and a vector of operators. |