Skip to main content

tokenize

Function tokenize 

Source
pub fn tokenize(input: &str) -> Result<Vec<Token>, String>
Expand description

Tokenize an input string with default (built-in) operators and keywords.

For custom operators, keywords, or middleware, use Tokenizer::new().