Function tokenize

Source
pub fn tokenize(input: &str) -> Result<Vec<Token>, Error>
Expand description

Tokenizes the given script (turns it into a series of tokens).

ยงErrors

Emits an appropriate error if the script is invalid and tokenization fails.