Module lex

Source
Expand description

Lexer

Translates a script into a reversed sequence of tokens

Structs§

TokenIter
Iterator that goes through a vector of tokens backward (our parser wants to read backward and this is more efficient anyway since we can use Vec::pop()).

Enums§

Token
Atom of a tokenized version of a script

Functions§

lex
Tokenize a script