Module esparse::lex
[−]
[src]
Lexical analysis (tokenizing/lexing).
The Lexer
splits its input source code into a sequence of input elements called tokens, represented by the Tok
structure. It also removes whitespace and comments and attaches them to the next token.
Examples
use esparse::lex::Lexer; let mut lexer = Lexer::new_unnamed("1 + 2"); for tok in lexer { println!("{} ", tok.tt); }
Structs
Error |
The error type for lexical analysis. |
Lexer |
A lexical analyzer for JavaScript source code. |
Stream |
Generic stream structure for source code. |
Tok |
A token (an atomic parsing unit). |
Enums
ErrorKind |
The specific kind of error that occurred. |
ParseStrLitError |
An error type for parsing string literals. |
Tt |
The content of a token. |
Functions
str_lit_value |
Parses a string literal and extracts its value, stripping the leading and trailing quotation marks and interpreting escape sequences. |