Module esparse::lex

Lexical analysis (tokenizing/lexing).

The Lexer splits its input source code into a sequence of input elements called tokens, represented by the Tok structure. It also removes whitespace and comments and attaches them to the next token.

Examples

use esparse::lex::Lexer;

let mut lexer = Lexer::new("<input>", "1 + 2");

for tok in lexer {
    println!("{} ", tok.tt);
}

Structs

Lexer	A lexical analyzer for JavaScript source code.
Stream	Generic stream structure for source code.
Tok	A token (an atomic parsing unit).

Enums

ParseStrLitError	An error type for parsing string literals.
Tt	The content of a token.

Functions

str_lit_value

Parses a string literal and extracts its value, stripping the leading and trailing quotation marks and interpreting escape sequences.