Module esparse::lex

Lexical analysis (tokenizing/lexing).

The Lexer splits its input source code into a sequence of input elements called tokens, represented by the Tok structure. It also removes whitespace and comments and attaches them to the next token.

Examples

use esparse::lex::Lexer;

let mut lexer = Lexer::new_unnamed("1 + 2");

for tok in lexer {
    println!("{} ", tok.tt);
}

Structs

Error	The error type for lexical analysis.
Lexer	A lexical analyzer for JavaScript source code.
LocStream	Generic source code stream with row/column information.
PosStream	Generic source code stream with byte offset information.
Tok	A token (an atomic parsing unit).

Enums

ErrorKind	The specific kind of error that occurred.
ParseStrLitError	An error type for parsing string literals.
Tt	The content of a token.

Traits

Stream

Trait for source code streams.

Functions

str_lit_value

Parses a string literal and extracts its value, stripping the leading and trailing quotation marks and interpreting escape sequences.