Module boa_parser::lexer

Boa’s lexical analyzer(Lexer) for ECMAScript source code.

The Lexer splits its input source code into a sequence of input elements called tokens, represented by the Token structure. It also removes whitespace and comments and attaches them to the next token.

This is tightly coupled with the parser due to the javascript goal-symbol requirements as documented by the spec.

More information:

• ECMAScript reference

Re-exports

• pub use self::error::Error;
• pub use self::token::Token;
• pub use self::token::TokenKind;

Modules

• error
  This module contains the errors used by the lexer.
• regex
  Boa’s lexing for ECMAScript regex literals.
• token
  Boa’s implementation of all ECMAScript Tokens.

Structs

• Lexer
  Lexer or tokenizer for the Boa JavaScript Engine.