Module lexer

Boa’s lexical analyzer(Lexer) for ECMAScript source code.

The Lexer splits its input source code into a sequence of input elements called tokens, represented by the Token structure. It also removes whitespace and comments and attaches them to the next token.

This is tightly coupled with the parser due to the javascript goal-symbol requirements as documented by the spec.

More information:

• ECMAScript reference

Re-exports

pub use self::error::Error;

pub use self::token::Token;

pub use self::token::TokenKind;

Modules

error

This module contains the errors used by the lexer.

regex

Boa’s lexing for ECMAScript regex literals.

token

Boa’s implementation of all ECMAScript Tokens.

Structs

Lexer

Lexer or tokenizer for the Boa JavaScript Engine.