Module boa::syntax::lexer

Expand description

A lexical analyzer for JavaScript source code.

This module contains the Boa lexer or tokenizer implementation.

The Lexer splits its input source code into a sequence of input elements called tokens, represented by the Token structure. It also removes whitespace and comments and attaches them to the next token.

This is tightly coupled with the parser due to the javascript goal-symbol requirements as documented by the spec.

More information:

ECMAScript reference

Re-exports

pub use crate::profiler::BoaProfiler;

pub use crate::syntax::ast::Position;

pub use error::Error;

pub use token::Token;

pub use token::TokenKind;

Modules

error

This module contains the errors used by the lexer.

token

This module implements all of the Tokens used in the JavaScript programing language.

Structs

Lexer

Lexer or tokenizer for the Boa JavaScript Engine.