Skip to main content

LexerConfig

oxc::parser::config

Trait LexerConfig 

Source 
pub trait LexerConfig: Default {
    type ByteHandlers: Index<usize, Output = unsafe fn(&mut Lexer<'_, Self>) -> Kind>;

    const TOKENS_METHOD_IS_STATIC: bool;

    // Required methods
    fn tokens(&self) -> bool;
    fn byte_handlers(&self) -> &Self::ByteHandlers;
}
Expand description

Lexer config.

See ParserConfig for more details.

We have to define a different byte handler table for each config, as byte handler functions are generic over the LexerConfig.

Byte handler tables are defined within lexer (lexer/byte_handlers.rs). We need them to be defined as static items, for performance.

  1. static ensures that only 1 copy of each byte handler table appears in the binary (even if Lexer::handle_byte is inlined in multiple places).
  2. Indexing into byte handlers table is an extremely hot path, and even 1 extra indirection (pointer chasing) has a sizeable impact on performance.

An associated type and byte_handlers method is the only way to make this work with Rust’s type system and borrow checker.

Required Associated Constants§

Source

const TOKENS_METHOD_IS_STATIC: bool

true if tokens method returns a static value.

This const is useful in situations where code can be more efficient when the return value of tokens method is known at compile time.

Implementors of this trait MUST only set this const to true if tokens method returns a static value.

Required Associated Types§

Source

type ByteHandlers: Index<usize, Output = unsafe fn(&mut Lexer<'_, Self>) -> Kind>

Byte handlers table type.

Required Methods§

Source

fn tokens(&self) -> bool

Returns true if tokens are enabled.

Source

fn byte_handlers(&self) -> &Self::ByteHandlers

Get byte handlers table to use in lexer with this config.

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors§