Module lexer 

The lexer.

Enums

EscapeError

Errors and warnings that can occur during string unescaping.

LexerError

Kind of LexerError.

Functions

is_id_continue

True if c is valid as a non-first character of an identifier.

is_id_start

True if c is valid as a first character of an identifier.

is_ident

The passed string is lexically an identifier.

is_whitespace

True if c is considered a whitespace according to Rust language definition.

tokenize

Creates an iterator that produces tokens from the input string.