Module lsts::token

source · [−]
Expand description

The Token module defines lexical tokens, spans, and a stream tokenizer

Structs

Span
Token
TokenReader

Enums

Symbol

Functions

is_ident_char
is_value_char
span_of
tokenize_file
tokenize_string