Module tokenizer

Module tokenizer 

Source
Expand description

Tokenizing an input string for further parsing.

Structs§

Token
A valid token for the query language.
TokenSpan
The location of a token.

Enums§

TokenKind
All possible kinds of tokens.