Skip to main content

Module lexer

Module lexer 

Source

Structs§

ParseOptions
Parse-affecting preview options: the ADR-0371 preview gates that widen what the parser accepts (the lexer’s token stream and the parser’s own grammar), resolved by the caller (the front derives them from the store’s RELATE_PREVIEW set). The default is every gate closed – the dialect floor. One struct so a speculative or nested parse inherits its caller’s dialect by plain field copy; the flags are Copy, never re-derived per site.

Enums§

CommentSkip
What skip_comment found at a position in the input.

Functions§

skip_comment
The Cypher comment starting at input[pos..] — the one spelling of the comment rule, shared by the lexer and by callers that re-read a literal token’s text through the same grammar.
tokenize
Tokenizes a Cypher query string into a list of tokens.
tokenize_with_options
Tokenizes with explicit ParseOptions; tokenize is the same call with every gate closed.