Module xswag_syntax_java::lex
[−]
[src]
Definition of the Token type and the Tokenizer
This module contains functionality for the "lexing" stage, which is responsible for converting the raw source code (a string) into a stream of tokens.
Structs
| Error |
Type returned by the Tokenizer in case of a lexing error. Contains a detailed report and may contain poisoned token that could be used to proceed normally. |
| TokenSpan |
A token with it's span in the source text |
| Tokenizer |
The Java Tokenizer |
Enums
| Keyword |
Represents one of the Java keywords |
| Lit |
A Java literal |
| Token |
A Java token |
Type Definitions
| Result |
Result with a possible |