Enum koto_lexer::Token
source · pub enum Token {
Show 74 variants
Error,
Whitespace,
NewLine,
CommentSingle,
CommentMulti,
Number,
Id,
Wildcard,
StringStart(StringType),
StringEnd,
StringLiteral,
At,
Colon,
Comma,
Dot,
Ellipsis,
Function,
RoundOpen,
RoundClose,
SquareOpen,
SquareClose,
CurlyOpen,
CurlyClose,
Range,
RangeInclusive,
Add,
Subtract,
Multiply,
Divide,
Remainder,
Assign,
AddAssign,
SubtractAssign,
MultiplyAssign,
DivideAssign,
RemainderAssign,
Equal,
NotEqual,
Greater,
GreaterOrEqual,
Less,
LessOrEqual,
Pipe,
As,
And,
Break,
Catch,
Continue,
Debug,
Else,
ElseIf,
Export,
False,
Finally,
For,
From,
If,
Import,
In,
Loop,
Match,
Not,
Null,
Or,
Return,
Self_,
Switch,
Then,
Throw,
True,
Try,
Until,
While,
Yield,
}
Expand description
The tokens that can emerge from the lexer
Variants§
Error
Whitespace
NewLine
CommentSingle
CommentMulti
Number
Id
Wildcard
StringStart(StringType)
StringEnd
StringLiteral
At
Colon
Comma
Dot
Ellipsis
Function
RoundOpen
RoundClose
SquareOpen
SquareClose
CurlyOpen
CurlyClose
Range
RangeInclusive
Add
Subtract
Multiply
Divide
Remainder
Assign
AddAssign
SubtractAssign
MultiplyAssign
DivideAssign
RemainderAssign
Equal
NotEqual
Greater
GreaterOrEqual
Less
LessOrEqual
Pipe
As
And
Break
Catch
Continue
Debug
Else
ElseIf
Export
False
Finally
For
From
If
Import
In
Loop
Match
Not
Null
Or
Return
Self_
Switch
Then
Throw
True
Try
Until
While
Yield
Implementations§
source§impl Token
impl Token
sourcepub fn is_whitespace(&self) -> bool
pub fn is_whitespace(&self) -> bool
Returns true if the token should be counted as whitespace
sourcepub fn is_whitespace_including_newline(&self) -> bool
pub fn is_whitespace_including_newline(&self) -> bool
Returns true if the token should be counted as whitespace, including newlines
Trait Implementations§
source§impl PartialEq for Token
impl PartialEq for Token
impl Copy for Token
impl Eq for Token
impl StructuralPartialEq for Token
Auto Trait Implementations§
impl Freeze for Token
impl RefUnwindSafe for Token
impl Send for Token
impl Sync for Token
impl Unpin for Token
impl UnwindSafe for Token
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more