Struct parol_runtime::lexer::token::Token
source · pub struct Token<'t> {
pub token_type: TerminalIndex,
pub location: Location,
pub token_number: TokenNumber,
/* private fields */
}
Expand description
The Token<’t> type represents a scanned token. It has a reference to the scanned text in the text member.
The lifetime parameter 't
refers to the lifetime of the scanned text.
Fields§
§token_type: TerminalIndex
The index of the terminal in the augmented terminal list
location: Location
Position information
token_number: TokenNumber
Unique token number that allows ordering of tokens from different contexts, e.g. comment tokens can be intermingled with normal tokens.
Implementations§
source§impl<'t> Token<'t>
impl<'t> Token<'t>
sourcepub fn eoi(token_number: TokenNumber) -> Self
pub fn eoi(token_number: TokenNumber) -> Self
Creates an End-Of-Input token
sourcepub fn with<T>(
text: T,
token_type: TerminalIndex,
location: Location,
token_number: TokenNumber
) -> Self
pub fn with<T>( text: T, token_type: TerminalIndex, location: Location, token_number: TokenNumber ) -> Self
Creates a token with given values.
sourcepub fn with_location(self, location: Location) -> Self
pub fn with_location(self, location: Location) -> Self
Change the location of the token after it’s creation
sourcepub fn with_type(self, token_type: TerminalIndex) -> Self
pub fn with_type(self, token_type: TerminalIndex) -> Self
Change the location of the token after it’s creation
sourcepub fn is_skip_token(&self) -> bool
pub fn is_skip_token(&self) -> bool
Indicates whether the token is normally skipped by the TokenStream. The behavior is independent from the language.
sourcepub fn is_comment_token(&self) -> bool
pub fn is_comment_token(&self) -> bool
Indicates whether the token is a comment token.
sourcepub fn to_owned(&self) -> Token<'static>
pub fn to_owned(&self) -> Token<'static>
Creates an owned instance of the token from a shared reference
sourcepub fn into_owned(self) -> Token<'static>
pub fn into_owned(self) -> Token<'static>
Creates an owned instance of the token and consumes self
Trait Implementations§
source§impl FormatToken for Token<'_>
impl FormatToken for Token<'_>
source§impl<'t> Ord for Token<'t>
impl<'t> Ord for Token<'t>
source§impl<'t> PartialEq for Token<'t>
impl<'t> PartialEq for Token<'t>
source§impl<'t> PartialOrd for Token<'t>
impl<'t> PartialOrd for Token<'t>
1.0.0 · source§fn le(&self, other: &Rhs) -> bool
fn le(&self, other: &Rhs) -> bool
self
and other
) and is used by the <=
operator. Read moreimpl<'t> Eq for Token<'t>
impl<'t> StructuralPartialEq for Token<'t>
Auto Trait Implementations§
impl<'t> Freeze for Token<'t>
impl<'t> RefUnwindSafe for Token<'t>
impl<'t> Send for Token<'t>
impl<'t> Sync for Token<'t>
impl<'t> Unpin for Token<'t>
impl<'t> UnwindSafe for Token<'t>
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
source§impl<Q, K> Comparable<K> for Q
impl<Q, K> Comparable<K> for Q
source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
source§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key
and return true
if they are equal.