pub struct Lexeme<LexemeType> {
pub lexeme_type: LexemeType,
pub contents: String,
}
Expand description
A lexeme extracted from input text by a lexical analyzers.
Lexemes, also known as “tokens”, are sequences of consecutive characters separated from input text, and classified into categories (such as keywords, identifiers, operators), during the lexical analysis phase of the syntax-parsing pipeline. They represent atomic units of syntactic meaning.
Fields§
§lexeme_type: LexemeType
The type (category) of the lexeme.
contents: String
The original text that constituted the lexeme.
Implementations§
Trait Implementations§
impl<LexemeType: Eq> Eq for Lexeme<LexemeType>
impl<LexemeType> StructuralPartialEq for Lexeme<LexemeType>
Auto Trait Implementations§
impl<LexemeType> Freeze for Lexeme<LexemeType>where
LexemeType: Freeze,
impl<LexemeType> RefUnwindSafe for Lexeme<LexemeType>where
LexemeType: RefUnwindSafe,
impl<LexemeType> Send for Lexeme<LexemeType>where
LexemeType: Send,
impl<LexemeType> Sync for Lexeme<LexemeType>where
LexemeType: Sync,
impl<LexemeType> Unpin for Lexeme<LexemeType>where
LexemeType: Unpin,
impl<LexemeType> UnwindSafe for Lexeme<LexemeType>where
LexemeType: UnwindSafe,
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more