Struct Token 

pub struct Token<K: TokenKind> {
    pub kind: K,
    pub span: Span,
    pub lexeme: String,
    pub leading_trivia: Vec<TokenTrivia<K>>,
    pub trailing_trivia: Vec<TokenTrivia<K>>,
}

Token emitted by the lexer.

Fields

kind: K

Token kind enum.

span: Span

Source span covered by the token.

lexeme: String

Original textual lexeme.

leading_trivia: Vec<TokenTrivia<K>>

Trivia that appears before the token.

trailing_trivia: Vec<TokenTrivia<K>>

Trivia that appears immediately after the token.

Implementations

impl<K: TokenKind> Token<K>

pub fn new<'fn_lifetime>() -> TokenBuilder<'fn_lifetime, K, (), (), (), (), (), (), ()>

Creating a builder.

Required Fields

kind

• Type: K

Token kind enum.

lexeme

• Type: Into<String>

Original textual lexeme.

span

• Type: Span

Source span covered by the token.

Optional Fields

leading_trivia

• Type: Vec < TokenTrivia < K > >
• Default: Vec :: new()

Trivia that appears before the token.

trailing_trivia

• Type: Vec < TokenTrivia < K > >
• Default: Vec :: new()

Trivia that appears immediately after the token.

impl<K: TokenKind> Token<K>

pub fn create( kind: K, span: Span, lexeme: impl Into<String>, leading_trivia: Vec<TokenTrivia<K>>, trailing_trivia: Vec<TokenTrivia<K>>, ) -> Self

Construct a new token with the provided trivia.

For builder pattern usage, use TokenBuilder::new() instead.

pub fn with_trivia( self, leading: Vec<TokenTrivia<K>>, trailing: Vec<TokenTrivia<K>>, ) -> Self

Create a token with attached trivia.

Trait Implementations

impl<K: Clone + TokenKind> Clone for Token<K>

fn clone(&self) -> Token<K>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<K: Debug + TokenKind> Debug for Token<K>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.