Struct Token 

pub struct Token {
    pub id: TokenId,
    pub index: usize,
    pub length: usize,
}

A token in Earl contains where it appears in the code and its type or ID.

• The field id is what type of token this is.
• The field index is the index of the byte where this token begins.
• The field length is how many bytes this token spans.

Fields

id: TokenId

The ID or type of token.

index: usize

The index of where the token appears in the code.

length: usize

The length of the token in the code.

Implementations

impl Token

pub fn new(id: TokenId, index: usize, length: usize) -> Token

Creates a new token.

pub fn get_lexeme<'a>(&self, code: &'a str) -> &'a str

Returns the lexeme of this code.

Trait Implementations

impl Clone for Token

fn clone(&self) -> Token

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Token

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.