Struct commands::tokenizer::Token

pub struct Token<'text> {
    pub text: &'text str,
    pub token_type: TokenType,
    pub location: SourceLocation,
}

A token from a body of text.

The lifetime parameter 'text refers to the lifetime of the body of text that was tokenized, creating this token.

Fields

text: &'text str

The text of the token.

token_type: TokenType

The type of the token (Whitespace or Word).

location: SourceLocation

The location of the token in the source body of text.

Methods

impl<'text> Token<'text>

fn new(
    text: &'text str,
    token_type: TokenType,
    location: SourceLocation
) -> Token

Construct a Token. The lifetime parameter 'text refers to the lifetime of the text being tokenized.

Trait Implementations

impl<'text> Clone for Token<'text>

fn clone(&self) -> Token<'text>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'text> Copy for Token<'text>

impl<'text> Debug for Token<'text>

fn fmt(&self, __arg_0: &mut Formatter) -> Result

Formats the value using the given formatter.

impl<'text> PartialEq for Token<'text>

fn eq(&self, __arg_0: &Token<'text>) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, __arg_0: &Token<'text>) -> bool

This method tests for !=.