Struct Token 

pub struct Token<N> {
    pub name: N,
    pub str: String,
    pub pos: TextPosition,
}

A specific occurence of a token in the text that was tokenized.

Contains the name of the token type, str, a String that contains the text the token matched, and pos, the TextPosition of the token in the text.

Fields

name: N

str: String

pos: TextPosition

Implementations

impl<N> Token<N>

pub fn new(name: N, str: String, pos: TextPosition) -> Self

Creates a new token.

Examples

use sprout::tokenize::Token;
 
enum TokenName { ABC }
 
Token::new(TokenName::ABC, "abc".to_string(), TextPosition::new(4, 20, 69));

Trait Implementations

impl<N: Clone> Clone for Token<N>

fn clone(&self) -> Token<N>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<N: Debug> Debug for Token<N>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<N: Display> Display for Token<N>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<N: PartialEq> PartialEq for Token<N>

fn eq(&self, other: &Token<N>) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl<N: Eq> Eq for Token<N>

impl<N> StructuralPartialEq for Token<N>