Struct Token 

pub struct Token<'src> {
    pub kind: TokenKind,
    pub text: Cow<'src, str>,
    pub span: Span,
}

A token produced by the lexer.

Token text is borrowed from the source string (Cow::Borrowed) in the common case, avoiding per-token heap allocation. String literals with escape sequences are the only tokens that own their text on the heap.

Fields

kind: TokenKind

Token classification.

text: Cow<'src, str>

Source text of the token — borrowed from input in the common case.

span: Span

Source location.

Implementations

impl<'src> Token<'src>

pub fn text(&self) -> &str

Returns the token text as a &str.

Trait Implementations

impl<'src> Clone for Token<'src>

fn clone(&self) -> Token<'src>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'src> Debug for Token<'src>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'src> PartialEq for Token<'src>

fn eq(&self, other: &Token<'src>) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl<'src> StructuralPartialEq for Token<'src>