Struct koto_lexer::LexedToken

pub struct LexedToken {
    pub token: Token,
    pub source_bytes: Range<usize>,
    pub span: Span,
    pub indent: usize,
}

A Token along with additional metadata

Fields

token: Token

The token

source_bytes: Range<usize>

The byte positions in the source representing the token

span: Span

The token’s span

indent: usize

The indentation level of the token’s starting line

Implementations

impl LexedToken

pub fn line(&self) -> u32

A helper for getting the token’s starting line

pub fn slice<'a>(&self, source: &'a str) -> &'a str

A helper for getting the token’s string slice from the source

Trait Implementations

impl Clone for LexedToken

fn clone(&self) -> LexedToken

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for LexedToken

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for LexedToken

fn default() -> Self

Returns the “default value” for a type.

impl PartialEq for LexedToken

fn eq(&self, other: &LexedToken) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for LexedToken