Struct TextToken 

pub struct TextToken {
    pub kind: String,
    pub text: String,
    pub offset: usize,
}

A simple text-based tokeniser for round-trip validation.

Fields

kind: String

Token kind label

text: String

Raw text of the token

offset: usize

Byte offset in the source

Trait Implementations

impl Clone for TextToken

fn clone(&self) -> TextToken

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TextToken

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.