Trait SourceTokenProvider 

pub trait SourceTokenProvider {
    // Required method
    fn source_tokens(&self) -> &[(Token, ByteRange<usize>)];
}

Trait that any token structure can implement to provide access to source tokens.

This enables the unrolling system to work with any parser’s token representation.

Required Methods

fn source_tokens(&self) -> &[(Token, ByteRange<usize>)]

Get the original Logos tokens that comprise this token.

For atomic tokens (direct from Logos), this returns a slice containing just that token. For aggregate tokens (from transformations), this returns all the original tokens.

Implementors