Trait PatternLexer 

pub trait PatternLexer {
    // Required method
    fn tokenize_chunk(
        &mut self,
        text: &str,
    ) -> Result<Vec<TokenSpec>, ParseTreePatternError>;
}

Lexes one literal pattern chunk into the token specs it produces.

The matcher owns all split/tag/interpret logic; this trait is the single grammar-specific hook, supplying the real lexer’s output for a run of concrete input text. The trailing EOF must be excluded; off-default-channel tokens (whitespace, comments) may be returned on their own channel and are skipped by the interpreter exactly as in a normal parse, matching ANTLR’s tokenize. Implemented for FnMut(&str) -> Result<Vec<TokenSpec>, ParseTreePatternError> so a closure suffices.

Required Methods

fn tokenize_chunk( &mut self, text: &str, ) -> Result<Vec<TokenSpec>, ParseTreePatternError>

Tokenizes text into token specs (no EOF), each on its original channel.

Errors

Returns a ParseTreePatternError::Tokenization if the lexer rejects the chunk.

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl<F> PatternLexer for F

where F: FnMut(&str) -> Result<Vec<TokenSpec>, ParseTreePatternError>,