Struct moore_vhdl_syntax::lexer::tokenizer::Tokenizer [−][src]
pub struct Tokenizer<T: Grinder> { /* fields omitted */ }
Expand description
A grinder that combines character bundles into lexical tokens. This is the last stage of lexical analysis.