Luthor provides a collection of lexers for various formats and languages. It also exposes types that aid in building lexers of your own.
Pre-built lexers for various languages/formats.
A recursive function type used by lexers to manage their state.
Based on Rob Pike's "Lexical Scanning in Go" talk, these functions are
invoked in a call/return loop (letting the current function determine
the next) until a
The Tokenizer type is used to produce and store tokens for lexers.