regex-lexer 0.2.0

A regex-based lexer (tokenizer)
Documentation

regex-lexer

A regex-based lexer (tokenizer) in Rust.

Basic Usage

enum Tok {
    Num,
    // ...
}

let lexer = regex_lexer::LexerBuilder::new()
  .token(r"[0-9]+", Tok::Num)
  .ignore(r"\s+") // skip whitespace
  // ...
  .build();
  
let tokens = lexer.tokens(/* source */);

License

Licensed under either of

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusing in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.