lexer-generator

This crate is a small scale lexer package which is parsed from JSON

Example: Basic Tokenizing

Potential code one might use for lexing tokens for a calculator

key.json:

{
"literals": {
"number": "[0-9]*[0-9]",
"subtract": "-",
"add": "\\+",
"divide": "/",
"multiply": "\\*" 
},
"whitespace": "\n| |\r|\t"
}

main.rs:

let json: String = std::fs::read_to_string("key.json").unwrap();
let source: String = String::from("123 + 456 * 789");

let mut lexer = Lexer::from(json, source);
// parsing, runtime, whatever one would want to do with their tokens

"123 + 456 * 789" -> Token("number", "123"), Token("add", "*"), Token("number", "456"), Token("multiply", "*"), Token("number", "789") // ignoring line position and the incremental nature of the lexer