Forth Lexer
Given the forth program:
: add1 ( n -- n )
1 + \ adds one
;
Here's the output you'll get from this parser (an excerpt from our tests in parser.rs):
let mut lexer = new;
let tokens = lexer.parse;
let expected = vec!;
assert_eq!
If you use ropey
you can get the actual slice for a token by
let progn = "word1 word2 word3";
let rope = from_str;
let mut lexer = new;
let tokens = lexer.parse;
// Let's get the `Data<String>` second `Word` from the list
let word2 = if let Some = tokens.get else ;
let x = rope.slice; // Data implements RangeBounds
assert_eq!;
assert_eq!;