Expand description
The token buffer the parser reads.
Design: spec/06-lexer-and-parser.md section 6.3.
The input is the slice of tokens phase 7 produced, which always ends in one
TokenKind::Eof. A cursor is a position in that slice and nothing more: it holds no
parser state, it builds nothing and it reports nothing. That is what makes saving one and
restoring it safe, and it is why the recovery skips in crate::recover take a cursor
rather than the whole parser.
§Why the lookahead is bounded
Unbounded backtracking is how a C parser becomes quadratic on the input a fuzzer eventually
finds, so Cursor::peek refuses to look further than MAX_LOOKAHEAD tokens ahead and
panics rather than quietly widening the window. The bound is a budget rather than a fact
about the grammar: a decision that cannot be made inside it is either a save and a restore,
which is deliberate and rare, or a sign that the decision is being made in the wrong place,
and the panic is how that conversation starts.
Structs§
- Cursor
- A position in the token stream.
- Mark
- A saved position, taken by
Cursor::saveand given back toCursor::restore.
Constants§
- MAX_
LOOKAHEAD - How far ahead
Cursor::peekwill look.