Struct Cursor 

pub struct Cursor<'a> { /* private fields */ }

A position in the token stream.

Implementations

impl<'a> Cursor<'a>

pub fn new(tokens: &'a [Token]) -> Self

A cursor on the first token of tokens.

Panics

Panics if tokens does not end in TokenKind::Eof. Every method here relies on that token being there: it is what a peek past the end returns and it is what stops every recovery skip, so a stream without one would turn a malformed file into a hang.

pub fn current(&self) -> Token

The token the parser is looking at.

pub fn peek(&self, n: usize) -> Token

The token n places ahead, which is the final TokenKind::Eof once the end is reached rather than an out of range access.

Panics

Panics if n is greater than MAX_LOOKAHEAD.

pub fn span(&self) -> Span

Where the current token is, which is where a diagnostic about it points.

pub fn prev_end(&self) -> Span

The empty span just after the previous token, which is where something that should have been written and was not belongs.

Pointing a missing semicolon at the token that follows it is a small thing that reads badly, because the token that follows is usually on the next line and is not the problem. Before the first token there is no previous one, so this is the start of the current token instead.

pub fn is_eof(&self) -> bool

Whether the parser has reached the end of the translation unit.

pub fn at(&self, kind: TokenKind) -> bool

Whether the current token is exactly kind.

pub fn at_punct(&self, punct: Punct) -> bool

Whether the current token is the punctuator punct.

pub fn at_keyword(&self, keyword: Keyword) -> bool

Whether the current token is the keyword keyword.

pub fn bump(&mut self) -> Token

Steps over the current token and gives it back.

Stepping over the end is not an error and does not move: the cursor stays on the final TokenKind::Eof, so a loop that forgets to check for the end runs out of tokens instead of reading off the end of the slice. It will still spin, which is what Cursor::index is for.

pub fn eat(&mut self, kind: TokenKind) -> bool

Steps over the current token if it is kind, and reports whether it did.

pub fn eat_punct(&mut self, punct: Punct) -> bool

Steps over the current token if it is the punctuator punct.

pub fn eat_keyword(&mut self, keyword: Keyword) -> bool

Steps over the current token if it is the keyword keyword.

pub fn index(&self) -> usize

How many tokens the cursor has stepped over.

The parser’s loops compare this across an iteration to check that they made progress. A production that returns without consuming anything is the classic way a recursive descent parser hangs on malformed input, and it is a bug in the parser rather than something to recover from, so the check belongs in an assertion and not in a if.

pub fn save(&self) -> Mark

The current position, to be given back to Cursor::restore.

pub fn restore(&mut self, mark: Mark)

Goes back to a saved position.

This is not a general backtrack and it does not undo anything but the position. Between a save and a restore the parser must not report a diagnostic and must not put a node in the tree, because neither is taken back, and a speculative parse that leaves either behind produces an error about a reading of the source that was abandoned. The two constructs that need this are in spec/06-lexer-and-parser.md section 6.4.

Panics

Panics if mark came from a cursor on a different token stream.

Trait Implementations

impl<'a> Clone for Cursor<'a>

fn clone(&self) -> Cursor<'a>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'a> Debug for Cursor<'a>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.