Struct Parser 

pub struct Parser<'t, K> { /* private fields */ }

A cursor over a slice of Tokens, with error recovery, that a hand-written recursive-descent grammar drives.

Parser holds the borrowed token stream, the current position, and the diagnostics recorded so far. A grammar is a set of functions that take &mut Parser and return Option<T> — Some(node) on success, None after a recoverable error has been recorded. The cursor skips trivia automatically (anything TokenKind::is_trivia holds for), so the grammar only ever sees significant tokens, and it stops cleanly at the end of input.

Kinds are matched with predicates rather than equality — at(|k| matches!(k, Kind::Plus)) — so a kind that carries data (an interned identifier, a literal) works without a PartialEq bound, and matching a category never accidentally compares the payload.

Examples

use parser_lang::{Parser, Span, Token, TokenKind};

#[derive(Clone, Copy, PartialEq)]
enum Kind { Num, Plus, Eof }
impl TokenKind for Kind {
    fn is_eof(&self) -> bool { matches!(self, Kind::Eof) }
}

// `1 + 2`, terminated.
let tokens = [
    Token::new(Kind::Num, Span::new(0, 1)),
    Token::new(Kind::Plus, Span::new(2, 3)),
    Token::new(Kind::Num, Span::new(4, 5)),
    Token::new(Kind::Eof, Span::empty(5)),
];

let mut p = Parser::new(&tokens);
assert!(p.at(|k| matches!(k, Kind::Num)));
p.bump();
assert!(p.eat(|k| matches!(k, Kind::Plus)).is_some());
assert!(p.at(|k| matches!(k, Kind::Num)));
p.bump();
assert!(p.at_end());

Implementations

impl<'t, K: TokenKind> Parser<'t, K>

pub fn new(tokens: &'t [Token<K>]) -> Self

Creates a cursor over tokens, positioned at the first significant token.

Leading trivia is skipped immediately. The stream need not end with an end-of-input token, but if it does the cursor stops on it rather than running past.

Examples

use parser_lang::{Parser, Span, Token, TokenKind};

#[derive(Clone, Copy)]
enum Kind { Word, Space }
impl TokenKind for Kind {
    fn is_trivia(&self) -> bool { matches!(self, Kind::Space) }
}

// Leading whitespace is skipped on construction.
let tokens = [
    Token::new(Kind::Space, Span::new(0, 1)),
    Token::new(Kind::Word, Span::new(1, 5)),
];
let p = Parser::new(&tokens);
assert!(p.at(|k| matches!(k, Kind::Word)));

pub fn peek(&self) -> Option<&'t Token<K>>

Returns the current significant token without consuming it, or None at the end of input.

Examples

use parser_lang::{Parser, Span, Token, TokenKind};
let tokens = [Token::new(K::A, Span::new(0, 1))];
let p = Parser::new(&tokens);
assert_eq!(p.peek().map(|t| t.span()), Some(Span::new(0, 1)));

pub fn peek_kind(&self) -> Option<&'t K>

Returns the kind of the current significant token, or None at the end.

pub fn span(&self) -> Span

Returns the span of the current token, or an empty span at the end of input (positioned just past the last token), so an error reported at the end still points somewhere sensible.

pub fn at_end(&self) -> bool

Returns true at the end of input: when there is no current token, or the current token is the end-of-input marker.

pub fn at(&self, pred: impl FnOnce(&K) -> bool) -> bool

Returns true if the current token’s kind satisfies pred. Always false at the end of input.

Examples

use parser_lang::{Parser, Span, Token, TokenKind};
let tokens = [Token::new(K::Plus, Span::new(0, 1))];
let p = Parser::new(&tokens);
assert!(p.at(|k| matches!(k, K::Plus)));
assert!(!p.at(|k| matches!(k, K::Minus)));

pub fn bump(&mut self) -> Option<&'t Token<K>>

Consumes and returns the current significant token, advancing to the next one. Returns None (and does not move) at the end of input.

pub fn eat(&mut self, pred: impl FnOnce(&K) -> bool) -> Option<&'t Token<K>>

Consumes the current token if its kind satisfies pred, returning it; otherwise leaves the cursor untouched and returns None.

Examples

use parser_lang::{Parser, Span, Token, TokenKind};
let tokens = [Token::new(K::Num, Span::new(0, 1))];
let mut p = Parser::new(&tokens);
assert!(p.eat(|k| matches!(k, K::Comma)).is_none()); // not a comma
assert!(p.eat(|k| matches!(k, K::Num)).is_some());   // consumed

pub fn expect( &mut self, pred: impl FnOnce(&K) -> bool, description: &str, ) -> Option<&'t Token<K>>

Consumes the current token if its kind satisfies pred; otherwise records an “expected {description}” diagnostic at the current position and returns None.

This is the workhorse for required tokens: the grammar names what it wanted (description), and on a mismatch the error is recorded for later rendering while parsing continues — the caller decides whether to recover.

Examples

use parser_lang::{Parser, Span, Token, TokenKind};
let tokens = [Token::new(K::Num, Span::new(0, 1))];
let mut p = Parser::new(&tokens);
assert!(p.expect(|k| matches!(k, K::RParen), "`)`").is_none());
assert!(p.has_errors());

pub fn error(&mut self, message: impl Into<Box<str>>)

Records an error diagnostic at the current position.

Examples

use parser_lang::{Parser, Span, Token, TokenKind};
let tokens = [Token::new(K::Bad, Span::new(0, 3))];
let mut p = Parser::new(&tokens);
p.error("unexpected token");
assert_eq!(p.errors().len(), 1);

pub fn error_at(&mut self, span: Span, message: impl Into<Box<str>>)

Records an error diagnostic at a specific span — for instance pointing back at an unclosed opening delimiter rather than at the current token.

pub fn recover(&mut self, sync: impl Fn(&K) -> bool)

Skips tokens until the current one satisfies sync, or the end of input is reached, leaving the cursor on the synchronizing token.

This is the recovery primitive: after recording an error, advance to a known landmark (a statement terminator, a closing brace) and resume parsing there, so one malformed construct does not derail the rest of the input. It always makes progress and always stops at the end marker, so it cannot run away.

Examples

use parser_lang::{Parser, Span, Token, TokenKind};
let tokens = [
    Token::new(K::Junk, Span::new(0, 1)),
    Token::new(K::Junk, Span::new(1, 2)),
    Token::new(K::Semi, Span::new(2, 3)),
    Token::new(K::Eof, Span::empty(3)),
];
let mut p = Parser::new(&tokens);
p.recover(|k| matches!(k, K::Semi));
assert!(p.at(|k| matches!(k, K::Semi)));

pub fn repeated<T>( &mut self, parse: impl FnMut(&mut Self) -> Option<T>, ) -> Vec<T>

Parses zero or more items, calling parse until it returns None, and collects the results.

A parse that returns Some without advancing the cursor would loop forever; this guards against that by stopping if no progress was made.

Examples

use parser_lang::{Parser, Span, Token, TokenKind};
let tokens = [
    Token::new(K::Num, Span::new(0, 1)),
    Token::new(K::Num, Span::new(1, 2)),
    Token::new(K::Eof, Span::empty(2)),
];
let mut p = Parser::new(&tokens);
let nums = p.repeated(|p| p.eat(|k| matches!(k, K::Num)).map(|t| t.span()));
assert_eq!(nums.len(), 2);

pub fn separated<T>( &mut self, sep: impl FnMut(&K) -> bool, parse: impl FnMut(&mut Self) -> Option<T>, ) -> Vec<T>

Parses a possibly-empty list of items produced by parse, separated by tokens matching sep (such as a comma), and collects the results.

Parsing stops after a separator that is not followed by another item (a trailing separator), or when parse first returns None (an empty list).

Examples

use parser_lang::{Parser, Span, Token, TokenKind};
// `1, 2, 3`
let tokens = [
    Token::new(K::Num, Span::new(0, 1)),
    Token::new(K::Comma, Span::new(1, 2)),
    Token::new(K::Num, Span::new(3, 4)),
    Token::new(K::Comma, Span::new(4, 5)),
    Token::new(K::Num, Span::new(6, 7)),
    Token::new(K::Eof, Span::empty(7)),
];
let mut p = Parser::new(&tokens);
let items = p.separated(
    |k| matches!(k, K::Comma),
    |p| p.eat(|k| matches!(k, K::Num)).map(|t| t.span()),
);
assert_eq!(items.len(), 3);

pub fn checkpoint(&self) -> Checkpoint

Takes a snapshot of the cursor and the error count, for speculative parsing.

Pair it with rewind to try a parse and back out of it cleanly if it does not work — the cursor returns to where it was and any diagnostics recorded in the meantime are dropped.

pub fn rewind(&mut self, checkpoint: Checkpoint)

Restores the cursor and error log to a Checkpoint, undoing everything done since it was taken.

Examples

use parser_lang::{Parser, Span, Token, TokenKind};
let tokens = [Token::new(K::Num, Span::new(0, 1))];
let mut p = Parser::new(&tokens);
let cp = p.checkpoint();
p.bump();
p.error("speculative");
p.rewind(cp); // cursor and the recorded error are both rolled back
assert!(!p.has_errors());
assert!(p.at(|k| matches!(k, K::Num)));

pub fn errors(&self) -> &[Diagnostic]

Returns the diagnostics recorded so far, in the order they occurred.

pub fn has_errors(&self) -> bool

Returns true if any diagnostic has been recorded.

pub fn into_errors(self) -> Vec<Diagnostic>

Consumes the parser, returning all recorded diagnostics in source order.