Module common 

Common parsing utilities shared across language implementations.

Every parser here consumes Tokens produced by lex rather than characters. Lexing first is what makes keyword matching whole-word matching: word_ci("day") compares the entire Word("days") slice and fails, where the old character-level keyword_ci("day") matched the prefix and needed a hand-rolled word-boundary assertion plus longest-first ordering to stay correct.

Writing a parser against this module

Parsers are generic over the input so they compose with whatever concrete token stream the caller builds:

use chumsky::prelude::*;
use temps_core::common::{ParserError, TokenInput, word_ci};

fn now_expr<'t, 's: 't, I>() -> impl Parser<'t, I, (), ParserError<'t, 's>> + Clone
where
    I: TokenInput<'t, 's>,
{
    word_ci("now")
}

and are driven by lexing the source and mapping the token slice into an input:

use chumsky::prelude::*;
use temps_core::{common::{token_stream, word_ci}, lexer::lex};

let input = "now";
let tokens = lex(input);
let result = word_ci("now")
    .then_ignore(end())
    .parse(token_stream(input, &tokens))
    .into_result();
assert!(result.is_ok());

Traits

TokenInput

The input bound every parser in this module is generic over.

Functions

digit_number

Parse a Token::Number of any width as an i64.

four_digit_number

Parse an exactly-4-digit Token::Number as a u16.

iso_datetime

Parse ISO 8601 datetime format.

opt_space

Match an optional Token::Space.

phrase_ci

Match a multi-token phrase case-insensitively, e.g. phrase_ci("day after tomorrow") or phrase_ci("a.m.").

phrase_cs

Case-sensitive counterpart of phrase_ci.

phrases_ci

Build a case-insensitive alternation over (phrase, value) pairs, trying the phrase with the most tokens first.

phrases_cs

Case-sensitive counterpart of phrases_ci.

punct

Match a single punctuation character, e.g. punct(':').

space

Match exactly one Token::Space.

token_stream

Turn a source string and its lexed tokens into a parser input.

two_digit_number

Parse a 1 or 2 digit Token::Number as a u8.

word_ci

Match a whole Token::Word against target, case-insensitively.

word_cs

Match a whole Token::Word against target, case-sensitively.

Type Aliases

BoxedParser

A boxed token parser.

ParserError

The error type used throughout the parsers.