Struct LexerBuilder

pub struct LexerBuilder<'r, 't, T: 't> { /* private fields */ }

Builder struct for Lexer.

Implementations

impl<'r, 't, T: 't> LexerBuilder<'r, 't, T>

pub fn new() -> Self

Create a new LexerBuilder.

pub fn token<F>(self, re: &'r str, f: F) -> Self

where F: Fn(Location, &'t str, Location) -> Option<T> + 'static,

Add a new token that matches the regular expression re. This uses the same syntax as the regex crate.

If re gives the longest match, then f is called on the matched string.

• If f returns Some(tok), emit the token tok.
• Otherwise, skip this token and emit nothing.

#[derive(Debug, PartialEq, Eq)]
enum Token {
    Num(usize),
    // ...
}

let lexer = regex_lexer_lalrpop::LexerBuilder::new()
    .token(r"[0-9]*", |_, num, _| Some(Token::Num(num.parse().unwrap())))
    .token(r"\s+", |_, _, _| None) // skip whitespace
    // ...
    .build()?;

assert_eq!(
    lexer.tokens("1 2 3").collect::<Vec<_>>(),
    vec![Ok(Token::Num(1)), Ok(Token::Num(2)), Ok(Token::Num(3))],
);

If multiple regexes all have the same longest match, then whichever is defined last is given priority.

#[derive(Debug, PartialEq, Eq)]
enum Token<'t> {
    Ident(&'t str),
    Then,
    // ...
}

let lexer = regex_lexer_lalrpop::LexerBuilder::new()
    .token(r"[a-zA-Z_][a-zA-Z0-9_]*", |_, id, _| Some(Token::Ident(id)))
    .token(r"then", |_, _, _| Some(Token::Then))
    // ...
    .build()?;

assert_eq!(lexer.tokens("then").next(), Some(Ok(Token::Then)));
assert_eq!(lexer.tokens("then_perish").next(), Some(Ok(Token::Ident("then_perish"))));

pub fn build(self) -> Result<Lexer<'t, T>, Error>

Construct a Lexer which matches these tokens.

Errors

If a regex cannot be compiled, a regex::Error is returned.

Trait Implementations

impl<'r, 't, T: 't> Debug for LexerBuilder<'r, 't, T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Shows the matched regexes

impl<'r, 't, T: 't> Default for LexerBuilder<'r, 't, T>

fn default() -> Self

Returns the “default value” for a type.