Struct regex_lexer::LexerBuilder

pub struct LexerBuilder<'r, 't, T: 't> { /* fields omitted */ }

Builder struct for Lexer.

Implementations

impl<'r, 't, T: 't> LexerBuilder<'r, 't, T>

pub fn new() -> Self

Create a new LexerBuilder.

pub fn token<F>(self, re: &'r str, f: F) -> Self where
    F: Fn(&'t str) -> Option<T> + 'static, 

Add a new token that matches the regular expression re. This uses the same syntax as the regex crate.

If re gives the longest match, then f is called on the matched string.

• If f returns Some(tok), emit the token tok.
• Otherwise, skip this token and emit nothing.

#[derive(Debug, PartialEq, Eq)]
enum Token {
    Num(usize),
    // ...
}

let lexer = regex_lexer::LexerBuilder::new()
    .token(r"[0-9]*", |num| Some(Token::Num(num.parse().unwrap())))
    .token(r"\s+", |_| None) // skip whitespace
    // ...
    .build()?;

assert_eq!(
    lexer.tokens("1 2 3").collect::<Vec<_>>(),
    vec![Token::Num(1), Token::Num(2), Token::Num(3)],
);

If multiple regexes all have the same longest match, then whichever is defined last is given priority.

#[derive(Debug, PartialEq, Eq)]
enum Token<'t> {
    Ident(&'t str),
    Then,
    // ...
}

let lexer = regex_lexer::LexerBuilder::new()
    .token(r"[a-zA-Z_][a-zA-Z0-9_]*", |id| Some(Token::Ident(id)))
    .token(r"then", |_| Some(Token::Then))
    // ...
    .build()?;

assert_eq!(lexer.tokens("then").next(), Some(Token::Then));
assert_eq!(lexer.tokens("then_perish").next(), Some(Token::Ident("then_perish")));

pub fn build(self) -> Result<Lexer<'t, T>, Error>

Construct a Lexer which matches these tokens.

Errors

If a regex cannot be compiled, a regex::Error is returned.

Trait Implementations

impl<'r, 't, T: 't> Debug for LexerBuilder<'r, 't, T>

fn fmt(&self, f: &mut Formatter) -> Result

Shows the matched regexes

impl<'r, 't, T: 't> Default for LexerBuilder<'r, 't, T>

fn default() -> Self

Returns the "default value" for a type.