Crate logos_nom_bridge

logos-nom-bridge

A logos::Lexer wrapper than can be used as an input for nom.

Simple example

// First, create a `logos` lexer:

#[derive(Clone, Debug, PartialEq, Eq, logos::Logos)]
enum Token {
    #[token("+")]
    Plus,

    #[token("-")]
    Minus,

    #[regex(r"-?[0-9]+", |lex| lex.slice().parse())]
    Number(i64),

    #[error]
    #[regex(r"[ \t\n\f]+", logos::skip)]
    Error,
}

// Then, write a nom parser that accepts a `Tokens<'_, Token>` as input:

use logos_nom_bridge::Tokens;

type Input<'source> = Tokens<'source, Token>;

#[derive(Debug, PartialEq, Eq)]
enum Op {
    Number(i64),
    Addition(Box<(Op, Op)>),
    Subtraction(Box<(Op, Op)>),
}

fn parse_expression(input: Input<'_>) -> nom::IResult<Input<'_>, Op> {
    // zip
}

// Finally, you can use it to parse a string:

let input = "10 + 3 - 4";
let tokens = Tokens::new(input);

let (rest, parsed) = parse_expression(tokens).unwrap();

assert!(rest.is_empty());
assert_eq!(
    parsed,
    Op::Addition(Box::new((
        Op::Number(10),
        Op::Subtraction(Box::new((
            Op::Number(3),
            Op::Number(4),
        ))),
    ))),
)

Macros

You can implement nom::Parser for your token type with the token_parser macro:

logos_nom_bridge::token_parser!(token: Token);

If some enum variants of your token type contain data, you can implement a nom::Parser for them using the data_variant_parser macro:

#[derive(Clone, Debug, PartialEq, Eq, logos::Logos)]
enum Token {
    #[regex(r"-?[0-9]+", |lex| lex.slice().parse())]
    Number(i64),

    // etc.
}

logos_nom_bridge::data_variant_parser! {
    fn parse_number(input) -> Result<Op>;
    pattern = Token::Number(n) => Op::Number(n);
}

Macros

data_variant_parser

Generates a nom parser function to parse an enum variant that contains data.

token_parser

Automatically implements nom::Parser for your token type.

Structs

IndexIterator

An iterator, that (similarly to std::iter::Enumerate) produces byte offsets of the tokens.

Tokens

A logos::Lexer wrapper than can be used as an input for nom.