Crate logos_nom_bridge
source · [−]Expand description
logos-nom-bridge
A logos::Lexer
wrapper than can be used as an input for
nom.
Simple example
// First, create a `logos` lexer:
#[derive(Clone, Debug, PartialEq, Eq, logos::Logos)]
enum Token {
#[token("+")]
Plus,
#[token("-")]
Minus,
#[regex(r"-?[0-9]+", |lex| lex.slice().parse())]
Number(i64),
#[error]
#[regex(r"[ \t\n\f]+", logos::skip)]
Error,
}
// Then, write a nom parser that accepts a `Tokens<'_, Token>` as input:
use logos_nom_bridge::Tokens;
type Input<'source> = Tokens<'source, Token>;
#[derive(Debug, PartialEq, Eq)]
enum Op {
Number(i64),
Addition(Box<(Op, Op)>),
Subtraction(Box<(Op, Op)>),
}
fn parse_expression(input: Input<'_>) -> nom::IResult<Input<'_>, Op> {
// zip
}
// Finally, you can use it to parse a string:
let input = "10 + 3 - 4";
let tokens = Tokens::new(input);
let (rest, parsed) = parse_expression(tokens).unwrap();
assert!(rest.is_empty());
assert_eq!(
parsed,
Op::Addition(Box::new((
Op::Number(10),
Op::Subtraction(Box::new((
Op::Number(3),
Op::Number(4),
))),
))),
)
Macros
You can implement nom::Parser
for your token type with the token_parser
macro:
logos_nom_bridge::token_parser!(token: Token);
If some enum variants of your token type contain data, you can implement a nom::Parser
for them using the data_variant_parser
macro:
#[derive(Clone, Debug, PartialEq, Eq, logos::Logos)]
enum Token {
#[regex(r"-?[0-9]+", |lex| lex.slice().parse())]
Number(i64),
// etc.
}
logos_nom_bridge::data_variant_parser! {
fn parse_number(input) -> Result<Op>;
pattern = Token::Number(n) => Op::Number(n);
}
Macros
Generates a nom parser function to parse an enum variant that contains data.
Automatically implements nom::Parser
for your token type.
Structs
An iterator, that (similarly to std::iter::Enumerate
) produces byte offsets of the tokens.
A logos::Lexer
wrapper than can be used as an input for
nom.