marser

marser is a parser-combinator library for writing PEG-style grammars in Rust with a focus on useful errors, error recovery and good performance.

Why build yet another parser-combinator library? You can read about it here.

It supports:

• Zero-copy parsing for faster parsers
• Multiple input formats - use &str and &[u8] / &[T] or implement the Input trait yourself.
• Packrat-style caching - just wrap your parsers in .memoized() to cache results at each position.
• Simple debugging of your parsers using a custom TUI
• no_std

Quickstart

To add this library to your Rust project run:

cargo add marser

This library has a couple of optional features. You can find them below.

Example

This example parses dice notation like 2d6 into a struct:

use marser::capture;
use marser::matcher::one_or_more;
use marser::parser::Parser;

// the struct we want to parse into
#[derive(Debug, PartialEq)]
struct Roll {
    count: u32,
    sides: u32,
}

// A parser that can parse a number
fn number<'src>() -> impl Parser<'src, &'src str, Output = u32> + Clone {
    // capture defines a parser. It consists of a matcher (the part before `=>`) 
    // and a Rust expression that builds the output value (the part after `=>`).
    capture!(
        bind_slice!( // bind_slice! stores the matched part of the input inside a variable
            one_or_more('0'..='9'), // matches any sequence of digits
            number_slice as &'src str // the matched digits are available as `number_slice` of type `&'src str`
        )
        => // we can then define how to build the output value from the bound variables
            number_slice // we use the captured slice of digits
                .parse() // and parse it into a u32
                .expect("matched only digits")
    )
}

// A parser that can parse a roll like `2d6`
fn roll<'src>() -> impl Parser<'src, &'src str, Output = Roll> + Clone {
    // we again define a parser with capture!, this time for the whole roll
    capture!(
        ( // we define a sequence by putting multiple matchers in a tuple
          // they are matched one after another
            bind!(number(), count), // first we expect a number. We use bind! to store its value in `count`
            'd', // then we expect the literal character 'd'
            bind!(number(), sides) // then we expect another number, which we store in `sides`
        )
        => // finally we define how to build the output value from the bound variables
            Roll { count, sides }
    )
}

fn main() {
    // we can then use this parser we defined to parse a string
    let (roll, _errors) = roll().parse_str("2d6").unwrap();
    assert_eq!(roll, Roll { count: 2, sides: 6 });
}

Runnable examples live under examples/ (see also below).

Learn more

• Guide on docs.rs
• API documentation
• crates.io
• Design rationale: A Grammar-First Approach to Parser Combinators in Rust

Cargo features

Requirements

• Rust 1.88 or later

Examples in this repository

Examples need the annotate-snippets feature for rendering of errors

Run JSON from a git clone:

cargo run --example json --features annotate-snippets -- tests/data/json1.json

Error output sample

Input:

{
    "foo": 123,
    "bar": [1, ,2 ,3
}

Example diagnostic, rendered using annotate-snippets:

This parser can also still produce a recovered output:

{
    "foo": 123,
    "bar": [
        1,
        2,
        3
    ]
}

The json example also has tracing support, so parsing can be stepped through in the trace viewer. See screenshot below. Left side is the rust source code for the parser, right side is the file being parsed.

Performance compared to other libraries:

Below is a comparison of the speed of different libraries for parsing json, including marser. I used json because there are already parsers using different libraries written for it

Code for other libraries taken from parse-rosetta. Read more here.

The difference in speed between the marser implementation with error recovery and diagnostics ("marser") and the implementation without error recovery and diagnostics ("marser-bare") is quite small because marser works in two modes. First the parser is run without error recovery logic. If the parser encounters an error, it is restarted with error recovery included. This makes it so that the performance cost of including error recovery and diagnostics is only very little.

Early release

Early release: marser is my first published Rust library. Feedback on the API, error messages, and docs is welcome — open an issue

License

This project is licensed under the MIT License.

AI assistance

Parts of this repository were drafted or expanded with AI tools (guide, library docs, tests, macros, trace crates, examples, and parts of this README). The maintainer reviewed this material. If you spot a mistake, please open an issue.