pest_derive 1.0.0-beta.5

pest's derive macro
Documentation

pest. The Elegant Parser (beta)

Join the chat at https://gitter.im/dragostis/pest Build Status codecov Crates.io Crates.io Docs

pest is a PEG parser with simplicity and speed in mind.

Elegant grammar

Defining a grammar for a list of alpha-numeric identifiers where the first identifier does not start with a digit is as straight-forward as:

alpha = { 'a'..'z' | 'A'..'Z' }
digit = { '0'..'9' }

ident = { (alpha | digit)+ }

ident_list = _{ !digit ~ ident ~ (" " ~ ident)+ }
          // ^
          // ident_list rule is silent which means it produces no tokens

This is then saved in a .pest grammar file and is never mixed up with Rust code which results in an always up-to-date formal definition of the grammar which is very easy to maintain.

Pairs API

The grammar can be used to derive a Parser implementation automatically. Parsing returns an iterator of nested token pairs:

extern crate pest;
#[macro_use]
extern crate pest_derive;

use pest::Parser;

#[derive(Parser)]
#[grammar = "ident.pest"]
struct IdentParser;

fn main() {
    let pairs = IdentParser::parse_str(Rule::ident_list, "a1 b2").unwrap_or_else(|e| panic!("{}", e));

    // Because ident_list is silent, the iterator will contain idents
    for pair in pairs {
        // A pair is a combination of the rule which matched and a span of input
        println!("Rule:    {:?}", pair.as_rule());
        println!("Span:    {:?}", pair.clone().into_span());
        println!("Text:    {}", pair.clone().into_span().as_str());

        // A pair can be converted to an iterator of the tokens which make it up:
        for inner_pair in pair.into_inner() {
            match inner_pair.as_rule() {
                Rule::alpha => println!("Letter:  {}", inner_pair.into_span().as_str()),
                Rule::digit => println!("Digit:   {}", inner_pair.into_span().as_str()), 
                _ => unreachable!()
            };
        }
    }
}

This produces the following output:

Rule:    ident
Span:    Span { start: 0, end: 2 }
Text:    a1
Letter:  a
Digit:   1
Rule:    ident
Span:    Span { start: 3, end: 5 }
Text:    b2
Letter:  b
Digit:   2

Meaningful error reporting

Parsing "123" instead of "a1 b2" in the code above will result in the following panic:

thread 'main' panicked at ' --> 1:1
  |
1 | 123
  | ^---
  |
  = unexpected digit', src/main.rs:12

while parsing "ab *" will result in:

thread 'main' panicked at ' --> 1:4
  |
1 | ab *
  |    ^---
  |
  = expected ident', src/main.rs:12

Sheer performance

pest provides parsing performance in the same league as carefully written manual parsers. The following JSON benchmark puts it somewhere in between one of the most optimized JSON parsers, ujson4c, and a static native-speed parser, nom.

The first entry of pest scores 22ms, while the second scores 78ms since it's mapping Pairs to a custom JSON AST.

The benchmark uses a large 2MB JSON file. Tested on a 2.6GHz Intel® Core™ i5 running macOS.

Other features

  • precedence climbing
  • input handling
  • custom errors
  • runs on stable Rust

Usage

pest requires Cargo and Rust.

Add the following to Cargo.toml:

pest = "^1.0.0-beta"
pest_derive = "^1.0.0-beta"

and in your Rust lib.rs or main.rs:

extern crate pest;
#[macro_use]
extern crate pest_derive;

Projects using pest