Crate chomp [−] [src]

Chomp is a fast monadic-style parser combinator library for the Rust programming language. It was written as the culmination of the experiments detailed in these blog posts:

For its current capabilities, you will find that Chomp performs consistently as well, if not better, than optimized C parsers, while being vastly more expressive. For an example that builds a performant HTTP parser out of smaller parsers, see http_parser.rs.

Example

use chomp::{Input, U8Result, parse_only};
use chomp::{take_while1, token};

#[derive(Debug, Eq, PartialEq)]
struct Name<'a> {
    first: &'a [u8],
    last:  &'a [u8],
}

fn name(i: Input<u8>) -> U8Result<Name> {
    parse!{i;
        let first = take_while1(|c| c != b' ');
                    token(b' ');  // skipping this char
        let last  = take_while1(|c| c != b'\n');

        ret Name{
            first: first,
            last:  last,
        }
    }
}

assert_eq!(parse_only(name, "Martin Wernstål\n".as_bytes()), Ok(Name{
    first: b"Martin",
    last: "Wernstål".as_bytes()
}));

Usage

Chomp's functionality is split between three modules:

parsers contains the basic parsers used to parse streams of input.
combinators contains functions which take parsers and return new ones.
primitives contains the building blocks used to make new parsers. This is advanced usage and is far more involved than using the pre-existing parsers, but is sometimes unavoidable.

A parser is, at its simplest, a function that takes a slice of input and returns a ParserResult<I, T, E>, where I, T, and E are the input, output, and error types, respectively. Parsers are usually parameterized over values or other parsers as well, so these appear as extra arguments in the parsing function. As an example, here is the signature of the token parser, which matches a particular input.

fn token<I: Copy + Eq>(i: Input<I>, t: I) -> SimpleResult<I, I> {...}

Notice that the first argument is an Input<I>, and the second argument is some I. Input<I> is just a datatype over the current state of the parser and a slice of input I, and prevents the parser writer from accidentally mutating the state of the parser. Later, when we introduce the parse! macro, we will see that using a parser in this macro just means supplying all of the arguments but the input, as so:

token(b'T');

Note that you cannot do this outside of the parse! macro. SimpleResult<I, T> is a convenience type alias over ParseResult<I, T, Error<u8>>, and Error<I> is just a convenient "default" error type that will be sufficient for most uses. For more sophisticated usage, one can always write a custom error type.

A very useful parser is the satisfy parser:

fn satisfy<I: Copy, F>(i: Input<I>, f: F) -> SimpleResult<I, I>
   where F: FnOnce(I) -> bool { ... }

Besides the input state, satisfy's only parameter is a predicate function and will succeed only if the next piece of input satisfies the supplied predicate. Here's an example that might be used in the parse! macro:

satisfy(|c| {
    match c {
        b'c' | b'h' | b'a' | b'r' => true,
        _ => false,
    }
})

This parser will only succeed if the character is one of the characters in "char".

Lastly, here is the parser combinator count, which will attempt to run a parser a number of times on its input.

pub fn count<'a, I, T, E, F, U>(i: Input<'a, I>, num: usize, p: F) -> ParseResult<'a, I, T, E>
  where I: Copy,
        U: 'a,
        F: FnMut(Input<'a, I>) -> ParseResult<'a, I, U, E>,
        T: FromIterator<U> { ... }

Using parsers is almost entirely done using the parse! macro, which enables us to do three distinct things:

Sequence parsers over the remaining input
Store intermediate results into datatypes
Return a datatype at the end, which may be the result of any arbitrary computation over the intermediate results.

In other words, just as a normal Rust function usually looks something like this:

fn f() -> (u8, u8, u8) {
    let a = read_number();
    let b = read_number();
    launch_missiles();
    return (a, b, a + b);
}

A Chomp parser with a similar structure looks like this:

fn f(i: Input<u8>) -> U8Result<(u8, u8, u8)> {
    parse!{i;
        let a = digit();
        let b = digit();
                string(b"missiles");
        ret (a, b, a + b)
    }
}

fn digit(i: Input<u8>) -> U8Result<u8> {
    satisfy(i, |c| b'0' <= c && c <= b'9').map(|c| c - b'0')
}

Readers familiar with Haskell or F# will recognize this as a "monadic computation" or "computation expression".

You use the parse! macro as follows:

Write the input parameter first, with a semicolon.
Write any number of valid parser actions or identifier bindings, where:
- a parser action takes the form parser(params*), with the input parameter omitted.
- an identifier binding takes the form let identifer = parser(params*);, with the input parameter omitted.
Write the final line of the macro, which must always be either a parser action, or a return statement which takes the form ret expression. The type of expression becomes the return type of the entire parser, should it succeed.

The entire grammar for the macro is listed elsewhere in this documentation.

Reexports

pub use combinators::{count, option, or, many, many1, sep_by, sep_by1, many_till, skip_many, skip_many1, matched_by};

pub use parsers::{any, eof, not_token, peek, peek_next, satisfy, satisfy_with, scan, string, run_scanner, take, take_remainder, take_till, take_while, take_while1, token};

pub use parsers::Error;

Modules

ascii	Utilities and parsers for dealing with ASCII data in `u8` format.
buffer	Utilities for parsing streams of data.
combinators	Basic combinators.
parsers	Basic parsers.
primitives	Module used to construct fundamental parsers and combinators.

Macros

parse!	Macro emulating `do`-notation for the parser monad, automatically threading the linear type.
parser!	Macro wrapping an invocation to `parse!` in a closure, useful for creating parsers inline.

Structs

Input	Linear type containing the parser state, this type is threaded though `bind` and is also the initial type passed to a parser.
ParseResult	The basic return type of a parser.

SimpleResult	Result returned from the basic parsers.
U8Result	Result for dealing with the basic parsers when parsing a stream of `u8`.

Crate chomp [−] [src]

Example

Usage

Reexports

Modules

Macros

Structs

Enums

Functions

Type Definitions