Module autumn::combinators

Traits for combining and modifying parsers

Combinators provide ways of modifying the behavior of parsers and combining parsers that is consistent across all parser implementations.

Parsing sequences

Parsing sequences of characters or values from input is done using parsers that produce a Concat and the ConcatParserExt trait that is implemented for all such parsers.

The ConcatParserExt provides three combinators for manipulating parsers over sequences of values:

• maybe repeats a parser zero or one times,
• multiple repeats a parser one or more times, and
• and uses two parsers in sequence, concatenating their results into a single list.

/// Parses C-like identifiers
fn identifier(source: &str, location: Span) -> ParseResult<String> {
    alphabetic
        .or("_")
        .and(alphanumeric.or("_").multiple().maybe())
        .copy_string()
        .parse(source, location)
}

When using both multiple and maybe to achieve zero or more repetitions, multiple().maybe() must be used; maybe().multiple() can find an infinite number of ways to apply any parser on even an empty string.

and and multiple when used with parsers producing a Span must ensure that all spans are adjacent (don't use drop or skip to join a pair of Span results that are not continuous).

Multiple potential parsers

The or will take two parsers and attempt to evaluate both of them returning any successful results produced by either of them. This can include duplicate results if a given parse would be acceptable for either parser.

Changing the result of a parser

There are to ways in which the result of a parser can be changed:

• the output value and type can be simply mapped to a new value with a new type using the map operation, and
• changing a successful (or already failing) parse into a potentially failing parse with the and_then operation.

#[derive(Clone)]
struct InvalidIdentifier(String);

/// Parses C-like identifiers
fn identifier(
    source: &str,
    location: Span,
) -> ParseResult<Option<String>, InvalidIdentifier> {
    alphabetic
        .or("_")
        .and(alphanumeric.or("_").multiple())
        .copy_string()
        .map(Some)
        .on_none(
            any_character
                .str_condition(|s| !s.chars().any(char::is_whitespace))
                .multiple()
                .copy_string()
                .and_then(|identifier| throw(None, InvalidIdentifier(identifier)))
        )
        .catch()
        .parse(source, location)
}

Discarding the result of a parser

When using two parsers sequentially you may wish to discard the result of either sequence rather than concatenate the results.

The skip method can be used to combine two parsers, discarding the result of the first and keeping the result of the second.

The drop method can be used to combine two parsers, keeping the result of the first and discarding the result of the second.

// Produces "right"
fn right() -> impl Parser<String> {
    "left".skip("right").copy_string()
}
let skipped = parse(right(), "leftright");

// Produces "left"
fn left() -> impl Parser<String> {
    "left".drop("right").copy_string()
}
let dropped = parse(left(), "leftright");

Adding conditions to a parser

Two methods are provided to require the result of a parser to meet a certain condition:

• condition allows a certain condition to be specified as a function or closure, and
• matching allows a value to be provided which the result must match.

Ensuring a parser consumes all input

The end can be used on any parser to ensure that after any successful parse there is no text left un-parsed in the source. If not used, invoking a parser directly can produce all prefixes that are themselves a valid parse of the source string.

Finding the source location for the result of a parser

When parsing a structured text input a parser may wish to know the location of a segment that was parsed to produce a result. The meta method will map a parser such that its result is wrapped in a Meta, binding it with a Span specifying the location in the source of the parsed text.

/// Parses integers
fn integer(source: &str, location: Span) -> ParseResult<String> {
    digit.multiple().copy_string().parse(source, location)
}

/// Parse a list of integers and get the source location of each integer
fn integer_list(source: &str, location: Span) -> ParseResult<Vec<Meta<String, Span>>> {
    integer
        .meta()
        .delimited_by(",".maybe_space_after(), ..)
        .surrounded_by("[".maybe_space_after(), "]")
        .collect()
        .parse(source, location)
}

Matching parser types with dynamic dispatch

There are some cases where a parser may need to produce one of many values within the argument to the and_then method. In this case it helps to be able to unify the types of the values with dynamic dispatch.

This can be achieved simply with the boxed method.

The value, error, and throw functions all produce boxed parsers with types that will match as they are commonly used in this case.

fn alphabet(source: &str, location: Span) -> ParseResult<String, &'static str> {
    "abcde"
        .and(digit)
        .copy_string()
        .and_then(|text| {
            let text = text;
            if text.ends_with("0") {
                throw(text, "Token must not end with 0")
            } else {
                value(text)
            }
        })
        .catch()
        .parse(source, location)
}

Structs

And	The result of the and function in the ConcatParserExt trait
AndThen	The result of the and_then function in the ParserExt trait
Boxed	The result of the boxed function in the BoxedParserExt trait
Catch	The result of the catch function in the ParserExt trait
Collect	The result of the collect function in the ListParserExt trait
Condition	The result of the condition function in the ParserExt trait
DelimitedBy	The result of the delimited_by function in the ParserExt trait
Drop	The result of the drop function in the ParserExt trait
End	The result of the end function in the ParserExt trait
Fold	The result of the repeat function in the ParserExt trait
ListMap	The result of the to_list function in the ParserExt trait
Map	The result of the map function in the ParserExt trait
Matching	The result of the matching function in the ParserExt trait
Maybe	The result of the maybe function in the ConcatParserExt trait
MaybeDrop	The result of the maybe_drop function in the ParserExt trait
MaybeSpaceAfter	The result of the maybe_space_after function in the ParserExt trait
MaybeSpaceAround	The result of the maybe_space_around function in the ParserExt trait
MaybeSpaceBefore	The result of the maybe_space_before function in the ParserExt trait
MetaMap	The result of the meta function in the ParserExt trait
Multiple	The result of the multiple function in the ConcatParserExt trait
OnFailure	The result of the on_failure function in the ParserExt trait
OnNone	The result of the on_none function in the ParserExt trait
Or	The result of the or function in the ParserExt trait
Pair	The result of the pair function in the ParserExt trait
Referenced	The result of the ref_parser function in the BoxedParserExt trait
Repeat	The result of the repeated function in the ConcatParserExt trait
Skip	The result of the skip function in the ParserExt trait
SpaceAfter	The result of the space_after function in the ParserExt trait
SpaceAround	The result of the space_around function in the ParserExt trait
SpaceBefore	The result of the space_before function in the ParserExt trait
StrCondition	The result of the str_condition function in the TextParserExt trait
StrictlyDelimitedBy	The result of the strictly_delimited_by function in the ParserExt trait
StringMap	The result of the copy_string function in the TextParserExt trait
SurroundedBy	The result of the surrounded_by function in the ParserExt trait

Traits

BoxedParserExt	Boxing parsers for dynamic dispatch
ConcatParserExt	Combinators on parsers that produce a Concat
ListParserExt	Combinators on parsers that produce a List
ParserExt	Combinators that can be used on all parsers
TextParserExt	Combinators on parser that produce a list of characters