Crate loess

Expand description

README / Examples (click to expand)

§Loess

Loess is a parser library and parser generator for proc macros.

For a simple but representative example of using Loess, see the inline-json5 crate.

Premade Rust grammar implementations can be found in loess-rust, with additional temporary wrappers available in loess-rust-opaque.

Here’s what to expect:

Fast builds. Loess’s core is compact and language agnostic. Direct grammar implementations like loess-rust should compile fairly quickly too.
A simple, flexible API. Loess is relatively unopinionated about how or what you parse, and you can construct (and destructure) Input at any time.
Really good error reporting from proc macros implemented with Loess, by default.

Many parsing errors are easily or automatically recoverable, which means multiple errors can be reported at once while preserving as much regular output as possible (which means no or much fewer cascading errors!).

Panics inside your parser can also be caught and reported as located errors with the original panic message.
A reasonably powerful parser-generator.

grammar! can emit documentation (for enums) and PeekFrom, PopFrom and IntoTokens implementations on grammar types in general.
Powerful quote_into macros that expand efficiently and are language-agnostic.

You can cleanly loop and/or branch within the template.
Low-allocation workflow.

Loess can (usually) move tokens from input to output without cloning them. (You can still clone all included grammar types explicitly, including when pasting in quotes.)

Here’s what not to expect:

A general Syn-replacement (at least not soon).

Loess is mainly aimed at implementing domain-specific languages that may cite fragments of Rust verbatim in their output. There is currently no focus on parsing Rust code or transforming it in-depth.

§Using `$crate` for full caller independence

loess::IntoTokens-methods take an (optionally empty) root: &TokenStream parameter, which all emitted fully qualified paths should be prefixed with.

In combination with a wrapper crate: This achieves full isolation regarding caller dependencies:

(click to expand code blocks)

// wrapper crate

#[macro_export]
macro_rules! my_macro {
    ($($tt:tt)*) => ( $crate::__::my_macro!([$crate] $($tt)*) );
}

#[doc(hidden)]
pub mod __ {
    pub use core; // Expected by `Errors`.
    pub use my_macro_impl::my_macro;
}

// my_macro_impl (proc macro)

use loess::{
    grammar, parse_once, parse_all,
    Errors, Input, IntoTokens,
    rust_grammar::{SquareBrackets},
};
use proc_macro2::{Span, TokenStream, TokenTree};

// […]

fn macro_impl(input: TokenStream) -> TokenStream {
    let mut input = Input {
        tokens: input.into_iter().collect(),
        end: Span::call_site(),
    };
    let mut errors = Errors::new();

    // `root` is implicitly a `TokenStream`.
    let Ok(SquareBrackets { contents: root, .. }) = parse_once(
            &mut input,
            &mut errors,
        ) else { return errors.collect_tokens(&TokenStream::new()) };

    grammar! {
        /// This represents your complete input grammar.
        /// This here is a placeholder, so it's empty.
        struct Grammar: PopFrom {}
    }

    // Checks for exhaustiveness.
    let parsed = parse_all(&mut input, &mut errors).next();
    let mut output = errors.collect_tokens(&root);

    if let Some(Grammar {}) = parsed {
        // Emit your output here.
    }

    output
}

Loess is a parser library and parser generator for proc macros.

For a simple but representative example of using Loess, see the inline-json5 crate.

In most cases you’ll want to:

generate custom grammar implementations with grammar! (You can also easily implement parts manually.),
create (mutable) instances of Input and Errors,
step through the input with parse_once, parse_once_with and/or parse_once_with_infallible,
consume the last of the input with parse_all, parse_all_with or parse_all_with_infallible,
perform any fallible transforms you need, possibly pushing more Errors into your Errors,
if errors is your Errors, have let mut output: proc_macro2::TokenStream = errors.collect_tokens(); convert it into the start of your output,
emit your regular output with quote_into_mixed_site! (recommended), quote_into_with_exact_span! or quote_into_call_site!, which accept interpolation and control flow directives.

You can call either Iterator::collect (for repeats) or Iterator::next (for one value) on step 3. Either way, the parsing iterator will check for unconsumed tokens remaining in the Input when dropped and report to the Errors accordingly.

You can combine step 2 into step 3 with a grammar!-generated top-level grammar, but for proc macros embedded in a runtime library, in most cases I recommend getting $crate from a wrapper macro_rules!-macro first. (See full example above.)

Some parsing errors are recoverable, but still translate to compile_error! calls being generated in step 5. Your macro should seamlessly continue to operate in such cases, which helps prevent noise from cascading errors due to e.g. missing items, making it much easier for your macro’s users to find problems with the input.

You can download a .code-snippets file for Loess’s macros and quote macro directives here: https://github.com/Tamschi/Asteracea/blob/develop/.vscode/Loess.code-snippets

§Features

None are default, as DSL macros might not need Rust’s grammar at all.

§`"rust_grammar"`

Enables rust_grammar.

§`"opaque_rust_grammar"` _{enables "rust_grammar", depends on syn and quote}

Adds additional opaque Rust grammar tokens, to consume, paste and clone for example Statements and Patterns.

These preliminary implementations are Syn-based and can’t be inspected.

Modules§

error_priorities: ConstErrorPriority types for use with Exhaustive and EndOfInput.
rust_grammarDeprecated: With "rust_grammar": Tokens representing the stable Rust programming language, closely following The Rust Reference.

Macros§

grammar: Parser- and serialiser-generator macro.
quote_into_call_site: Like quote_into_mixed_site!, but resolved according to Span::call_site().
quote_into_mixed_site: Simple generic quotation (statement) macro that works well with Loess’s types.
quote_into_with_exact_span: Like quote_into_mixed_site!, but using $span directly for quoted tokens.
raw_quote_into_call_site: Like raw_quote_into_mixed_site!, but resolved according to Span::call_site().
raw_quote_into_mixed_site: Simple generic quotation (statement) macro that efficiently emits tokens verbatim.
raw_quote_into_with_exact_span: Like raw_quote_into_mixed_site!, but using $span directly for quoted tokens.

Structs§

Eager: Wraps a collection type to eagerly parse values that are PeekFrom, but to stop when PopFrom::peek_pop_from returns None.
EndOfInput: Fails to parse and emits an Error with the given ConstErrorPriority for any unconsumed tokens in Input.
Error: A Span-located proc macro error with ErrorPriority.
Usually submitted through Errors::push.
ErrorPriority: An opaque Error priority.
Errors: A collection of Errors submitted during e.g. parsing with PopFrom.
Exhaustive: Doesn’t fail to parse but emits an Error with the given ConstErrorPriority for any unconsumed tokens in Input after T.
HandledPanic: A substitute panic that isn’t reported as Error. (Read for panic handling info!)
Input: Input tokens with end-Span.
For use with PeekFrom and PopFrom.

Traits§

ConstErrorPriority: ErrorPriority as generic type argument.
IntoTokens: Spreads self into its contained or representative TokenTrees.
PeekFrom: Determines if Self may be be parseable from an Input.
This is often a cursory check!
PopFrom: Consumes from Input to create Result<Self, ()> and emit to Errors.
SimpleSpanned: Has a single Span.

Functions§

parse_all: Conveniently parses remaining Input through PopFrom, catching and submitting panics to the given Errors.
parse_all_with: Parses remaining Input through FnMut, catching and submitting panics to the given Errors.
parse_all_with_infallible: Low-level function that parses remaining Input through FnMut without also catching Err(()), catching and submitting panics to the given Errors.
parse_once: Convenient non-repeating PopFrom::pop_from-unwind-catcher that reports panics to the given Errors.
parse_once_with: FnOnce-unwind-catcher that reports panics to the given Errors.
parse_once_with_infallible: Low-level FnOnce-unwind-catcher that reports panics to the given Errors without also catching Err(()).

Crate loessCopy item path