Expand description
README / Examples (click to expand)
§Loess
Loess is a parser library and parser generator for proc macros.
For a simple but representative example of using Loess, see the inline-json5 crate.
Premade Rust grammar implementations can be found in loess-rust, with additional temporary wrappers available in loess-rust-opaque.
Here’s what to expect:
-
Fast builds. Loess’s core is compact and language agnostic. Direct grammar implementations like loess-rust should compile fairly quickly too.
-
A simple, flexible API. Loess is relatively unopinionated about how or what you parse, and you can construct (and destructure)
Input
at any time. -
Really good error reporting from proc macros implemented with Loess, by default.
Many parsing errors are easily or automatically recoverable, which means multiple errors can be reported at once while preserving as much regular output as possible (which means no or much fewer cascading errors!).
Panics inside your parser can also be caught and reported as located errors with the original panic message.
-
A reasonably powerful parser-generator.
grammar!
can emit documentation (for enums) andPeekFrom
,PopFrom
andIntoTokens
implementations on grammar types in general. -
Powerful
quote_into
macros that expand efficiently and are language-agnostic.You can cleanly loop and/or branch within the template.
-
Low-allocation workflow.
Loess can (usually) move tokens from input to output without cloning them. (You can still clone all included grammar types explicitly, including when pasting in quotes.)
Here’s what not to expect:
-
A general Syn-replacement (at least not soon).
Loess is mainly aimed at implementing domain-specific languages that may cite fragments of Rust verbatim in their output. There is currently no focus on parsing Rust code or transforming it in-depth.
§Using $crate
for full caller independence
loess::IntoTokens
-methods take an (optionally empty) root: &TokenStream
parameter,
which all emitted fully qualified paths should be prefixed with.
In combination with a wrapper crate: This achieves full isolation regarding caller dependencies:
(click to expand code blocks)
// wrapper crate
#[macro_export]
macro_rules! my_macro {
($($tt:tt)*) => ( $crate::__::my_macro!([$crate] $($tt)*) );
}
#[doc(hidden)]
pub mod __ {
pub use core; // Expected by `Errors`.
pub use my_macro_impl::my_macro;
}
// my_macro_impl (proc macro)
use loess::{
grammar, parse_once, parse_all,
Errors, Input, IntoTokens,
rust_grammar::{SquareBrackets},
};
use proc_macro2::{Span, TokenStream, TokenTree};
// […]
fn macro_impl(input: TokenStream) -> TokenStream {
let mut input = Input {
tokens: input.into_iter().collect(),
end: Span::call_site(),
};
let mut errors = Errors::new();
// `root` is implicitly a `TokenStream`.
let Ok(SquareBrackets { contents: root, .. }) = parse_once(
&mut input,
&mut errors,
) else { return errors.collect_tokens(&TokenStream::new()) };
grammar! {
/// This represents your complete input grammar.
/// This here is a placeholder, so it's empty.
struct Grammar: PopFrom {}
}
// Checks for exhaustiveness.
let parsed = parse_all(&mut input, &mut errors).next();
let mut output = errors.collect_tokens(&root);
if let Some(Grammar {}) = parsed {
// Emit your output here.
}
output
}
Loess is a parser library and parser generator for proc macros.
For a simple but representative example of using Loess, see the inline-json5 crate.
In most cases you’ll want to:
- generate custom grammar implementations with
grammar!
(You can also easily implement parts manually.), - create (mutable) instances of
Input
andErrors
, - step through the input with
parse_once
,parse_once_with
and/orparse_once_with_infallible
, - consume the last of the input with
parse_all
,parse_all_with
orparse_all_with_infallible
, - perform any fallible transforms you need, possibly pushing more
Error
s into yourErrors
, - if
errors
is yourErrors
, havelet mut output: proc_macro2::TokenStream = errors.collect_tokens();
convert it into the start of your output, - emit your regular output with
quote_into_mixed_site!
(recommended),quote_into_with_exact_span!
orquote_into_call_site!
, which accept interpolation and control flow directives.
You can call either Iterator::collect
(for repeats) or Iterator::next
(for one value) on step 3.
Either way, the parsing iterator will check for unconsumed tokens remaining in the Input
when dropped
and report to the Errors
accordingly.
You can combine step 2 into step 3 with a grammar!
-generated top-level grammar, but for proc macros embedded
in a runtime library, in most cases I recommend getting $crate
from a wrapper macro_rules!
-macro first.
(See full example above.)
Some parsing errors are recoverable, but still translate to compile_error!
calls being generated in step 5.
Your macro should seamlessly continue to operate in such cases, which helps prevent noise from cascading errors
due to e.g. missing items, making it much easier for your macro’s users to find problems with the input.
You can download a .code-snippets file for Loess’s macros and quote macro directives here: https://github.com/Tamschi/Asteracea/blob/develop/.vscode/Loess.code-snippets
§Features
None are default, as DSL macros might not need Rust’s grammar at all.
§"rust_grammar"
Enables rust_grammar
.
§"opaque_rust_grammar"
enables "rust_grammar"
, depends on syn
and quote
Adds additional opaque Rust grammar tokens, to consume, paste and clone for example Statements and Patterns.
These preliminary implementations are Syn-based and can’t be inspected.
Modules§
- error_
priorities ConstErrorPriority
types for use withExhaustive
andEndOfInput
.- rust_
grammar Deprecated - With
"rust_grammar"
: Tokens representing the stable Rust programming language, closely following The Rust Reference.
Macros§
- grammar
- Parser- and serialiser-generator macro.
- quote_
into_ call_ site - Like
quote_into_mixed_site!
, but resolved according toSpan::call_site()
. - quote_
into_ mixed_ site - Simple generic quotation (statement) macro that works well with Loess’s types.
- quote_
into_ with_ exact_ span - Like
quote_into_mixed_site!
, but using$span
directly for quoted tokens. - raw_
quote_ into_ call_ site - Like
raw_quote_into_mixed_site!
, but resolved according toSpan::call_site()
. - raw_
quote_ into_ mixed_ site - Simple generic quotation (statement) macro that efficiently emits tokens verbatim.
- raw_
quote_ into_ with_ exact_ span - Like
raw_quote_into_mixed_site!
, but using$span
directly for quoted tokens.
Structs§
- Eager
- Wraps a collection type to eagerly parse values that are
PeekFrom
, but to stop whenPopFrom::peek_pop_from
returnsNone
. - EndOf
Input - Fails to parse and emits an
Error
with the givenConstErrorPriority
for any unconsumed tokens inInput
. - Error
- A
Span
-located proc macro error withErrorPriority
.
Usually submitted throughErrors::push
. - Error
Priority - An opaque
Error
priority. - Errors
- A collection of
Error
s submitted during e.g. parsing withPopFrom
. - Exhaustive
- Doesn’t fail to parse but emits an
Error
with the givenConstErrorPriority
for any unconsumed tokens inInput
afterT
. - Handled
Panic - A substitute panic that isn’t reported as
Error
. (Read for panic handling info!) - Input
- Input
tokens
withend
-Span
.
For use withPeekFrom
andPopFrom
.
Traits§
- Const
Error Priority ErrorPriority
as generic type argument.- Into
Tokens - Spreads
self
into its contained or representativeTokenTree
s. - Peek
From - Determines if
Self
may be be parseable from anInput
.
This is often a cursory check! - PopFrom
- Consumes from
Input
to create
and emit toResult
<Self, ()>Errors
. - Simple
Spanned - Has a single
Span
.
Functions§
- parse_
all - Conveniently parses remaining
Input
throughPopFrom
, catching and submitting panics to the givenErrors
. - parse_
all_ with - Parses remaining
Input
throughFnMut
, catching and submitting panics to the givenErrors
. - parse_
all_ with_ infallible - Low-level function that parses remaining
Input
throughFnMut
without also catchingErr(())
, catching and submitting panics to the givenErrors
. - parse_
once - Convenient non-repeating
PopFrom::pop_from
-unwind-catcher that reports panics to the givenErrors
. - parse_
once_ with FnOnce
-unwind-catcher that reports panics to the givenErrors
.- parse_
once_ with_ infallible - Low-level
FnOnce
-unwind-catcher that reports panics to the givenErrors
without also catchingErr(())
.