Expand description
§Lightweight parsing for Rust proc macros
Venial is a small parser for Rust proc macros.
When writing proc macros that need to parse Rust code (such as attribute and derive macros), the most common solution is to use the syn crate. Syn can parse arbitrary valid Rust code, and even Rust-based DSLs, and return versatile data structures that can inspected and mutated in powerful ways.
It’s also extremely heavy. In one analysis of lqd’s early 2022 benchmark collection, the author estimates that syn is reponsible for 8% of compile times of the benchmark, which accounts for Rust’s most popular crates. There are subtleties (eg this isn’t necessarily critical path time, but syn is often in the critical path anyway), but the overall takeaway is clear: syn is expensive.
And yet, a lot of the power of syn is often unneeded. If we look at the crates that depend on syn, we can see that the 5 most downloaded are:
- serde_derive
- proc-macro-hack
- pin-project-internal
- anyhow
- thiserror-impl
Of these, proc-macro-hack is deprecated, and the other four only need to parse basic information on a type.
Other popular reverse-dependencies of syn (such as futures-macro, tokios-macros, async-trait, etc) do use syn’s more advanced features, but there’s still room for a lightweight parser in proc-macros.
Venial is that parser.
§Example
use venial::{parse_item, Item};
use quote::quote;
let enum_type = parse_item(quote!(
enum Shape {
Square(Square),
Circle(Circle),
Triangle(Triangle),
}
));
let enum_type = match enum_type {
Ok(Item::Enum(enum_type)) => enum_type,
_ => unreachable!(),
};
assert_eq!(enum_type.variants[0].0.name, "Square");
assert_eq!(enum_type.variants[1].0.name, "Circle");
assert_eq!(enum_type.variants[2].0.name, "Triangle");
§Spans
Spans mark the beginning and end of tokens in the source code and can be used to generate precise error messages, highlighting certain parts.
Item types provide a span()
method that returns their respective span.
Note that Rust’s proc_macro
crate does not support span joining in stable. That means that if you want items to hold the full
full span of the input, you’ll need a nightly compiler. Otherwise, the span will typically only include the first token-tree in the sequence.
For example, if you have a NamedField
you’d like to highlight in an error message, you can use named_field.span()
.
On nightly, a custom error could be rendered as follows:
error: my_attribute cannot be applied in this context
--> crate/rust/src/file.rs:58:5
|
58 | / #[my_attribute]
59 | | field: i32,
| |______________^
However, on stable, the span might only refer to the first token tree (here #
):
error: my_attribute cannot be applied in this context
--> crate/rust/src/file.rs:58:5
|
58 | #[my_attribute]
| ^
You can however control this behavior more precisely by choosing which sub-tokens to highlight. On stable compilers, you might want to
put focus on the field name named_field.name.span()
instead of the entire field named_field.span()
. This would yield:
error: my_attribute cannot be applied in this context
--> crate/rust/src/file.rs:58:5
|
59 | field: i32,
| ^^^^^
Re-exports§
pub use crate::Punctuated;
Structs§
- Attribute
- An outer or inner attribute.
- Constant
- Constant or static declaration.
- Enum
- Declaration of an
enum
. - Enum
Variant - The individual variant of an
Enum
. - Enum
Variant Value - The value of an
EnumVariant
, normally for c-like enums. - Error
- Convenient error type for displaying errors in your macros to users.
- Extern
Block - An
extern
block. - Extern
Crate - Extern crate declaration.
- FnQualifiers
- Keywords giving special information on a function.
- FnReceiver
Param Function
parameter which refers toself
in some way.- FnTyped
Param Function
parameter with explicit type.- Function
- Declaration of a function.
- Generic
ArgList - List of generic arguments, as in
Vec<i32, Alloc>
. - Generic
Bound - A parameter bound in a type’s generic list.
- Generic
Param - A parameter in a type’s generic list.
- Generic
Param List - The generic parameters declared right after your type’s name.
- Group
Span - Span information about a
Group
. - Impl
- Declaration of an
impl
block. - Inline
Generic Args - Generic arguments deduced from a type’s GenericParamList.
- Lifetime
- Lifetime declaration.
- Macro
- A macro invocation or
macro_rules!
declaration. - Module
- Declaration of a module.
- Named
Field - A field of a
Struct
or struct-likeEnumVariant
. - Named
Fields - Fields in a nominal
Struct
. - Path
- Implements a path expression according to the Rust documentation.
- Path
Segment - A segment of a
Path
, e.g.Type::<i32>
- Punctuated
- Comma-separated list of items.
- Struct
- Declaration of a struct.
- Trait
- Declaration of a trait.
- Tuple
Field - A field of a tuple
Struct
or tuple-likeEnumVariant
. - Tuple
Fields - Fields in a tuple-style
Struct
. - Type
Alias - Type definition.
Handles both associated types (in
impl
blocks) or type aliases (globally). - Type
Expr - Type expression in a
TupleField
orNamedField
. - Union
- Declaration of a union.
- UseDeclaration
- A
use
declaration for a path. - Value
Expr - Value expression, e.g. as the initializer of a
Constant
. - VisMarker
- Visibility marker, eg
pub
,pub(crate)
,pub(super)
, etc. - Where
Clause - All the stuff that comes after the
where
keyword. - Where
Clause Predicate - Item in a where clause.
Enums§
- Attribute
Value - The value of an
Attribute
. - Fields
- Fields of a
Struct
or anEnumVariant
. - FnParam
- A parameter of a
Function
. - Generic
Arg - A parameter in a type’s generic list.
- Impl
Member - The group representing an
impl
block. - Item
- The declaration of a Rust item.
- Trait
Member - Associated items in a trait.
Functions§
- consume_
item - Consume an item declaration from a token stream.
- parse_
item - Parses the token stream of an item declaration.