Crate venial

Source
Expand description

§Lightweight parsing for Rust proc macros

Venial is a small parser for Rust proc macros.

When writing proc macros that need to parse Rust code (such as attribute and derive macros), the most common solution is to use the syn crate. Syn can parse arbitrary valid Rust code, and even Rust-based DSLs, and return versatile data structures that can inspected and mutated in powerful ways.

It’s also extremely heavy. In one analysis of lqd’s early 2022 benchmark collection, the author estimates that syn is reponsible for 8% of compile times of the benchmark, which accounts for Rust’s most popular crates. There are subtleties (eg this isn’t necessarily critical path time, but syn is often in the critical path anyway), but the overall takeaway is clear: syn is expensive.

And yet, a lot of the power of syn is often unneeded. If we look at the crates that depend on syn, we can see that the 5 most downloaded are:

  • serde_derive
  • proc-macro-hack
  • pin-project-internal
  • anyhow
  • thiserror-impl

Of these, proc-macro-hack is deprecated, and the other four only need to parse basic information on a type.

Other popular reverse-dependencies of syn (such as futures-macro, tokios-macros, async-trait, etc) do use syn’s more advanced features, but there’s still room for a lightweight parser in proc-macros.

Venial is that parser.

§Example

use venial::{parse_item, Item};
use quote::quote;

let enum_type = parse_item(quote!(
    enum Shape {
        Square(Square),
        Circle(Circle),
        Triangle(Triangle),
    }
));

let enum_type = match enum_type {
    Ok(Item::Enum(enum_type)) => enum_type,
    _ => unreachable!(),
};

assert_eq!(enum_type.variants[0].0.name, "Square");
assert_eq!(enum_type.variants[1].0.name, "Circle");
assert_eq!(enum_type.variants[2].0.name, "Triangle");

§Spans

Spans mark the beginning and end of tokens in the source code and can be used to generate precise error messages, highlighting certain parts. Item types provide a span() method that returns their respective span.

Note that Rust’s proc_macro crate does not support span joining in stable. That means that if you want items to hold the full full span of the input, you’ll need a nightly compiler. Otherwise, the span will typically only include the first token-tree in the sequence.

For example, if you have a NamedField you’d like to highlight in an error message, you can use named_field.span(). On nightly, a custom error could be rendered as follows:

error: my_attribute cannot be applied in this context
  --> crate/rust/src/file.rs:58:5
   |
58 | /     #[my_attribute]
59 | |     field: i32,
   | |______________^

However, on stable, the span might only refer to the first token tree (here #):

error: my_attribute cannot be applied in this context
  --> crate/rust/src/file.rs:58:5
   |
58 |      #[my_attribute]
   |      ^

You can however control this behavior more precisely by choosing which sub-tokens to highlight. On stable compilers, you might want to put focus on the field name named_field.name.span() instead of the entire field named_field.span(). This would yield:

error: my_attribute cannot be applied in this context
 --> crate/rust/src/file.rs:58:5
   |
59 |     field: i32,
   |     ^^^^^

Re-exports§

pub use crate::Punctuated;

Structs§

Attribute
An outer or inner attribute.
Constant
Constant or static declaration.
Enum
Declaration of an enum.
EnumVariant
The individual variant of an Enum.
EnumVariantValue
The value of an EnumVariant, normally for c-like enums.
Error
Convenient error type for displaying errors in your macros to users.
ExternBlock
An extern block.
ExternCrate
Extern crate declaration.
FnQualifiers
Keywords giving special information on a function.
FnReceiverParam
Function parameter which refers to self in some way.
FnTypedParam
Function parameter with explicit type.
Function
Declaration of a function.
GenericArgList
List of generic arguments, as in Vec<i32, Alloc>.
GenericBound
A parameter bound in a type’s generic list.
GenericParam
A parameter in a type’s generic list.
GenericParamList
The generic parameters declared right after your type’s name.
GroupSpan
Span information about a Group.
Impl
Declaration of an impl block.
InlineGenericArgs
Generic arguments deduced from a type’s GenericParamList.
Lifetime
Lifetime declaration.
Macro
A macro invocation or macro_rules! declaration.
Module
Declaration of a module.
NamedField
A field of a Struct or struct-like EnumVariant.
NamedFields
Fields in a nominal Struct.
Path
Implements a path expression according to the Rust documentation.
PathSegment
A segment of a Path, e.g. Type::<i32>
Punctuated
Comma-separated list of items.
Struct
Declaration of a struct.
Trait
Declaration of a trait.
TupleField
A field of a tuple Struct or tuple-like EnumVariant.
TupleFields
Fields in a tuple-style Struct.
TypeAlias
Type definition. Handles both associated types (in impl blocks) or type aliases (globally).
TypeExpr
Type expression in a TupleField or NamedField.
Union
Declaration of a union.
UseDeclaration
A use declaration for a path.
ValueExpr
Value expression, e.g. as the initializer of a Constant.
VisMarker
Visibility marker, eg pub, pub(crate), pub(super), etc.
WhereClause
All the stuff that comes after the where keyword.
WhereClausePredicate
Item in a where clause.

Enums§

AttributeValue
The value of an Attribute.
Fields
Fields of a Struct or an EnumVariant.
FnParam
A parameter of a Function.
GenericArg
A parameter in a type’s generic list.
ImplMember
The group representing an impl block.
Item
The declaration of a Rust item.
TraitMember
Associated items in a trait.

Functions§

consume_item
Consume an item declaration from a token stream.
parse_item
Parses the token stream of an item declaration.