Crate syn [−] [src]
Syn is a parsing library for parsing a stream of Rust tokens into a syntax tree of Rust source code.
Currently this library is geared toward the custom derive use case but contains some APIs that may be useful for Rust procedural macros more generally.
-
Data structures — Syn provides a complete syntax tree that can represent any valid Rust source code. The syntax tree is rooted at
syn::File
which represents a full source file, but there are other entry points that may be useful to procedural macros includingsyn::Item
,syn::Expr
andsyn::Type
. -
Custom derives — Of particular interest to custom derives is
syn::DeriveInput
which is any of the three legal input items to a derive macro. An example below shows using this type in a library that can derive implementations of a trait of your own. -
Parser combinators — Parsing in Syn is built on a suite of public parser combinator macros that you can use for parsing any token-based syntax you dream up within a
functionlike!(...)
procedural macro. Every syntax tree node defined by Syn is individually parsable and may be used as a building block for custom syntaxes, or you may do it all yourself working from the most primitive tokens. -
Location information — Every token parsed by Syn is associated with a
Span
that tracks line and column information back to the source of that token. These spans allow a procedural macro to display detailed error messages pointing to all the right places in the user's code. There is an example of this below. -
Feature flags — Functionality is aggressively feature gated so your procedural macros enable only what they need, and do not pay in compile time for all the rest.
Version requirement: Syn supports any compiler version back to Rust's very first support for procedural macros in Rust 1.15.0. Some features especially around error reporting are only available in newer compilers or on the nightly channel.
Example of a custom derive
The canonical custom derive using Syn looks like this. We write an ordinary
Rust function tagged with a proc_macro_derive
attribute and the name of
the trait we are deriving. Any time that derive appears in the user's code,
the Rust compiler passes their data structure as tokens into our macro. We
get to execute arbitrary Rust code to figure out what to do with those
tokens, then hand some tokens back to the compiler to compile into the
user's crate.
[dependencies]
syn = "0.13"
quote = "0.5"
[lib]
proc-macro = true
extern crate proc_macro; extern crate syn; #[macro_use] extern crate quote; use proc_macro::TokenStream; use syn::DeriveInput; #[proc_macro_derive(MyMacro)] pub fn my_macro(input: TokenStream) -> TokenStream { // Parse the input tokens into a syntax tree let input: DeriveInput = syn::parse(input).unwrap(); // Build the output, possibly using quasi-quotation let expanded = quote! { // ... }; // Hand the output tokens back to the compiler expanded.into() }
The heapsize
example directory shows a complete working Macros 1.1
implementation of a custom derive. It works on any Rust compiler >=1.15.0.
The example derives a HeapSize
trait which computes an estimate of the
amount of heap memory owned by a value.
pub trait HeapSize { /// Total number of bytes of heap memory owned by `self`. fn heap_size_of_children(&self) -> usize; }
The custom derive allows users to write #[derive(HeapSize)]
on data
structures in their program.
#[derive(HeapSize)] struct Demo<'a, T: ?Sized> { a: Box<T>, b: u8, c: &'a str, d: String, }
Spans and error reporting
The heapsize2
example directory is an extension of the heapsize
example that demonstrates some of the hygiene and error reporting properties
of Macros 2.0. This example currently requires a nightly Rust compiler
>=1.24.0-nightly but we are working to stabilize all of the APIs involved.
The token-based procedural macro API provides great control over where the
compiler's error messages are displayed in user code. Consider the error the
user sees if one of their field types does not implement HeapSize
.
#[derive(HeapSize)] struct Broken { ok: String, bad: std::thread::Thread, }
In the Macros 1.1 string-based procedural macro world, the resulting error would point unhelpfully to the invocation of the derive macro and not to the actual problematic field.
error[E0599]: no method named `heap_size_of_children` found for type `std::thread::Thread` in the current scope
--> src/main.rs:4:10
|
4 | #[derive(HeapSize)]
| ^^^^^^^^
By tracking span information all the way through the expansion of a
procedural macro as shown in the heapsize2
example, token-based macros in
Syn are able to trigger errors that directly pinpoint the source of the
problem.
error[E0277]: the trait bound `std::thread::Thread: HeapSize` is not satisfied
--> src/main.rs:7:5
|
7 | bad: std::thread::Thread,
| ^^^^^^^^^^^^^^^^^^^^^^^^ the trait `HeapSize` is not implemented for `Thread`
Parsing a custom syntax using combinators
The lazy-static
example directory shows the implementation of a
functionlike!(...)
procedural macro in which the input tokens are parsed
using nom
-style parser combinators.
The example reimplements the popular lazy_static
crate from crates.io as a
procedural macro.
lazy_static! { static ref USERNAME: Regex = Regex::new("^[a-z0-9_-]{3,16}$").unwrap(); }
The implementation shows how to trigger custom warnings and error messages on the macro input.
warning: come on, pick a more creative name
--> src/main.rs:10:16
|
10 | static ref FOO: String = "lazy_static".to_owned();
| ^^^
Debugging
When developing a procedural macro it can be helpful to look at what the
generated code looks like. Use cargo rustc -- -Zunstable-options --pretty=expanded
or the cargo expand
subcommand.
To show the expanded code for some crate that uses your procedural macro,
run cargo expand
from that crate. To show the expanded code for one of
your own test cases, run cargo expand --test the_test_case
where the last
argument is the name of the test file without the .rs
extension.
This write-up by Brandon W Maister discusses debugging in more detail: Debugging Rust's new Custom Derive system.
Optional features
Syn puts a lot of functionality behind optional features in order to optimize compile time for the most common use cases. The following features are available.
derive
(enabled by default) — Data structures for representing the possible input to a custom derive, including structs and enums and types.full
— Data structures for representing the syntax tree of all valid Rust source code, including items and expressions.parsing
(enabled by default) — Ability to parse input tokens into a syntax tree node of a chosen type.printing
(enabled by default) — Ability to print a syntax tree node as tokens of Rust source code.visit
— Trait for traversing a syntax tree.visit-mut
— Trait for traversing and mutating in place a syntax tree.fold
— Trait for transforming an owned syntax tree.clone-impls
(enabled by default) — Clone impls for all syntax tree types.extra-traits
— Debug, Eq, PartialEq, Hash impls for all syntax tree types.proc-macro
(enabled by default) — Runtime dependency on the dynamic library libproc_macro from rustc toolchain.
Modules
buffer |
A stably addressed token buffer supporting efficient traversal based on a cheaply copyable cursor. |
fold |
Syntax tree traversal to transform the nodes of an owned syntax tree. |
punctuated |
A punctuated sequence of syntax tree nodes separated by punctuation. |
spanned |
A trait that can provide the |
synom |
Parsing interface for parsing a token stream into a syntax tree node. |
token |
Tokens representing Rust punctuation, keywords, and delimiters. |
visit |
Syntax tree traversal to walk a shared borrow of a syntax tree. |
visit_mut |
Syntax tree traversal to mutate an exclusive borrow of a syntax tree in place. |
Macros
Token |
A type-macro that expands to the name of the Rust type representation of a given token. |
alt |
Run a series of parsers, returning the result of the first one which succeeds. |
braces |
Parse inside of |
brackets |
Parse inside of |
call |
Invoke the given parser function with zero or more arguments. |
cond |
Execute a parser only if a condition is met, otherwise return None. |
cond_reduce |
Execute a parser only if a condition is met, otherwise fail to parse. |
custom_keyword |
Parse the given word as a keyword. |
do_parse |
Run a series of parsers, optionally naming each intermediate result, followed by a step to combine the intermediate results. |
epsilon |
Parses nothing and always succeeds. |
input_end |
Parse nothing and succeed only if the end of the enclosing block has been reached. |
keyword |
Parse a single Rust keyword token. |
many0 |
Parse zero or more values using the given parser. |
map |
Transform the result of a parser by applying a function or closure. |
named |
Define a parser function with the signature expected by syn parser combinators. |
not |
Invert the result of a parser by parsing successfully if the given parser fails to parse and vice versa. |
option |
Turn a failed parse into |
parens |
Parse inside of |
parse_quote |
Quasi-quotation macro that accepts input like the |
punct |
Parse a single Rust punctuation token. |
reject |
Unconditionally fail to parse anything. |
switch |
Pattern-match the result of a parser to select which other parser to run. |
syn |
Parse any type that implements the |
tuple |
Run a series of parsers and produce all of the results in a tuple. |
value |
Produce the given value without parsing anything. |
Structs
Abi |
The binary interface of a function: |
AngleBracketedGenericArguments |
Angle bracketed arguments of a path segment: the |
ArgCaptured |
An explicitly typed pattern captured by a function signature. |
ArgSelf |
Self captured by value in a function signature: |
ArgSelfRef |
Self captured by reference in a function signature: |
Arm |
One arm of a |
Attribute |
An attribute like |
BareFnArg |
An argument in a function type: the |
Binding |
A binding (equality constraint) on an associated type: |
Block |
A braced block containing Rust statements. |
BoundLifetimes |
A set of bound lifetimes: |
ConstParam |
A const generic parameter: |
DataEnum |
An enum input to a |
DataStruct |
A struct input to a |
DataUnion |
A tagged union input to a |
DeriveInput |
Data structure sent to a |
ExprArray |
A slice literal expression: |
ExprAssign |
An assignment expression: |
ExprAssignOp |
A compound assignment expression: |
ExprBinary |
A binary operation: |
ExprBlock |
A blocked scope: |
ExprBox |
A box expression: |
ExprBreak |
A |
ExprCall |
A function call expression: |
ExprCast |
A cast expression: |
ExprCatch |
A catch expression: |
ExprClosure |
A closure expression: |
ExprContinue |
A |
ExprField |
Access of a named struct field ( |
ExprForLoop |
A for loop: |
ExprGroup |
An expression contained within invisible delimiters. |
ExprIf |
An |
ExprIfLet |
An |
ExprInPlace |
A placement expression: |
ExprIndex |
A square bracketed indexing expression: |
ExprLit |
A literal in place of an expression: |
ExprLoop |
Conditionless loop: |
ExprMacro |
A macro invocation expression: |
ExprMatch |
A |
ExprMethodCall |
A method call expression: |
ExprParen |
A parenthesized expression: |
ExprPath |
A path like |
ExprRange |
A range expression: |
ExprReference |
A referencing operation: |
ExprRepeat |
An array literal constructed from one repeated element: |
ExprReturn |
A |
ExprStruct |
A struct literal expression: |
ExprTry |
A try-expression: |
ExprTuple |
A tuple expression: |
ExprType |
A type ascription expression: |
ExprUnary |
A unary operation: |
ExprUnsafe |
An unsafe block: |
ExprVerbatim |
Tokens in expression position not interpreted by Syn. |
ExprWhile |
A while loop: |
ExprWhileLet |
A while-let loop: |
ExprYield |
A yield expression: |
Field |
A field of a struct or enum variant. |
FieldPat |
A single field in a struct pattern. |
FieldValue |
A field-value pair in a struct literal. |
FieldsNamed |
Named fields of a struct or struct variant such as |
FieldsUnnamed |
Unnamed fields of a tuple struct or tuple variant such as |
File |
A complete file of Rust source code. |
FnDecl |
Header of a function declaration, without including the body. |
ForeignItemFn |
A foreign function in an |
ForeignItemStatic |
A foreign static item in an |
ForeignItemType |
A foreign type in an |
ForeignItemVerbatim |
Tokens in an |
Generics |
Lifetimes and type parameters attached to a declaration of a function, enum, trait, etc. |
Ident |
A word of Rust code, which may be a keyword or legal variable name. |
ImplGenerics |
Returned by |
ImplItemConst |
An associated constant within an impl block. |
ImplItemMacro |
A macro invocation within an impl block. |
ImplItemMethod |
A method within an impl block. |
ImplItemType |
An associated type within an impl block. |
ImplItemVerbatim |
Tokens within an impl block not interpreted by Syn. |
Index |
The index of an unnamed tuple struct field. |
ItemConst |
A constant item: |
ItemEnum |
An enum definition: |
ItemExternCrate |
An |
ItemFn |
A free-standing function: |
ItemForeignMod |
A block of foreign items: |
ItemImpl |
An impl block providing trait or associated items: |
ItemMacro |
A macro invocation, which includes |
ItemMacro2 |
A 2.0-style declarative macro introduced by the |
ItemMod |
A module or module declaration: |
ItemStatic |
A static item: |
ItemStruct |
A struct definition: |
ItemTrait |
A trait definition: |
ItemType |
A type alias: |
ItemUnion |
A union definition: |
ItemUse |
A use declaration: |
ItemVerbatim |
Tokens forming an item not interpreted by Syn. |
Label |
A lifetime labeling a |
Lifetime |
A Rust lifetime: |
LifetimeDef |
A lifetime definition: |
LitBool |
A boolean literal: |
LitByte |
A byte literal: |
LitByteStr |
A byte string literal: |
LitChar |
A character literal: |
LitFloat |
A floating point literal: |
LitInt |
An integer literal: |
LitStr |
A UTF-8 string literal: |
LitVerbatim |
A raw token literal not interpreted by Syn, possibly because it represents an integer larger than 64 bits. |
Local |
A local |
Macro |
A macro invocation: |
MetaList |
A structured list within an attribute, like |
MetaNameValue |
A name-value pair within an attribute, like |
MethodSig |
A method's signature in a trait or implementation: |
MethodTurbofish |
The |
ParenthesizedGenericArguments |
Arguments of a function path segment: the |
PatBox |
A box pattern: |
PatIdent |
A pattern that binds a new variable: |
PatLit |
A literal pattern: |
PatMacro |
A macro in expression position. |
PatPath |
A path pattern like |
PatRange |
A range pattern: |
PatRef |
A reference pattern: |
PatSlice |
A dynamically sized slice pattern: |
PatStruct |
A struct or struct variant pattern: |
PatTuple |
A tuple pattern: |
PatTupleStruct |
A tuple struct or tuple variant pattern: |
PatVerbatim |
Tokens in pattern position not interpreted by Syn. |
PatWild |
A pattern that matches any value: |
Path |
A path at which a named item is exported: |
PathSegment |
A segment of a path together with any path arguments on that segment. |
PathTokens |
A helper for printing a self-type qualified path as tokens. |
PredicateEq |
An equality predicate in a |
PredicateLifetime |
A lifetime predicate in a |
PredicateType |
A type predicate in a |
QSelf |
The explicit Self type in a qualified path: the |
TraitBound |
A trait used as a bound on a type parameter. |
TraitItemConst |
An associated constant within the definition of a trait. |
TraitItemMacro |
A macro invocation within the definition of a trait. |
TraitItemMethod |
A trait method within the definition of a trait. |
TraitItemType |
An associated type within the definition of a trait. |
TraitItemVerbatim |
Tokens within the definition of a trait not interpreted by Syn. |
Turbofish |
Returned by |
TypeArray |
A fixed size array type: |
TypeBareFn |
A bare function type: |
TypeGenerics |
Returned by |
TypeGroup |
A type contained within invisible delimiters. |
TypeImplTrait |
An |
TypeInfer |
Indication that a type should be inferred by the compiler: |
TypeMacro |
A macro in the type position. |
TypeNever |
The never type: |
TypeParam |
A generic type parameter: |
TypeParen |
A parenthesized type equivalent to the inner type. |
TypePath |
A path like |
TypePtr |
A raw pointer type: |
TypeReference |
A reference type: |
TypeSlice |
A dynamically sized slice type: |
TypeTraitObject |
A trait object type |
TypeTuple |
A tuple type: |
TypeVerbatim |
Tokens in type position not interpreted by Syn. |
UseGlob |
A glob import in a |
UseGroup |
A braced group of imports in a |
UseName |
An identifier imported by a |
UsePath |
A path prefix of imports in a |
UseRename |
An renamed identifier imported by a |
Variant |
An enum variant. |
VisCrate |
A crate-level visibility: |
VisPublic |
A public visibility level: |
VisRestricted |
A visibility level restricted to some path: |
WhereClause |
A |
Enums
AttrStyle |
Distinguishes between attributes that decorate an item and attributes that are contained within an item. |
BareFnArgName |
Name of an argument in a function type: the |
BinOp |
A binary operator: |
Data |
The storage of a struct, enum or union data structure. |
Expr |
A Rust expression. |
Fields |
Data stored within an enum variant or struct. |
FloatSuffix |
The suffix on a floating point literal if any, like the |
FnArg |
An argument in a function signature: the |
ForeignItem |
An item within an |
GenericArgument |
An individual generic argument, like |
GenericMethodArgument |
An individual generic argument to a method, like |
GenericParam |
A generic type parameter, lifetime, or const generic: |
ImplItem |
An item within an impl block. |
IntSuffix |
The suffix on an integer literal if any, like the |
Item |
Things that can appear directly inside of a module or scope. |
Lit |
A Rust literal such as a string or integer or boolean. |
MacroDelimiter |
A grouping token that surrounds a macro body: |
Member |
A struct or tuple struct field accessed in a struct literal or field expression. |
Meta |
Content of a compile-time structured attribute. |
NestedMeta |
Element of a compile-time attribute list. |
Pat |
A pattern in a local binding, function signature, match expression, or various other places. |
PathArguments |
Angle bracketed or parenthesized arguments of a path segment. |
RangeLimits |
Limit types of a range, inclusive or exclusive. |
ReturnType |
Return type of a function signature. |
Stmt |
A statement, usually ending in a semicolon. |
StrStyle |
The style of a string literal, either plain quoted or a raw string like
|
TraitBoundModifier |
A modifier on a trait bound, currently only used for the |
TraitItem |
An item declaration within the definition of a trait. |
Type |
The possible types that a Rust value could have. |
TypeParamBound |
A trait or lifetime used as a bound on a type parameter. |
UnOp |
A unary operator: |
UseTree |
A suffix of an import tree in a |
Visibility |
The visibility level of an item: inherited or |
WherePredicate |
A single predicate in a |
Functions
parse |
Parse tokens of source code into the chosen syntax tree node. |
parse2 |
Parse a proc-macro2 token stream into the chosen syntax tree node. |
parse_file |
Parse the content of a file of Rust code. |
parse_str |
Parse a string of Rust code into the chosen syntax tree node. |