Crate syn_pub_items

Syn is a parsing library for parsing a stream of Rust tokens into a syntax tree of Rust source code.

Currently this library is geared toward use in Rust procedural macros, but contains some APIs that may be useful more generally.

• Data structures — Syn provides a complete syntax tree that can represent any valid Rust source code. The syntax tree is rooted at syn::File which represents a full source file, but there are other entry points that may be useful to procedural macros including syn::Item, syn::Expr and syn::Type.

• Custom derives — Of particular interest to custom derives is syn::DeriveInput which is any of the three legal input items to a derive macro. An example below shows using this type in a library that can derive implementations of a trait of your own.

• Parsing — Parsing in Syn is built around parser functions with the signature fn(ParseStream) -> Result<T>. Every syntax tree node defined by Syn is individually parsable and may be used as a building block for custom syntaxes, or you may dream up your own brand new syntax without involving any of our syntax tree types.

• Location information — Every token parsed by Syn is associated with a Span that tracks line and column information back to the source of that token. These spans allow a procedural macro to display detailed error messages pointing to all the right places in the user’s code. There is an example of this below.

• Feature flags — Functionality is aggressively feature gated so your procedural macros enable only what they need, and do not pay in compile time for all the rest.

Version requirement: Syn supports any compiler version back to Rust’s very first support for procedural macros in Rust 1.15.0. Some features especially around error reporting are only available in newer compilers or on the nightly channel.

Example of a custom derive

The canonical custom derive using Syn looks like this. We write an ordinary Rust function tagged with a proc_macro_derive attribute and the name of the trait we are deriving. Any time that derive appears in the user’s code, the Rust compiler passes their data structure as tokens into our macro. We get to execute arbitrary Rust code to figure out what to do with those tokens, then hand some tokens back to the compiler to compile into the user’s crate.

[dependencies]
syn = "0.15"
quote = "0.6"

[lib]
proc-macro = true

extern crate proc_macro;

use proc_macro::TokenStream;
use quote::quote;
use syn::{parse_macro_input, DeriveInput};

#[proc_macro_derive(MyMacro)]
pub fn my_macro(input: TokenStream) -> TokenStream {
    // Parse the input tokens into a syntax tree
    let input = parse_macro_input!(input as DeriveInput);

    // Build the output, possibly using quasi-quotation
    let expanded = quote! {
        // ...
    };

    // Hand the output tokens back to the compiler
    TokenStream::from(expanded)
}

The heapsize example directory shows a complete working Macros 1.1 implementation of a custom derive. It works on any Rust compiler 1.15+. The example derives a HeapSize trait which computes an estimate of the amount of heap memory owned by a value.

pub trait HeapSize {
    /// Total number of bytes of heap memory owned by `self`.
    fn heap_size_of_children(&self) -> usize;
}

The custom derive allows users to write #[derive(HeapSize)] on data structures in their program.

#[derive(HeapSize)]
struct Demo<'a, T: ?Sized> {
    a: Box<T>,
    b: u8,
    c: &'a str,
    d: String,
}

Spans and error reporting

The token-based procedural macro API provides great control over where the compiler’s error messages are displayed in user code. Consider the error the user sees if one of their field types does not implement HeapSize.

#[derive(HeapSize)]
struct Broken {
    ok: String,
    bad: std::thread::Thread,
}

By tracking span information all the way through the expansion of a procedural macro as shown in the heapsize example, token-based macros in Syn are able to trigger errors that directly pinpoint the source of the problem.

error[E0277]: the trait bound `std::thread::Thread: HeapSize` is not satisfied
 --> src/main.rs:7:5
  |
7 |     bad: std::thread::Thread,
  |     ^^^^^^^^^^^^^^^^^^^^^^^^ the trait `HeapSize` is not implemented for `Thread`

Parsing a custom syntax

The lazy-static example directory shows the implementation of a functionlike!(...) procedural macro in which the input tokens are parsed using Syn’s parsing API.

The example reimplements the popular lazy_static crate from crates.io as a procedural macro.

lazy_static! {
    static ref USERNAME: Regex = Regex::new("^[a-z0-9_-]{3,16}$").unwrap();
}

The implementation shows how to trigger custom warnings and error messages on the macro input.

warning: come on, pick a more creative name
  --> src/main.rs:10:16
   |
10 |     static ref FOO: String = "lazy_static".to_owned();
   |                ^^^

Debugging

When developing a procedural macro it can be helpful to look at what the generated code looks like. Use cargo rustc -- -Zunstable-options --pretty=expanded or the cargo expand subcommand.

To show the expanded code for some crate that uses your procedural macro, run cargo expand from that crate. To show the expanded code for one of your own test cases, run cargo expand --test the_test_case where the last argument is the name of the test file without the .rs extension.

This write-up by Brandon W Maister discusses debugging in more detail: Debugging Rust’s new Custom Derive system.

Optional features

Syn puts a lot of functionality behind optional features in order to optimize compile time for the most common use cases. The following features are available.

• derive (enabled by default) — Data structures for representing the possible input to a custom derive, including structs and enums and types.
• full — Data structures for representing the syntax tree of all valid Rust source code, including items and expressions.
• parsing (enabled by default) — Ability to parse input tokens into a syntax tree node of a chosen type.
• printing (enabled by default) — Ability to print a syntax tree node as tokens of Rust source code.
• visit — Trait for traversing a syntax tree.
• visit-mut — Trait for traversing and mutating in place a syntax tree.
• fold — Trait for transforming an owned syntax tree.
• clone-impls (enabled by default) — Clone impls for all syntax tree types.
• extra-traits — Debug, Eq, PartialEq, Hash impls for all syntax tree types.
• proc-macro (enabled by default) — Runtime dependency on the dynamic library libproc_macro from rustc toolchain.

Re-exports

pub use attr::AttrStyle;

pub use attr::Attribute;

pub use attr::AttributeArgs;

pub use attr::Meta;

pub use attr::MetaList;

pub use attr::MetaNameValue;

pub use attr::NestedMeta;

pub use data::Field;

pub use data::Fields;

pub use data::FieldsNamed;

pub use data::FieldsUnnamed;

pub use data::Variant;

pub use data::VisCrate;

pub use data::VisPublic;

pub use data::VisRestricted;

pub use data::Visibility;

pub use expr::Expr;

pub use expr::ExprArray;

pub use expr::ExprAssign;

pub use expr::ExprAssignOp;

pub use expr::ExprAsync;

pub use expr::ExprBinary;

pub use expr::ExprBlock;

pub use expr::ExprBox;

pub use expr::ExprBreak;

pub use expr::ExprCall;

pub use expr::ExprCast;

pub use expr::ExprClosure;

pub use expr::ExprContinue;

pub use expr::ExprField;

pub use expr::ExprForLoop;

pub use expr::ExprGroup;

pub use expr::ExprIf;

pub use expr::ExprInPlace;

pub use expr::ExprIndex;

pub use expr::ExprLet;

pub use expr::ExprLit;

pub use expr::ExprLoop;

pub use expr::ExprMacro;

pub use expr::ExprMatch;

pub use expr::ExprMethodCall;

pub use expr::ExprParen;

pub use expr::ExprPath;

pub use expr::ExprRange;

pub use expr::ExprReference;

pub use expr::ExprRepeat;

pub use expr::ExprReturn;

pub use expr::ExprStruct;

pub use expr::ExprTry;

pub use expr::ExprTryBlock;

pub use expr::ExprTuple;

pub use expr::ExprType;

pub use expr::ExprUnary;

pub use expr::ExprUnsafe;

pub use expr::ExprVerbatim;

pub use expr::ExprWhile;

pub use expr::ExprYield;

pub use expr::Index;

pub use expr::Member;

pub use expr::Arm;

pub use expr::Block;

pub use expr::FieldPat;

pub use expr::FieldValue;

pub use expr::GenericMethodArgument;

pub use expr::Label;

pub use expr::Local;

pub use expr::MethodTurbofish;

pub use expr::Pat;

pub use expr::PatBox;

pub use expr::PatIdent;

pub use expr::PatLit;

pub use expr::PatMacro;

pub use expr::PatPath;

pub use expr::PatRange;

pub use expr::PatRef;

pub use expr::PatSlice;

pub use expr::PatStruct;

pub use expr::PatTuple;

pub use expr::PatTupleStruct;

pub use expr::PatVerbatim;

pub use expr::PatWild;

pub use expr::RangeLimits;

pub use expr::Stmt;

pub use generics::BoundLifetimes;

pub use generics::ConstParam;

pub use generics::GenericParam;

pub use generics::Generics;

pub use generics::LifetimeDef;

pub use generics::PredicateEq;

pub use generics::PredicateLifetime;

pub use generics::PredicateType;

pub use generics::TraitBound;

pub use generics::TraitBoundModifier;

pub use generics::TypeParam;

pub use generics::TypeParamBound;

pub use generics::WhereClause;

pub use generics::WherePredicate;

pub use generics::ImplGenerics;

pub use generics::Turbofish;

pub use generics::TypeGenerics;

pub use item::ArgCaptured;

pub use item::ArgSelf;

pub use item::ArgSelfRef;

pub use item::FnArg;

pub use item::FnDecl;

pub use item::ForeignItem;

pub use item::ForeignItemFn;

pub use item::ForeignItemMacro;

pub use item::ForeignItemStatic;

pub use item::ForeignItemType;

pub use item::ForeignItemVerbatim;

pub use item::ImplItem;

pub use item::ImplItemConst;

pub use item::ImplItemExistential;

pub use item::ImplItemMacro;

pub use item::ImplItemMethod;

pub use item::ImplItemType;

pub use item::ImplItemVerbatim;

pub use item::Item;

pub use item::ItemConst;

pub use item::ItemEnum;

pub use item::ItemExistential;

pub use item::ItemExternCrate;

pub use item::ItemFn;

pub use item::ItemForeignMod;

pub use item::ItemImpl;

pub use item::ItemMacro;

pub use item::ItemMacro2;

pub use item::ItemMod;

pub use item::ItemStatic;

pub use item::ItemStruct;

pub use item::ItemTrait;

pub use item::ItemTraitAlias;

pub use item::ItemType;

pub use item::ItemUnion;

pub use item::ItemUse;

pub use item::ItemVerbatim;

pub use item::MethodSig;

pub use item::TraitItem;

pub use item::TraitItemConst;

pub use item::TraitItemMacro;

pub use item::TraitItemMethod;

pub use item::TraitItemType;

pub use item::TraitItemVerbatim;

pub use item::UseGlob;

pub use item::UseGroup;

pub use item::UseName;

pub use item::UsePath;

pub use item::UseRename;

pub use item::UseTree;

pub use file::File;

pub use lifetime::Lifetime;

pub use lit::FloatSuffix;

pub use lit::IntSuffix;

pub use lit::Lit;

pub use lit::LitBool;

pub use lit::LitByte;

pub use lit::LitByteStr;

pub use lit::LitChar;

pub use lit::LitFloat;

pub use lit::LitInt;

pub use lit::LitStr;

pub use lit::LitVerbatim;

pub use lit::StrStyle;

pub use mac::Macro;

pub use mac::MacroDelimiter;

pub use derive::Data;

pub use derive::DataEnum;

pub use derive::DataStruct;

pub use derive::DataUnion;

pub use derive::DeriveInput;

pub use op::BinOp;

pub use op::UnOp;

pub use ty::Abi;

pub use ty::BareFnArg;

pub use ty::BareFnArgName;

pub use ty::ReturnType;

pub use ty::Type;

pub use ty::TypeArray;

pub use ty::TypeBareFn;

pub use ty::TypeGroup;

pub use ty::TypeImplTrait;

pub use ty::TypeInfer;

pub use ty::TypeMacro;

pub use ty::TypeNever;

pub use ty::TypeParen;

pub use ty::TypePath;

pub use ty::TypePtr;

pub use ty::TypeReference;

pub use ty::TypeSlice;

pub use ty::TypeTraitObject;

pub use ty::TypeTuple;

pub use ty::TypeVerbatim;

pub use path::AngleBracketedGenericArguments;

pub use path::Binding;

pub use path::Constraint;

pub use path::GenericArgument;

pub use path::ParenthesizedGenericArguments;

pub use path::Path;

pub use path::PathArguments;

pub use path::PathSegment;

pub use path::QSelf;

pub use error::Error;

pub use error::Result;

pub use macros::*;

pub use gen::*;

Modules

attr

buffer

A stably addressed token buffer supporting efficient traversal based on a cheaply copyable cursor.

data

derive

error

expr

ext

Extension traits to provide parsing methods on foreign types.

file

gen

generics

ident

item

keyword

lifetime

lit

lookahead

mac

macros

op

parse

Parsing interface for parsing a token stream into a syntax tree node.

path

print

punctuated

A punctuated sequence of syntax tree nodes separated by punctuation.

span

spanned

A trait that can provide the Span of the complete contents of a syntax tree node.

thread

token

Tokens representing Rust punctuation, keywords, and delimiters.

tt

ty

Macros

Token

A type-macro that expands to the name of the Rust type representation of a given token.

ast_enum

ast_enum_of_structs

ast_struct

braced

Parse a set of curly braces and expose their content to subsequent parsers.

bracketed

Parse a set of square brackets and expose their content to subsequent parsers.

custom_keyword

Define a type that supports parsing and printing a given identifier as if it were a keyword.

generate_to_tokens

maybe_ast_struct

parenthesized

Parse a set of parentheses and expose their content to subsequent parsers.

parse_macro_input

Parse the input TokenStream of a macro, triggering a compile error if the tokens fail to parse.

parse_quote

Quasi-quotation macro that accepts input like the quote! macro but uses type inference to figure out a return type for those tokens.

to_tokens_call

Structs

Ident

A word of Rust code, which may be a keyword or legal variable name.

private

Functions

parse

Parse tokens of source code into the chosen syntax tree node.

parse2

Parse a proc-macro2 token stream into the chosen syntax tree node.

parse_file

Parse the content of a file of Rust code.

parse_str

Parse a string of Rust code into the chosen syntax tree node.