Skip to main content

macro_lang/
lib.rs

1//! # macro_lang
2//!
3//! Hygienic macro expansion for language implementations: pattern matching over
4//! token trees, template substitution, and the hygiene bookkeeping that keeps a
5//! macro's names from colliding with its caller's.
6//!
7//! This is the `macro_rules!` / Scheme `syntax-rules` model, generic over a
8//! language's token kind. A [`Macro`] is a list of [`Rule`]s; each rule pairs a
9//! [`Pattern`] that an invocation must match with a [`Template`] it expands to.
10//! An [`Expander`] runs the expansion: it matches the invocation's [`Tree`]s
11//! against each rule in turn, writes out the first matching template, and marks
12//! every token the template introduces with a fresh hygiene [`Context`].
13//!
14//! The crate owns no syntax. A language's front end lexes source into
15//! `token_lang` tokens, groups them into [`Tree`]s, lowers its own macro
16//! definition syntax into patterns and templates, and drives expansion — finding
17//! invocations, resolving macro names, and splicing results back in.
18//!
19//! ## Hygiene in one paragraph
20//!
21//! Tokens written in source carry [`Context::ROOT`]. Each expansion mints a new
22//! context for the literal tokens its template writes, while the trees it
23//! substitutes from the invocation keep the context they arrived with. A name
24//! resolver that treats two identifiers as the same binding only when both
25//! spelling *and* context match then gets hygiene for free: a temporary the macro
26//! introduces cannot capture a same-named variable passed in by the caller, and
27//! cannot be captured by one. [`Expander::origin`] reports which expansion minted
28//! a context, for definition-site fallback and for diagnostics.
29//!
30//! ## Example
31//!
32//! A macro whose template introduces a temporary named `t`, invoked on input
33//! that already contains a `t`. After expansion the two `t`s have equal
34//! spelling but different contexts, so they can never be confused.
35//!
36//! ```
37//! use macro_lang::{Context, Expander, Fragment, Macro, Pattern, Rule, Template, Tree};
38//! use intern_lang::Interner;
39//! use token_lang::Span;
40//!
41//! let mut names = Interner::new();
42//! let x = names.intern("x");
43//!
44//! // with_tmp!($x:tt) => t = $x
45//! let with_tmp = Macro::new(names.intern("with_tmp"), vec![Rule {
46//!     pattern: vec![Pattern::Bind { name: x, fragment: Fragment::Tree }],
47//!     template: vec![
48//!         Template::token('t', Span::new(100, 101)),
49//!         Template::token('=', Span::new(102, 103)),
50//!         Template::Var(x),
51//!     ],
52//! }])?;
53//!
54//! // with_tmp!(t) — the caller passes its own `t`.
55//! let mut expander = Expander::new();
56//! let input = [Tree::token('t', Span::new(9, 10))];
57//! let out = expander.expand(&with_tmp, &input, Span::new(0, 11), Context::ROOT)?;
58//!
59//! let contexts: Vec<Context> = out
60//!     .iter()
61//!     .filter_map(|tree| match tree {
62//!         Tree::Token { token, ctx } if token.kind == 't' => Some(*ctx),
63//!         _ => None,
64//!     })
65//!     .collect();
66//!
67//! // The macro's `t` was minted a fresh context; the caller's `t` kept the root.
68//! assert_eq!(contexts.len(), 2);
69//! assert!(!contexts[0].is_root());
70//! assert!(contexts[1].is_root());
71//! # Ok::<(), Box<dyn std::error::Error>>(())
72//! ```
73//!
74//! ## Matching
75//!
76//! Patterns are compiled into a nondeterministic automaton and simulated over the
77//! input in a single left-to-right pass, merging parses that reach the same
78//! point. Matching never backtracks, so its cost grows linearly with the input
79//! (and polynomially with the pattern) even for adversarial input; and a rule
80//! that matches in more than one way is reported as [`ExpandError::Ambiguous`]
81//! instead of silently picking one parse.
82//!
83//! ## Bounded expansion
84//!
85//! Macros are untrusted input, and a two-line macro can describe unbounded
86//! work. Every expander enforces a [`Budget`] — a recursion limit, a cap on the
87//! number of expansions, and a cap on the number of tokens written — with
88//! defaults applied automatically, so a macro that recurses forever, doubles
89//! its argument every round, or fans out into exponentially many calls stops
90//! with [`ExpandError::RecursionLimit`] or [`ExpandError::Budget`] in bounded
91//! time and memory. A driver that knows how deep each invocation sits passes
92//! that depth to [`Expander::expand_at`], which makes the recursion limit hold
93//! even for a macro that rebuilds its own invocation from captured tokens.
94//!
95//! ## Features
96//!
97//! - `std` (default) — the standard library. Without it the crate is
98//!   `#![no_std]` and needs only `alloc`; expansion uses no operating-system
99//!   facilities either way.
100//!
101//! ## Stability
102//!
103//! The public surface is frozen and stable as of `1.0.0`: it follows Semantic
104//! Versioning, with no breaking changes before `2.0`. The full surface, the
105//! matching and hygiene semantics that are part of the contract, and the SemVer
106//! promise are catalogued in
107//! [`docs/API.md`](https://github.com/jamesgober/macro-lang/blob/main/docs/API.md#stability).
108
109#![cfg_attr(not(feature = "std"), no_std)]
110#![cfg_attr(docsrs, feature(doc_cfg))]
111#![forbid(unsafe_code)]
112#![deny(
113    warnings,
114    missing_docs,
115    unsafe_op_in_unsafe_fn,
116    unused_must_use,
117    unused_results,
118    clippy::unwrap_used,
119    clippy::expect_used,
120    clippy::panic,
121    clippy::todo,
122    clippy::unimplemented,
123    clippy::unreachable,
124    clippy::dbg_macro,
125    clippy::print_stdout,
126    clippy::print_stderr,
127    clippy::undocumented_unsafe_blocks
128)]
129
130extern crate alloc;
131
132mod budget;
133mod compile;
134mod context;
135mod definition;
136mod error;
137mod expander;
138mod matcher;
139mod pattern;
140mod template;
141mod transcribe;
142mod tree;
143
144pub use budget::{Budget, Limit, Usage};
145pub use context::{Context, Origin};
146pub use definition::{Macro, Rule};
147pub use error::{ExpandError, MacroError};
148pub use expander::Expander;
149pub use pattern::{Fragment, Kleene, Pattern};
150pub use template::Template;
151pub use tree::Tree;
152
153/// Compiles and runs the `rust` code blocks in `README.md` and `docs/API.md` as
154/// part of `cargo test`, so the published examples cannot drift from the API.
155///
156/// Present only while collecting doctests (`#[cfg(doctest)]`); it is not part of
157/// the public surface and does not appear in the built library or its docs.
158#[cfg(doctest)]
159#[doc = include_str!("../README.md")]
160#[doc = include_str!("../docs/API.md")]
161pub struct MarkdownDocTests;