Module capture 

Input capture and source mapping.

The whole point of cinrs is that a diagnostic produced anywhere in the pipeline — by our own lexer/parser/sema, or by rustc on the code we generate — must point at the exact C token that caused it. To make that possible the front end never works on a TokenStream directly: it first recovers a plain-text C translation unit plus a SourceMap that can turn any byte offset in that text back into a proc_macro2::Span.

Capture strategies

There are two shapes of input, and the raw-token shape has two strategies:

• String-literal mode (InputMode::StringLiteral). The entire macro input is a single Rust string literal ("…", r"…", r#"…"#). We unescape it and use the text as-is. This accepts any C code, including the lexemes the Rust 2024 lexer rejects (hex float literals, 'ab', L"…", \ line continuations, ##). The catch is that stable Rust has no way to build a span pointing inside a string literal (Literal::subspan is unstable), so every diagnostic is reported at the literal as a whole and the line/column inside the C text is appended to the message instead. Such a file is flagged as not SourceFile::precise — unless the caller hands capture a Subspan hook, which is what the nightly feature of cinrs-macros does; see Subspan.

• Raw-token mode, primary path (InputMode::FileSlice). We flatten the token trees, ask the first token for Span::local_file, read that .rs file from disk and slice out first.start() .. last.end(). This gives back the exact original text, including whitespace, newlines and comments — which the preprocessor needs, since it is line-oriented and since #error reproduces what was written. The slice is validated against every token’s Span::source_text before it is trusted.

• File mode (InputMode::CFile), which is what include_c99!("…") captures with capture_c_file: the text is a .c file read from disk, and nothing in the .rs file corresponds to a position in it, so every range resolves to the span of the macro invocation and the file names its own position in the message — exactly as an #included header does.

• Raw-token mode, search path (also InputMode::FileSlice). A host may hand a procedural macro tokens with no positions at all: rust-analyzer reports no file, no source text and line 1 column 0 for every token alike. The text is still on disk, and which text it is can be proved — the locate module searches the crate’s .rs files, the directory CARGO_MANIFEST_DIR names, for an invocation whose token sequence is exactly the one we were handed. A match gives the same three things the primary path gives (path, text, anchors), so everything downstream — diagnostics, __FILE__, a quoted #include, include_str! tracking, the unit id — is unchanged. See Origin for what capture has to be told for this to be possible, and note that it can only run when the primary path found no position whatsoever: in a normal build it costs nothing because it never happens.

• Raw-token mode, fallback path (InputMode::Reconstructed). When there is no local file (macro-generated input, some IDE contexts such as rust-analyzer, or unit tests that build a TokenStream with TokenStream::from_str), or when the slice fails validation, we rebuild the text from the tokens. Where their positions are usable, every token goes at its original line/column, padded with newlines and spaces: comments are lost, but the line structure — and therefore all reported positions — survives. Where they are not (every token at the same position, which is what rust-analyzer gives an unsaved buffer), the text is rebuilt from the tokens alone: one space between two tokens, none where the host says they were written together, so that ->, <<= and ++ survive and - - stays two tokens. A directive is a line, and tokens keep no lines; the forms whose end the tokens themselves give away (#include <…>, #ifdef X, #endif, …) are written on a line of their own and anything else — #define, #if — is one clear diagnostic rather than a guess.

Coordinates

SourceMap owns any number of SourceFiles in a single, global byte offset space: file i occupies base .. base + text.len(). A Pos is therefore enough to identify both a file and an offset inside it, which is what will let #include drop extra files into the same map without changing a single signature in the lexer, parser or AST.

A file also remembers which .rs file it came from and which line of it the text’s own line 1 sits on, which is what the preprocessor’s __FILE__ and __LINE__ are made of.

Structs

FileId

Identifies one file inside a SourceMap.

Origin

What the host can tell the front end about an invocation, over and above the tokens themselves.

Source

The captured macro input: a SourceMap plus the id of the root file.

SourceFile

A single file of C source together with its Rust-token anchors.

SourceMap

Maps byte offsets back to proc_macro2 spans.

SourceRange

A half-open range [start, end) in a SourceMap’s global offset space.

Subspan

Turns a byte range of the macro input’s source spelling into a span pointing exactly at those bytes.

Enums

InputMode

How the C source text of a file was obtained.

Functions

capture

Recovers the C source text of a macro invocation.

capture_c_file

Makes a .c file read from disk the source of a translation unit.

capture_with

Recovers the C source text of a macro invocation, with everything the host can say about the invocation itself.

invocation_directory

The directory of the .rs file an invocation whose whole input is input is written in, found by searching the crate’s sources.

string_literal_value

The text a Literal denotes, when it is a plain or raw string literal.

Type Aliases

Pos

A position in the global byte-offset space managed by a SourceMap.