Struct Preprocessed 

pub struct Preprocessed {
    pub tokens: Vec<Token>,
    pub text: String,
    pub map: SourceMap,
    pub diagnostics: Vec<Diagnostic>,
}

The expanded token stream and where it came from.

Fields

tokens: Vec<Token>

The tokens the parser will see.

text: String

The text those tokens index into.

Concatenated from every file that contributed, so a token’s span still slices something. The map says which file a range belongs to.

map: SourceMap

Where each token came from.

diagnostics: Vec<Diagnostic>

Everything wrong with the directives.

Implementations

impl Preprocessed

pub fn provenance(&self, expanded: Span) -> Option<Provenance>

Where a range of the expanded text was written, for a node of the model rather than a diagnostic: the file’s path, the range in its text (SourceMap::original), and the macro expansions that produced the text at its start, outermost first, each with the path and range of its invocation. None before the first token.

pub fn locate(&self, diagnostic: &mut Diagnostic)

Moves diagnostic out of the expanded text into the file it points at.

The parser and everything after it see only the expanded text, in which comments are gone, directives are gone, includes are inlined and every token stands one space from the next — so an offset into it is an offset into no file a reader has. Read against the unit, the first lines of comment were enough to put file.idl:3:7 where file.idl:6:10 was meant. This maps primary and every label through SourceMap::original, and sets Diagnostic::file when the place is an included file. A label in a file other than the primary’s names that file in its note.

For diagnostics raised on the expanded text only: the preprocessor’s own are in file offsets already.

Trait Implementations

impl Clone for Preprocessed

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Preprocessed

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.