1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
use rowan::NodeCache;
use crate::{
kind::{CppLanguageLevel, Dialect},
lexer::LexerConfig,
symbols::{MacroFacts, SymbolTable},
};
pub struct ParserConfig<'cache> {
pub level: CppLanguageLevel,
/// **Is this translation unit CUDA?** — the one switch the grammar's CUDA rules are behind.
///
/// A separate question from `level` and from `dialect`, and separate from *both* on purpose: `nvcc` compiles
/// CUDA with a host compiler, so a `.cu` file is C++20 *and* MSVC *and* CUDA at once. What this flag adds is the
/// syntax those two do not have — the execution-configuration brackets of a kernel launch
/// (`kernel<<<grid, block>>>(args)`) and the qualifiers that place a function or a variable on the host or the
/// device (`__global__`, `__device__`, `__shared__`, …) — and it is off by default so that a plain C++ file is
/// parsed exactly as it was written.
///
/// It is set by the analysis from the project's toolchain (`__CUDACC__` in the predefined macros, or an `nvcc`
/// command), which is what keeps CUDA's spelling from being accepted in a project that has never seen it: a
/// `__global__` in ordinary C++ is a name, not a qualifier, and a `<<<` is three shift operators.
pub cuda: bool,
/// Which compiler's own reserved spellings mean what — see [`Dialect`].
///
/// A separate question from `level`, and the reason is written on [`Dialect`]: `-std=gnu++20` is C++20 *and*
/// GNU's spellings at once, which a single "level" cannot say. The analysis layer sets this from the
/// toolchain's own predefined macros (`__GNUC__` / `_MSC_VER`).
pub dialect: Dialect,
lexer_config: LexerConfig,
node_cache: Option<&'cache mut NodeCache>,
symbol_table: Option<&'cache dyn SymbolTable>,
/// **The macros the file's includes define**, when the caller knows them.
///
/// The seam this type was always documented to have and never did. Without it the grammar has one way to
/// decide that a name is a macro — its **spelling** ([`crate::grammar`]'s `written_like_a_macro`) — and a
/// spelling cannot separate two shapes that differ in what the name is *for* rather than in how it is written.
/// The measurement that says so is on `the_name_before_is_a_macro` in the expression grammar: a rule keyed on
/// the spelling took MSVC's `<xmemory>` from 582 declarations covering the file to 153 covering a fifth, and
/// left `<chrono>` at a hundredth of itself, because `_STD declval<_Alloc&>()` and `_STD _Convert_size<size_type>(…)`
/// are spelled identically and are not the same construct.
///
/// `None` is "nobody says", and a grammar rule that asks must then fall back to the spelling it used before
/// this field existed — see [`ParserConfig::is_a_macro_at`].
macro_facts: Option<&'cache dyn MacroFacts>,
/// **Where one file's text ends and the next begins**, as token offsets into the text being parsed.
///
/// A scope may not cross a file boundary. A header that opens a `namespace` and never closes it — which is a
/// real idiom, and MSVC's `/analyze` headers do it — would otherwise hold **every file spliced after it**, and
/// the declarations of those files would be filed under a scope that is not theirs. This is the parser's half
/// of a rule the analysis layer used to enforce by quarantining files; that repair was deleted, and
/// [`crate::ParserConfig::with_file_boundaries`] is what was supposed to replace it.
///
/// Empty for a parse of one file's own text, where there is no boundary to respect.
file_boundaries: Vec<usize>,
/// **Consume a statement block's tokens without building anything over them** — the mode the indexer parses in.
///
/// # Why a mode and not a rule
///
/// A function body cannot declare anything another file can name, and all three tools this project compares
/// itself to refuse to build syntax for one while indexing: clangd sets `SkipFunctionBodies = true` for its
/// preamble, IntelliJ's `skipChildProcessingWhenBuildingStubs` exists to *"reduce the number of the AST nodes
/// the platform walks"*, and Visual Studio's own documentation says its browsing parser *"skips the content of
/// blocks … only records the location and parameters of a function, and ignores its contents"*.
///
/// Measured over a 126-file MSVC standard-library closure (`examples/body_weight.rs`):
///
/// ```text
/// internal nodes 754 604 inside a CompoundStat 486 103 64.4%
/// tokens 1 377 789 inside a CompoundStat 853 387 61.9%
/// ```
///
/// **Two thirds of every `descendants()` walk in the analysis layer is inside a block**, and so is two thirds of
/// what the parser builds. The tokens stay — a lossless tree keeps every one of them, so this buys nothing from
/// the lexer and much less than 64% of the memory — but the *structure* over them is not built.
///
/// # What it costs, and therefore who may set it
///
/// A body is where a `return` statement is, and [`crate::sema::declarations::deduced_returns_of`] reads one to
/// answer `auto` — so a summary built in this mode records no deduced return type, and a diagnostic inside a
/// body is not reported by a parse in this mode.
///
/// That is why it is a **mode** rather than the only behaviour: the indexer wants the summaries and nothing else,
/// and the two consumers that want bodies — the view a reader is looking at, and the parse of a **rendering**,
/// which is where the diagnostics of the file being edited come from — must not set it. `Session::view` and
/// `FileIndexer::index_rendering` are those two places, and they are named here because a flag that spreads is a
/// capability that disappears quietly.
///
/// # And it was measured, and it is **not enabled**
///
/// Enabling it for the summary parse of a 126-file closure made the index **no faster**:
///
/// ```text
/// off on
/// wall 830.0 ms 821.0 ms
/// parse 1138.7 ms 951.7 ms −16%
/// sweep 1182.4 ms 1299.8 ms +10%
/// three tests pass FAIL locals, another file's local, a check inside a body
/// ```
///
/// **64% of the nodes go and 16% of `parse` goes with them**, which is the number that matters: building nodes is
/// about a quarter of parsing, so this removes a sixth of one stage. The walks that actually dominate — `sweep`,
/// and `facts` inside it — are per-**declaration** and do not care how deeply the statements around a declaration
/// were nested. The mechanism is kept because the next person to read "64% of the nodes are bodies" will reach
/// for it, and this paragraph is what should stop them.
skip_bodies: bool,
}
impl<'cache> ParserConfig<'cache> {
pub fn new(level: CppLanguageLevel, node_cache: Option<&'cache mut NodeCache>) -> Self {
Self {
cuda: false,
level,
dialect: Dialect::default(),
lexer_config: LexerConfig::new(level),
node_cache,
symbol_table: None,
macro_facts: None,
file_boundaries: Vec::new(),
skip_bodies: false,
}
}
/// **Parse so that a statement block is one node with its tokens flat under it** — see
/// [`ParserConfig::skip_bodies`] for the measurement, and for the two callers that must not ask for this.
pub fn with_skipped_bodies(mut self) -> Self {
self.skip_bodies = true;
self
}
/// **Read this translation unit as CUDA** — see [`ParserConfig::cuda`].
///
/// Off unless a toolchain said `nvcc`: the launch brackets and the placement qualifiers are then ordinary C++
/// (`<<<` is three shifts) rather than syntax this parser invents for a project that never had them.
pub fn with_cuda(mut self, cuda: bool) -> Self {
self.cuda = cuda;
self
}
/// Whether a statement block's structure is being built. See [`ParserConfig::skip_bodies`].
pub fn skips_bodies(&self) -> bool {
self.skip_bodies
}
/// **Tell the parser where one file's text ends and the next begins.**
///
/// A scope may not cross a file boundary: the `{` a header left open is closed at the end of that header,
/// rather than paired with a `}` another file wrote or leaving the next file's declarations inside it. The
/// offsets are the first token of each file **after the first**, which is what
/// `cpp_code_analysis::RenderedUnit::file_boundaries` answers for a stitched unit.
///
/// # Why the grammar, and not the event stream
///
/// Measured before this existed, by closing the open nodes in the mark-event stream at each boundary: the tree
/// shape does not move — `TranslationUnit → NamespaceDecl → CompoundStat → Declaration` either way — because
/// the holder is a `CompoundStat` whose `{` the **grammar** paired with the `}` at the end of the file. The
/// leak happens while the tokens are read, so it has to be refused there.
pub fn with_file_boundaries(mut self, boundaries: Vec<usize>) -> Self {
self.file_boundaries = boundaries;
self
}
/// The boundaries in force, in ascending order — see [`ParserConfig::with_file_boundaries`].
pub fn file_boundaries(&self) -> &[usize] {
&self.file_boundaries
}
pub fn lexer_config(&self) -> LexerConfig {
self.lexer_config
}
/// Parse for a **target compiler**: what its reserved spellings mean. See [`Dialect`].
pub fn with_dialect(mut self, dialect: Dialect) -> Self {
self.dialect = dialect;
// **A dialect is a vendor**, and that is the only thing it ever said: which compiler's own spellings this parse
// should recognise. Passing it on as a vendor keeps the two layers consistent instead of leaving the flags in two
// vocabularies that can disagree.
let vendor = match dialect {
Dialect::Gnu => Some(crate::features::Vendor::Gnu),
Dialect::Msvc => Some(crate::features::Vendor::Msvc),
};
if let Some(vendor) = vendor {
self.lexer_config.language = self.lexer_config.language.with_vendor(vendor);
}
self
}
/// **The syntax this parse accepts**, as a set of standards and vendors — see [`crate::features`].
pub fn with_language(mut self, language: crate::features::LanguageConfig) -> Self {
self.lexer_config.language = language;
self
}
pub fn language(&self) -> crate::features::LanguageConfig {
self.lexer_config.language()
}
pub fn dialect(&self) -> Dialect {
self.dialect
}
/// Replace the lexer settings. Used by the parser to re-lex a header name, and by callers that
/// need a non-default dialect (for example `$` in identifiers).
pub fn with_lexer_config(mut self, lexer_config: LexerConfig) -> Self {
self.lexer_config = lexer_config;
self
}
/// Parse with an **external symbol table**: what the caller has already resolved about names this file
/// cannot see. See [`crate::symbols`] for the contract and for the order the evidence is consulted in — the
/// table is a *preference*, not a requirement, and parsing without one is the same as parsing with
/// [`crate::NoSymbols`].
pub fn with_symbol_table(mut self, symbol_table: &'cache dyn SymbolTable) -> Self {
self.symbol_table = Some(symbol_table);
self
}
/// The table in force, if the caller supplied one.
///
/// `None` is the ordinary case — a buffer parsed with no context at all — and every caller in the grammar must
/// treat it as "no evidence from outside", never as "nothing is a type".
pub fn symbol_table(&self) -> Option<&'cache dyn SymbolTable> {
self.symbol_table
}
/// **Parse with the macros the file's includes define.** See the field's note.
///
/// The caller that has an include closure — the index, and a session's view of an open file — is the only one
/// that can supply this, and it is the difference between a grammar rule *guessing* that `_STD` is a macro from
/// its spelling and *knowing* it.
pub fn with_macros_from_includes(mut self, macro_facts: &'cache dyn MacroFacts) -> Self {
self.macro_facts = Some(macro_facts);
self
}
/// The macros in force, if the caller supplied any.
pub fn macro_facts(&self) -> Option<&'cache dyn MacroFacts> {
self.macro_facts
}
/// **Is `name` a macro at `at`?** — the question every grammar rule that used to read a spelling is asking.
///
/// Three answers, and the third is the one that keeps this honest: `Some(true)` and `Some(false)` are the
/// environment speaking, and **`None` is nobody speaking** — which a caller must read as "use the spelling",
/// never as "not a macro". [`crate::NothingAtAll`] and an absent environment both answer `None` for every name,
/// so a parse without one behaves exactly as it did before this method existed.
pub fn is_a_macro_at(&self, name: &str, at: usize) -> Option<bool> {
let facts = self.macro_facts?;
Some(
facts.is_a_macro_at(name, at)
|| facts.body_text_in_force(name).is_some()
|| facts.body_text_of(name, at).is_some(),
)
}
pub fn node_cache(&mut self) -> Option<&mut NodeCache> {
self.node_cache.as_deref_mut()
}
}
/// Not derived: `level` and `lexer_config` have to agree, and the derived version would silently
/// pair an explicit level with a stale lexer configuration whenever either default changes.
#[allow(clippy::derivable_impls)]
impl Default for ParserConfig<'_> {
fn default() -> Self {
Self {
cuda: false,
level: CppLanguageLevel::default_level(),
dialect: Dialect::default(),
lexer_config: LexerConfig::default(),
node_cache: None,
symbol_table: None,
macro_facts: None,
file_boundaries: Vec::new(),
skip_bodies: false,
}
}
}