pub struct CppParser<'a> {
pub parse_config: ParserConfig<'a>,
/* private fields */
}Fields§
§parse_config: ParserConfig<'a>Implementations§
Source§impl<'a> CppParser<'a>
impl<'a> CppParser<'a>
Sourcepub fn parse(text: &'a str, config: ParserConfig<'a>) -> CppSyntaxTree
pub fn parse(text: &'a str, config: ParserConfig<'a>) -> CppSyntaxTree
Parse text into a lossless syntax tree.
This never fails: every input file, however broken or mid-edit, produces a tree covering
all of text (invariant I1). Problems are reported through CppSyntaxTree::get_errors
and through ErrorNode/MissingNode nodes, not through a Result.
Sourcepub fn parse_with_audit(
text: &'a str,
config: ParserConfig<'a>,
) -> (CppSyntaxTree, EventStreamAudit)
pub fn parse_with_audit( text: &'a str, config: ParserConfig<'a>, ) -> (CppSyntaxTree, EventStreamAudit)
Like CppParser::parse, but also reports the raw event stream’s balance.
Tests use this because the tree alone cannot reveal a recovery bug: an unclosed NodeStart
still yields a well-formed tree, just one where a subtree swallowed its following siblings.
Sourcepub fn parse_tokens(
text: &'a str,
tokens: &[CppTokenData],
config: ParserConfig<'a>,
) -> CppSyntaxTree
pub fn parse_tokens( text: &'a str, tokens: &[CppTokenData], config: ParserConfig<'a>, ) -> CppSyntaxTree
Parse a run of a longer text’s tokens, not the whole of text.
This is what an editor needs: a file’s text arrives as a slice of a stream that was lexed once
for something much larger — the stitched, macro-expanded translation unit — and reparsing the
whole of it for a keystroke in one file is the wrong work (see docs/parse-architecture.md).
§The contract
- The tree covers
tokensand nothing else.to_source_textgives back the byte span those tokens spell, and a run of every token oftextis the same parseCppParser::parsemakes of it — same shape, same tokens, same text. - Node ranges are relative to the slice, token ranges are not. A
CppSyntaxNode’stext_rangestarts at zero because a green tree is built from token lengths; the absolute anchor istokens[0].range.start_offset— and every token inCppSyntaxTree::get_tokenskeeps the range it was lexed with, so nothing is lost, but a query that maps a node’s range to a file offset has to add the anchor. - The grammar sees end-of-input after the last token, because that is the truth about what it was given. A run that ends in the middle of a construct therefore parses as a truncated construct, and the caller’s job is to cut at a boundary it can answer for — which is what a region of a file is.
- Only the grammar’s errors are reported. The tokens were lexed by the caller, so lexical problems belong to whoever lexed the whole text; this tree’s errors are what the grammar said about this run.
Sourcepub fn parse_tokens_with_audit(
text: &'a str,
tokens: &[CppTokenData],
config: ParserConfig<'a>,
) -> (CppSyntaxTree, EventStreamAudit)
pub fn parse_tokens_with_audit( text: &'a str, tokens: &[CppTokenData], config: ParserConfig<'a>, ) -> (CppSyntaxTree, EventStreamAudit)
CppParser::parse_tokens, plus the event stream’s balance — see
CppParser::parse_with_audit for why that is a separate question from the tree’s shape.
Sourcepub fn parse_with_events(
text: &'a str,
config: ParserConfig<'a>,
) -> (CppSyntaxTree, Vec<MarkEvent>)
pub fn parse_with_events( text: &'a str, config: ParserConfig<'a>, ) -> (CppSyntaxTree, Vec<MarkEvent>)
Like CppParser::parse, but also returns the raw event stream.
The event stream is the parser’s real output and the tree is a fold of it, so when a tree is
wrongly nested the stream is where the cause is visible. Kept public because diagnosing a
shape bug otherwise means adding a temporary println! inside the parser.
Sourcepub fn init(&mut self)
pub fn init(&mut self)
Position the cursor on the first non-trivia token, emitting the leading trivia as events.
Emitting the leading trivia matters: without it the whitespace and comments before the first real token would never be attached to the tree and the CST would silently stop being lossless.
pub fn is_eof(&self) -> bool
pub fn origin_text(&self) -> &'a str
Sourcepub fn token_count(&self) -> usize
pub fn token_count(&self) -> usize
How many tokens the parser was handed, trivia included.
The bound for a scan that walks the tokens ahead of the cursor by index rather than by
peek_token_kind_at, which counts significant tokens and so cannot step past an unknown
amount of trivia one position at a time.
pub fn current_token(&self) -> CppTokenKind
pub fn current_token_index(&self) -> usize
Sourcepub fn anchor(&self) -> ParseAnchor
pub fn anchor(&self) -> ParseAnchor
Mark the current position for a later question about what has been parsed here.
Two positions describe a moment in a parse and they are not interchangeable: CppParser::open_marks is
a length of the marker stack, for closing the nodes opened since, while Checkpoint is where the event
stream and cursor stood. Neither can answer a question about tokens already consumed — a marker records
an event index and the marker stack is not one — and passing the wrong one is a silent mistake rather than
a loud one, because all three are small integers. That is what this type exists to prevent.
pub fn current_token_range(&self) -> SourceRange
Sourcepub fn crossed_a_file_boundary_since(&self, since: usize) -> bool
pub fn crossed_a_file_boundary_since(&self, since: usize) -> bool
Has the cursor crossed a file boundary since since?
A scope may not cross a file boundary — see ParserConfig::with_file_boundaries. A block’s body loop asks
this with the offset its own { (or its own start) sits at, and stops where that file’s text stopped, so
that the { a header left open is closed at the end of that header rather than paired with a } written
later, or holding every file spliced after it.
§Why it takes the block’s own start
“Is the cursor past some boundary” is not the question: a block that begins inside file n has every token of file n past the boundaries of the files before it, and would stop immediately. What matters is a boundary strictly after the block began and at or before the cursor — a boundary the block has crossed.
[since, at] rather than (since, at] is deliberate. A block that begins exactly on a boundary is a block
that begins in a new file, which is the state after a header’s leaked scope was closed: the boundary is the
cursor’s own position, and treating it as crossed would close the new file’s block at its own first token.
pub fn current_token_text(&self) -> &str
Sourcepub fn checkpoint(&self) -> Checkpoint
pub fn checkpoint(&self) -> Checkpoint
Record a checkpoint that CppParser::rollback can restore.
Sourcepub fn rollback(&mut self, checkpoint: Checkpoint)
pub fn rollback(&mut self, checkpoint: Checkpoint)
Rewind to checkpoint, discarding every event, token and diagnostic produced since.
Any markers opened after the checkpoint are dropped along with their events, so callers
must not hold on to a Marker created inside a speculative region.
§Why the diagnostics go too
A rollback is always “that reading did not fit, try another” — a.b < c > d against a.b<c> d, a cast
against a parenthesised expression, an init-declarator against an expression statement. Whatever the
discarded reading complained about is not a fact about the file: it is a fact about a guess. Keeping it
reports code the user did not write, which is worse than saying nothing — the file that comes out is valid
C++ and the editor would underline it. The reading that is kept reports its own problems normally.
Sourcepub fn try_parse<T>(
&mut self,
f: impl FnOnce(&mut Self) -> Option<T>,
) -> Option<T>
pub fn try_parse<T>( &mut self, f: impl FnOnce(&mut Self) -> Option<T>, ) -> Option<T>
Run f speculatively: if it returns None, everything it consumed is rolled back.
This is the primitive that makes C++’s declaration/expression ambiguity tractable without a symbol table.
Sourcepub fn try_lex_header_name(&mut self) -> bool
pub fn try_lex_header_name(&mut self) -> bool
Try to consume a header name at the cursor, folding the <, name and > tokens the ordinary
sweep produced into a single CppTokenKind::HeaderName.
Only the parser knows when a header name is expected (#include and friends), and only the
parser can rewrite the token stream, so the two halves of header-name lexing live on either
side of this call. Returns whether a header name was consumed; on false the caller should
lex the token normally.
Sourcepub fn events_contain_any(
&self,
from_event: usize,
kinds: &[CppSyntaxKind],
) -> bool
pub fn events_contain_any( &self, from_event: usize, kinds: &[CppSyntaxKind], ) -> bool
Does any node opened at or after from_event have one of these kinds?
Used by the grammar to answer “what did I just parse?” without threading a return value
through every level. from_event is an event index from
CppParser::current_event_count.
pub fn joined_operator_at_the_cursor(&self) -> (CppTokenKind, usize)
Sourcepub fn current_event_count(&self) -> usize
pub fn current_event_count(&self) -> usize
Number of events recorded so far, for use as a from_event bound.
Sourcepub fn brace_balance_since(&self, from_event: usize) -> isize
pub fn brace_balance_since(&self, from_event: usize) -> isize
How many { tokens were consumed at or after from_event without a } to match them?
The question a recovery has to ask about a construct it gave up on. Closing the markers of a failed
sub-parse ([MarkerEventContainer::finish_marks_to]) keeps the abandonment from swallowing what follows,
but it cannot un-consume the tokens: a member that failed after eating a { — a requires-expression’s
body, a function body, a block — leaves a brace behind that the enclosing body then spends its own } on.
The class body ends early, every member after it is read at file scope, and the only diagnostic in the file
lands on the leftover brace at the end (which is exactly what bits/alloc_traits.h did before the brace
debt was paid).
Counted from the events, not the token stream: a rollback truncates the events, so a token read, thrown
away and read again counts once — the same reason CppParser::events_contain_any reads them.
Sourcepub fn events_contain_any_between(
&self,
from_event: usize,
to_event: usize,
kinds: &[CppSyntaxKind],
) -> bool
pub fn events_contain_any_between( &self, from_event: usize, to_event: usize, kinds: &[CppSyntaxKind], ) -> bool
Has a node of one of these kinds been opened between from_event and the cursor?
The counterpart to CppParser::events_contain_any, which asks about everything from from_event
onwards. This one is bounded on both sides, which is what lets a rule ask a question about the
declaration it is in the middle of: “has this declarator named anything yet?” is answered by a
CppSyntaxKind::NameExpr between where the declarator started and where the cursor stands, and the
lower bound is what keeps the type’s own name — a NameExpr inside the specifier sequence, recorded
before that point — from being mistaken for the declarator’s.
This matters because the two questions are asked by the same function for two different statements:
Widget w(1, 2, 3); names w and must be a declaration, while g(1, 2, 3); names nothing and must
stay a call. Both have a name in type position; only the first has one after it.
Sourcepub fn greater_than_is_an_operator(&self) -> bool
pub fn greater_than_is_an_operator(&self) -> bool
Does a > at the cursor compare, or does it close the list the cursor is in?
The one question the expression grammar asks about template arguments, and the answer is the standard’s:
a > closes the list only when it is not nested in a bracketed group opened since that list began.
See CppParser::innermost_template_argument_list for the mechanism and the input it was measured on.
Outside every list the answer is true, which is what keeps a > b an ordinary comparison everywhere else
in the file. >= and >>= are never this question’s business — they are different tokens and stay
operators either way — and >> is, which is why a >> b inside C<(a >> b)> reads as a shift.
Sourcepub fn dialect(&self) -> Dialect
pub fn dialect(&self) -> Dialect
Which compiler’s own reserved spellings mean what — see Dialect.
Read by the grammar in exactly one place (a_type_the_compiler_spells), which is the point: the question
this answers is not “how is this file written” but “what does the compiler reading it mean by
__int128”, and only the caller knows which compiler that is.
Sourcepub fn language(&self) -> LanguageConfig
pub fn language(&self) -> LanguageConfig
Which language the file is, as the grammar’s feature switches — see crate::ParserConfig::with_language.
Read where a construct is a language feature rather than a dialect’s spelling: _Generic is C11’s, so a
C++17 parse must see the identifier and only a C11-and-later one may read the associations. The lexer already
asks the same configuration (crate::lexer::LexerConfig::language), so a rule that asks here cannot disagree
with it about what the file is.
Sourcepub fn enter_template_arguments(&mut self) -> Option<usize>
pub fn enter_template_arguments(&mut self) -> Option<usize>
Enter a template argument list, recording the bracket depth it began at.
Returns what was in force before, for the caller to hand back to CppParser::leave_template_arguments.
Option<usize> rather than usize because the two states are not the same question: None is “no list
is open, so every > compares”, while Some(0) is “a list began at the outermost bracket depth”. The
version before this one spelled both as the number zero, which is how C<(1 > 2)> came to be read as a
mis-nested list rather than as a comparison inside parentheses.
Sourcepub fn leave_template_arguments(&mut self, previous: Option<usize>)
pub fn leave_template_arguments(&mut self, previous: Option<usize>)
Leave a template argument list, restoring what CppParser::enter_template_arguments returned.
Sourcepub fn is_in_a_constraint(&self) -> bool
pub fn is_in_a_constraint(&self) -> bool
Is the cursor inside a constraint, where a { is not a braced initialiser?
See the field’s documentation. Read by the postfix rule, which is the one that would otherwise take the
{ after a requires-clause for list-initialisation and swallow the body of the definition.
Sourcepub fn enter_constraint(&mut self) -> usize
pub fn enter_constraint(&mut self) -> usize
Enter a constraint. Returns the previous depth so the caller can restore it.
Sourcepub fn leave_constraint(&mut self, previous: usize)
pub fn leave_constraint(&mut self, previous: usize)
Leave a constraint, restoring the depth enter_constraint returned.
Sourcepub fn a_template_id_may_be_the_name(&self) -> bool
pub fn a_template_id_may_be_the_name(&self) -> bool
May the declaration being parsed be named by a bare template-id? See the field’s documentation.
Read by the declarator rule, which is the one that would otherwise refuse the template-id that names an instantiation or a specialization.
Sourcepub fn set_a_template_id_may_be_the_name(&mut self, value: bool) -> bool
pub fn set_a_template_id_may_be_the_name(&mut self, value: bool) -> bool
Record whether this declaration may be named by a bare template-id, handing back the previous answer.
Sourcepub fn set_last_declarator_is_function(&mut self, is_function: bool)
pub fn set_last_declarator_is_function(&mut self, is_function: bool)
Record whether the declarator just parsed declared a function. See the field’s docs.
Sourcepub fn last_declarator_is_function(&self) -> bool
pub fn last_declarator_is_function(&self) -> bool
Did the declarator parsed most recently declare a function?
Sourcepub fn declare_type_name(&mut self, name: &str)
pub fn declare_type_name(&mut self, name: &str)
Record that name was declared to be a type, at the current scope depth.
Called by the grammar where a name sits in type position: class Widget, typedef ... Integer,
using Alias = .... See TypeNames for why the parser needs this at all.
Sourcepub fn is_a_known_type_name(&self, name: &str) -> bool
pub fn is_a_known_type_name(&self, name: &str) -> bool
Is name a type as far as this file’s own declarations say?
The question that separates Widget w(1, 2); from g(1, 2);. A false is not “not a type” but “this
file does not say it is one” — a type from an included header, a template parameter, or a builtin this
table never saw. Callers must therefore treat it as one signal among several rather than as the answer.
A template parameter counts, and the two sources are kept apart because they are scoped differently:
see CppParser::template_parameters.
Sourcepub fn declare_template_parameter(&mut self, name: &str)
pub fn declare_template_parameter(&mut self, name: &str)
Record a template parameter that declares a type — typename T, class C.
Called by the template parameter rule, which is the only place that knows which spelling of a parameter
this is: int N and auto N declare values, and N[4] is not an array of anything.
Sourcepub fn template_parameter_count(&self) -> usize
pub fn template_parameter_count(&self) -> usize
How many template parameters are in scope, so that CppParser::truncate_template_parameters can cut
the list back to a caller’s own state.
Sourcepub fn truncate_template_parameters(&mut self, count: usize)
pub fn truncate_template_parameters(&mut self, count: usize)
Forget every template parameter recorded since count names were in scope.
The list is appended as heads are read and cut back by the declaration that owns them, which is what
keeps a parameter from being a type name for the rest of the file. Nested declarations save and restore
around themselves, so a class template’s members still see the class’s parameters: the length the member’s
own parse_declaration saved already includes them.
Sourcepub fn enter_type_name_scope(&mut self)
pub fn enter_type_name_scope(&mut self)
Enter a braced body, for the table’s scope approximation.
Sourcepub fn leave_type_name_scope(&mut self)
pub fn leave_type_name_scope(&mut self)
Leave a braced body.
Sourcepub fn last_consumed_token_kind(&self) -> Option<CppTokenKind>
pub fn last_consumed_token_kind(&self) -> Option<CppTokenKind>
The kind of the token the parser consumed most recently, skipping the trivia after it.
The answer to “what did the rule that just ran end on?”, which the cursor cannot give: after a bump the
cursor is on the next token, so a rule that has to describe its own last token has to walk back. Used by
the specifier sequence, which asks the question of the specifier it just consumed.
None at the start of the file, where nothing has been consumed.
Sourcepub fn token_text_at(&self, index: usize) -> &str
pub fn token_text_at(&self, index: usize) -> &str
The text of the token at index, or empty when there is none.
The companion to CppParser::token_kind_at, for the walks that go backwards over the tokens already
consumed — which is how the grammar asks what a declaration began with. peek_token_text_at cannot
answer that: it walks forward from the cursor.
Sourcepub fn the_name_a_declarator_introduced(
&self,
from_event: usize,
) -> Option<String>
pub fn the_name_a_declarator_introduced( &self, from_event: usize, ) -> Option<String>
The unqualified name the declarator that began at from_event introduced, if it named anything.
§Why the event stream and not a return value
A declarator’s name is not always where a rule can see it: typedef WCHAR *PWCHAR; has the name after a
*, typedef void (*F)(int); has it inside parentheses, and ns::Widget *w has a qualified spelling
whose last component is the name. Threading a Option<String> back through every return of
parse_declarator and its six helpers would touch rules that have no interest in the answer, and the
grammar already answers “what did I just parse?” from the events in
CppParser::events_contain_any_between. This is the same question with the text attached.
§How the name is picked
The first NameExpr at the declarator’s own depth, and then the last identifier inside it. Both halves
are load-bearing:
typedef WCHAR *PWCHAR; the NameExpr is one level in: `PWCHAR`
typedef void (*F)(int); the declarator's is two levels in: `F`
typedef void (*F)(Widget w); …and the *parameter's* NameExpr is deeper still, so `w` cannot win
ns::Widget *w; the NameExpr holds `ns :: Widget`, whose last identifier is the nameA declarator that names nothing — an abstract declarator in a type-id, a structured binding — answers
None, which is the honest answer and the one the callers act on.
Sourcepub fn type_names(&self) -> &TypeNames
pub fn type_names(&self) -> &TypeNames
The table itself, for a consumer that wants to audit what the parse recorded.
Sourcepub fn remembered_follower(&self, index: usize) -> Option<bool>
pub fn remembered_follower(&self, index: usize) -> Option<bool>
What was remembered for CppParser::follower_memo, if anything.
Sourcepub fn remember_follower(&self, index: usize, answer: bool)
pub fn remember_follower(&self, index: usize, answer: bool)
Remember an answer for CppParser::follower_memo.
Sourcepub fn token_range_at(&self, index: usize) -> Option<SourceRange>
pub fn token_range_at(&self, index: usize) -> Option<SourceRange>
The source range of the token at index, or None when there is none.
Beside CppParser::token_text_at for the rules that ask the evidence about a token that is not at the
cursor: a macro’s body is in force from an offset, so “what does this name stand for here” needs to know
where here is.
Sourcepub fn symbol_kind(&self, name: &str) -> Option<SymbolKind>
pub fn symbol_kind(&self, name: &str) -> Option<SymbolKind>
What the caller’s table says about name, if the caller supplied one.
None is “no evidence from outside” — never “not a type”. See [crate::symbols].
Sourcepub fn is_a_macro(&self, name: &str) -> Option<bool>
pub fn is_a_macro(&self, name: &str) -> Option<bool>
Is name a macro where the cursor stands? — the caller’s environment, when there is one.
None is “nobody says”, and every rule that asks must fall back to the reading it had before the question
could be asked at all. See ParserConfig::is_a_macro_at for what the three answers mean, and
CppParser::token_range_at for why the position is part of the question: a macro’s body is in force from
an offset, so _STD is a macro below the #include that defines it and an identifier above.
Sourcepub fn begin_declaration_type(&mut self)
pub fn begin_declaration_type(&mut self)
Start recording the declaration’s leading type name; see the field’s documentation.
The name currently in force is parked rather than dropped, because a speculative region may
run this and then be rolled back — see Checkpoint.
Sourcepub fn note_declaration_type_name(&mut self, name: String)
pub fn note_declaration_type_name(&mut self, name: String)
Record a name seen in type position, if none has been recorded for this declaration yet.
Only the first: unsigned int has two type words and the declaration’s type is the sequence, not either
word — but what the reader needs is only “is the leading name a type this file declared”, and the first
name is the one a qualifier would precede.
Sourcepub fn note_qualified_declaration_type(&mut self)
pub fn note_qualified_declaration_type(&mut self)
Record that the declaration’s type was written as a qualified name.
Called by the specifier sequence as it walks a name’s ::-separated segments. Read once, by the
declarator, to tell void ns::C::method() — where the last segment is the function being defined —
from Foo(1, 2);, where the name is a callee and the parentheses are an argument list.
Sourcepub fn has_qualified_declaration_type_name(&self) -> bool
pub fn has_qualified_declaration_type_name(&self) -> bool
Was the declaration’s type written as a qualified name? See the field’s documentation.
Sourcepub fn declaration_type_record(&self) -> (Option<String>, bool)
pub fn declaration_type_record(&self) -> (Option<String>, bool)
The recorded declaration type name, owned, for a caller that has to hold it across a nested parse.
The pair to CppParser::set_declaration_type_record; see that function for what is being saved and
why the saving happens at the call site rather than inside the specifier sequence.
Sourcepub fn set_declaration_type_record(&mut self, record: (Option<String>, bool))
pub fn set_declaration_type_record(&mut self, record: (Option<String>, bool))
Put back a record saved by CppParser::declaration_type_record.
A nested parse must not be allowed to answer for the declaration that contains it. A template
argument is read with [crate::grammar::cpp::types::parse_type_id], a type-id opens a specifier
sequence, and that sequence begins by calling CppParser::begin_declaration_type — which parks the
enclosing name and clears the field. Nothing restored it, so G<double> d(…) lost the "G" as soon as
its argument was read, and the declarator afterwards found whatever the argument had left: "S" for
G<S>, and nothing at all for G<double>, whose argument is a builtin keyword that no name-recording
branch covers.
The field answers “did this declaration write a name in type position” — the question that decides
whether the ( after the declarator is an initializer or a parameter list
([crate::grammar::cpp::decls::a_declaration_is_the_better_reading]) — so both answers were wrong for
a declaration that had written G in plain sight. Measured: G<double> d(static_cast<double>(x)); was
refused while S s(static_cast<double>(x)); parsed.
A local, not a parser field. Giving the specifier sequence a matching “end” that restored a shared park slot was tried and is not safe: the slot is written on entry and was only read on the success path, so every sequence that returned an error left it occupied, and the next successful sequence restored a stale name. Fifty headers went from 5 diagnostics to 399, and the parser’s own tests from 21 green binaries to 11. A local around one call has no state to leak, and the error paths need it least — a failed argument fails the whole template-id, and the rollback that follows restores the field anyway.
Sourcepub fn enter_class_body(&mut self)
pub fn enter_class_body(&mut self)
Enter a class body, for the declaration rule that distinguishes a bit-field from a member initializer.
Pushed on a stack rather than counted, because what the rule needs is the innermost brace: see
CppParser::is_at_class_member_level. Deliberately not part of Checkpoint: no speculative region
opens a brace and then rewinds past it, since the braces it would be rewinding over are its own rather
than a guess.
Sourcepub fn enter_block_body(&mut self)
pub fn enter_block_body(&mut self)
Enter a statement or declaration block — a function body, a nested { … }, an extern "C" { … }.
The other half of the stack above: a : inside one of these is never a bit-field’s width, whatever class
body encloses it.
Sourcepub fn note_declaration_ended_inside_specifiers(&mut self)
pub fn note_declaration_ended_inside_specifiers(&mut self)
Record that the specifier sequence finished a whole declaration — which only friend does.
Called by the specifier rule for friend, whose payload is the declaration that follows it, including
its ;. See the field’s documentation.
Sourcepub fn take_declaration_ended_inside_specifiers(&mut self) -> bool
pub fn take_declaration_ended_inside_specifiers(&mut self) -> bool
Take the flag above: was the statement already over when the specifiers ended?
A take rather than a read, so the answer cannot leak into the next declaration: the flag describes the sequence just parsed, and the caller that asks is the one that parses the declaration around it.
Sourcepub fn note_the_terminator_came_from_a_branch(&mut self)
pub fn note_the_terminator_came_from_a_branch(&mut self)
Record that this rule consumed the ; that ends the declaration, from inside a conditional branch.
See the field’s documentation: the readers that call this are the ones that read a construct’s tail once
per branch, and the ; each branch writes is part of the tail they read.
Sourcepub fn take_the_terminator_came_from_a_branch(&mut self) -> bool
pub fn take_the_terminator_came_from_a_branch(&mut self) -> bool
Take the flag above: did the construct just parsed already consume the declaration’s ;?
A take for the reason every flag here is: the answer describes one construct, and the caller that asks
(the rule that would otherwise require a ;) is the one that parses around it.
Sourcepub fn open_conditionals(&self) -> isize
pub fn open_conditionals(&self) -> isize
How many conditionals are open at the cursor, as far as the directives read so far say.
A construct whose tail is written once per branch has to tell the #else that closes a conditional it
opened itself from the #else of a conditional opened around it — the two spellings are otherwise
identical (void f(int a); / #else / void g(long a); reads as parameters just as well as a per-branch
tail does). Counting the directives a construct read does not answer that: a nested reading may have
consumed one on its way past, which is exactly what happens in type_traits:2269, where the expression
grammar’s operator-position seam takes the # if before the conditional can see it.
So the count is kept by the directive reader itself, and a construct records the value when it starts and compares it where the question is asked.
Sourcepub fn note_a_directive_name(&mut self)
pub fn note_a_directive_name(&mut self)
Record the directive name at the cursor, for CppParser::open_conditionals.
Called by the directive rule, which is the one place a directive is read — including the ones read by rules
that are not that rule’s caller. It is called with the cursor on the name (not on the #), and only when
the directive has one: the null directive (# alone on a line) names nothing.
Sourcepub fn leave_class_body(&mut self)
pub fn leave_class_body(&mut self)
Leave a class body.
Sourcepub fn leave_block_body(&mut self)
pub fn leave_block_body(&mut self)
Leave a statement or declaration block.
Sourcepub fn is_at_class_member_level(&self) -> bool
pub fn is_at_class_member_level(&self) -> bool
Is the cursor at the member level of a class body — the innermost brace being the class’s own?
The question finish_init_declarator puts to it: int bits : 3; and S() : a(1) {} have a : in the
same position, and only the enclosing construct says which they are — a member declarator’s :
introduces a width, while a function declarator’s introduces the member-initializer list of a
constructor. The function case is decided first and does not need this; what does is the member that
names no function.
The innermost brace is what matters, not “is a class body somewhere above”. Asking the weaker question
meant that everything inside a member function’s body counted as member level, which turned
for (auto &v: vec) in a header’s inline member into a bit-field of width vec — and the same for a
label (again:) or any other statement whose first name a declaration reading could take.
Sourcepub fn declaration_type_name(&self) -> Option<&str>
pub fn declaration_type_name(&self) -> Option<&str>
The declaration’s leading type name, when it has one.
Sourcepub fn note_class_head_name(&mut self, name: &str)
pub fn note_class_head_name(&mut self, name: &str)
The name of the class body the cursor is in, when it is in one and that class has a name.
What a constructor’s name has to equal: \Widget(int);\ inside \class Widget\ is a constructor, and the same two tokens inside \class Other\ are a member function whose return type was written somewhere the parser could not see it (an #if\ branch, a macro). Measured: without this the second shape was read as the first. Record the name a class head declared, for the body that follows it.
pub fn current_class_name(&self) -> Option<&str>
Sourcepub fn has_declaration_type_name(&self) -> bool
pub fn has_declaration_type_name(&self) -> bool
Has a name been recorded in type position for the declaration being parsed?
The weaker form of CppParser::declaration_type_name, for a caller that only needs to know whether
the specifier sequence saw a name at all — a declarator left without one is then not an abstract
declarator but the mark of a name that was taken for a type.
Sourcepub fn is_inside_a_body(&self) -> bool
pub fn is_inside_a_body(&self) -> bool
Is the cursor inside a braced body?
The signal a_declaration_is_the_better_reading uses: a function declaration inside a function body is
vanishingly rare, while a local variable with constructor arguments is everywhere, so a known type name
inside a body is read as a declaration.
A parser depth rather than a C++ scope — a class body counts the same as a function body. That is the
approximation TypeNames documents, and it is on the safe side here: what the answer licenses is the
declaration reading, which is the cheaper of the two mistakes.
What it must not count is a brace that is not a body: a linkage specification’s block
(extern "C++" { … }) introduces no scope for names at all — see parse_linkage_block — so the depth
does not move inside one, and the rules that ask “am I inside a body” (the macro-from-a-header rules of
stats.rs and decls.rs) keep answering no there. Counting it was a bug with wide reach: most of
libstdc++ is written inside extern "C++" { namespace std { … } }, so _GLIBCXX_BEGIN_NAMESPACE_VERSION
was read as a name instead of as the macro it is, and the declaration behind it came out as an expression
(expected ; after expression at using ::wint_t; — the first error of cwchar, cstdlib and every
other header with that shape).
Sourcepub fn is_at_file_scope(&self) -> bool
pub fn is_at_file_scope(&self) -> bool
Is the cursor at file scope, outside every braced body?
Named separately from CppParser::is_inside_a_body because callers read better for it, and because the
two together are exhaustive: a statement is either in a body or not.
Sourcepub fn peek_token_range_at(&self, offset: usize) -> SourceRange
pub fn peek_token_range_at(&self, offset: usize) -> SourceRange
Source range of the significant token at relative offset offset from the cursor.
The third of the peek_token_*_at family, and it exists for the questions that are about the source
rather than about the tokens: whether two things are written together or apart. operator T&&() and
operator bool() && are the same three tokens in the same order, and only the spacing separates the
type from the ref-qualifier.
A zero-width range at the end of input, mirroring CppParser::current_token_range, so a caller
comparing offsets never has to handle absence separately.
Sourcepub fn peek_token_text_at(&self, offset: usize) -> &str
pub fn peek_token_text_at(&self, offset: usize) -> &str
Text of the significant token at relative offset offset from the cursor. Empty past the end.
The companion to CppParser::peek_token_kind_at, needed because module and import are
contextual keywords: they must be recognised by spelling, not by kind.
Sourcepub fn token_kind_at(&self, index: usize) -> CppTokenKind
pub fn token_kind_at(&self, index: usize) -> CppTokenKind
Kind of the token at index, ignoring trivia. None past the end.
pub fn bump(&mut self)
pub fn peek_next_token(&self) -> CppTokenKind
Sourcepub fn peek_token_kind_at(&self, range: Range<usize>) -> Vec<CppTokenKind>
pub fn peek_token_kind_at(&self, range: Range<usize>) -> Vec<CppTokenKind>
Kinds of the next range.len() significant tokens, starting at the cursor.
Trivia is skipped, so this is “the next few things the grammar will see”. Shorter than
range near end of input, and padded with CppTokenKind::None so callers can index it
without a length check.
§Comparing the result
This returns a Vec, and Vec<T> == [T; N] is always false in Rust — there is no
PartialEq<[T; N]> impl. A comparison written as peek_token_kind_at(1..2) == [X] therefore
compiles and silently never matches, which is exactly the kind of bug that reads as “the
branch is dead” rather than “the comparison is wrong”. Call .as_slice() or use the
single-token helpers (CppParser::peek_next_token, CppParser::peek_token_text_at).
pub fn push_error(&mut self, err: CppParseError)
Sourcepub fn consume_current_token(&mut self)
pub fn consume_current_token(&mut self)
Emit the current token and move on, without attaching the trivia that follows it.
bump is the right call almost everywhere: trivia belongs to the construct it sits inside.
A grammar rule that wants to close its node before that trivia is emitted — because the trivia
belongs to the enclosing construct — advances with this and then calls
CppParser::emit_trivia_after_current_token.
Sourcepub fn consume_current_token_if(&mut self, kind: CppTokenKind) -> bool
pub fn consume_current_token_if(&mut self, kind: CppTokenKind) -> bool
Consume the current token with CppParser::consume_current_token if it has this kind.
Returns whether it was consumed. Used for tokens a rule requires but wants to treat as optional at the point of consumption, so that a missing one is reported by the caller rather than aborting a construct that is otherwise complete.
Sourcepub fn emit_trivia_after_current_token(&mut self)
pub fn emit_trivia_after_current_token(&mut self)
Emit the trivia the cursor is sitting on, without consuming any significant token.
The counterpart to CppParser::consume_current_token: after a rule has closed its node, the
layout it left behind still has to reach the tree — the CST is lossless — and this is what puts
it into whichever node is open now.
Sourcepub fn emit_layout_after_current_token(&mut self)
pub fn emit_layout_after_current_token(&mut self)
Emit the layout the cursor is sitting on and stop at the first comment.
§Why a rule wants this instead of bump
bump attaches everything after a token to whichever node is open, and a rule that is about to close its node
is still open — so a comment written below the construct it just read becomes a child of that construct.
Measured: struct S { int x; /// The y. int y; }; made /// The y. a child of the declaration of x, and the
module note in tests/doc_owner.rs says what the shape should be — the comment between two members belongs to
the class body, where a reader sees it.
Consuming only the layout and stopping short of the comment is what puts it there: the caller completes its node, and the comment is emitted afterwards into the node that encloses it both.
§The layout still has to be emitted
Stopping at a comment is not the same as stopping at all trivia. A version that consumed the ; and emitted
nothing stranded the layout entirely: five tests in tests/ast.rs failed on the tree no longer reproducing the
file, because the newline after a } was never written to the tree by anything. Whitespace and line breaks go
in here, as they did before; only the comment is left for later.
Sourcepub fn emit_missing_node(&mut self)
pub fn emit_missing_node(&mut self)
Emit a zero-width MissingNode, i.e. “a token was expected here but is not present”.
Zero-width nodes cost nothing in the tree and are what make completion work at a broken position: the cursor is inside a node of the expected kind rather than in an error blob.
Sourcepub fn open_marks(&self) -> usize
pub fn open_marks(&self) -> usize
Snapshot the set of currently open nodes. Pass the result to
CppParser::close_marks_above in every ?-using grammar function’s error path.
The snapshot is a count rather than a depth, and it is only meaningful as long as the stack below it is untouched: closing the nodes above it is driven by the open-node stack itself, so a rule that already closed some of its own nodes cannot confuse it.
Sourcepub fn close_marks_above(&mut self, base: usize)
pub fn close_marks_above(&mut self, base: usize)
Close every node opened after CppParser::open_marks was snapshotted, as if its closing
token had been present.
Call this on every early return from a grammar function. A ? return leaves markers open,
and an open NodeStart sits before the ancestor that eventually closes, so all
following tokens would be swallowed into it — a silent corruption of the whole rest of the
file rather than a local error.
Sourcepub fn recover_to_level(&mut self, base: usize)
pub fn recover_to_level(&mut self, base: usize)
Close any node opened after base, keeping the consumed tokens in the tree.
Unlike CppParser::rollback, which erases events, this keeps the text and only
re-balances the node stack. Used by statement-level recovery when the tokens are known to
belong to the current block but the statement parser gave up part way through.
The nodes are closed with their end events ([MarkerEventContainer::end_marks_to]), which is the
whole difference between a local error and a lost block: a node detached without one is balanced by the
tree builder at the end of the stream, so the statement that failed swallows everything written after
it — including the } that closes its own block. That is what bits/stl_map.h pays for:
__glibcxx_function_requires(…) is a macro written without its ;, the expression statement failed, and
the ExpressionStat left behind then took the rest of operator[]’s body and the rest of class map
with it (std::map had no find).
pub fn has_error(&self) -> bool
pub fn get_errors(&self) -> Vec<CppParseError>
Sourcepub fn events(&self) -> &[MarkEvent]
pub fn events(&self) -> &[MarkEvent]
Audit the event stream for balance. Used by tests to assert that a particular input’s recovery left no node dangling.
This is the check that catches the failure mode the marker stack exists to prevent: an
unclosed NodeStart does not make the tree ill-formed, it makes it wrongly nested, and
every token after the leak ends up in the wrong node.
The raw event stream, for debugging the parser’s recovery. Tests assert on it; production code should use the tree.
Sourcepub fn is_open(&self, kind: CppSyntaxKind) -> bool
pub fn is_open(&self, kind: CppSyntaxKind) -> bool
Is a node of this kind open at the cursor — one of the nodes being parsed right now?
CppParser::open_marks is the parser’s own record of what is open: the event positions of the nodes that
have been started and not finished. The positions are private but the events are not, so the question is
answered from both.
It is asked by rules that need to know where they are rather than what they are parsing. A template
argument list is the case it exists for: inside a constraint the < is read one way at the constraint’s
top level (where a declaration follows the whole constraint) and another way inside parentheses (where
the constraint itself continues). See [crate::grammar::cpp::exprs].
An empty node does not count: Marker::complete drops a node with nothing after its NodeStart, which is
the same exclusion CppParser::audit_events makes.
Sourcepub fn audit_events(&self) -> EventStreamAudit
pub fn audit_events(&self) -> EventStreamAudit
Note that a NodeStart with no children legitimately has no matching NodeEnd:
Marker::complete drops empty nodes so the tree does not fill up with zero-width wrappers.
Those are tracked in EventStreamAudit::empty_nodes and excluded from
EventStreamAudit::unclosed — everything left in unclosed is a genuine leak.
Auto Trait Implementations§
impl<'a> !Freeze for CppParser<'a>
impl<'a> !RefUnwindSafe for CppParser<'a>
impl<'a> !Send for CppParser<'a>
impl<'a> !Sync for CppParser<'a>
impl<'a> !UnwindSafe for CppParser<'a>
impl<'a> Forget for CppParser<'a>
impl<'a> Move for CppParser<'a>
impl<'a> Unpin for CppParser<'a>
impl<'a> UnsafeUnpin for CppParser<'a>
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more