Skip to main content

CppParser

Struct CppParser 

Source
pub struct CppParser<'a> {
    pub parse_config: ParserConfig<'a>,
    /* private fields */
}

Fields§

§parse_config: ParserConfig<'a>

Implementations§

Source§

impl<'a> CppParser<'a>

Source

pub fn parse(text: &'a str, config: ParserConfig<'a>) -> CppSyntaxTree

Parse text into a lossless syntax tree.

This never fails: every input file, however broken or mid-edit, produces a tree covering all of text (invariant I1). Problems are reported through CppSyntaxTree::get_errors and through ErrorNode/MissingNode nodes, not through a Result.

Source

pub fn parse_with_audit( text: &'a str, config: ParserConfig<'a>, ) -> (CppSyntaxTree, EventStreamAudit)

Like CppParser::parse, but also reports the raw event stream’s balance.

Tests use this because the tree alone cannot reveal a recovery bug: an unclosed NodeStart still yields a well-formed tree, just one where a subtree swallowed its following siblings.

Source

pub fn parse_tokens( text: &'a str, tokens: &[CppTokenData], config: ParserConfig<'a>, ) -> CppSyntaxTree

Parse a run of a longer text’s tokens, not the whole of text.

This is what an editor needs: a file’s text arrives as a slice of a stream that was lexed once for something much larger — the stitched, macro-expanded translation unit — and reparsing the whole of it for a keystroke in one file is the wrong work (see docs/parse-architecture.md).

§The contract
  • The tree covers tokens and nothing else. to_source_text gives back the byte span those tokens spell, and a run of every token of text is the same parse CppParser::parse makes of it — same shape, same tokens, same text.
  • Node ranges are relative to the slice, token ranges are not. A CppSyntaxNode’s text_range starts at zero because a green tree is built from token lengths; the absolute anchor is tokens[0].range.start_offset — and every token in CppSyntaxTree::get_tokens keeps the range it was lexed with, so nothing is lost, but a query that maps a node’s range to a file offset has to add the anchor.
  • The grammar sees end-of-input after the last token, because that is the truth about what it was given. A run that ends in the middle of a construct therefore parses as a truncated construct, and the caller’s job is to cut at a boundary it can answer for — which is what a region of a file is.
  • Only the grammar’s errors are reported. The tokens were lexed by the caller, so lexical problems belong to whoever lexed the whole text; this tree’s errors are what the grammar said about this run.
Source

pub fn parse_tokens_with_audit( text: &'a str, tokens: &[CppTokenData], config: ParserConfig<'a>, ) -> (CppSyntaxTree, EventStreamAudit)

CppParser::parse_tokens, plus the event stream’s balance — see CppParser::parse_with_audit for why that is a separate question from the tree’s shape.

Source

pub fn parse_with_events( text: &'a str, config: ParserConfig<'a>, ) -> (CppSyntaxTree, Vec<MarkEvent>)

Like CppParser::parse, but also returns the raw event stream.

The event stream is the parser’s real output and the tree is a fold of it, so when a tree is wrongly nested the stream is where the cause is visible. Kept public because diagnosing a shape bug otherwise means adding a temporary println! inside the parser.

Source

pub fn init(&mut self)

Position the cursor on the first non-trivia token, emitting the leading trivia as events.

Emitting the leading trivia matters: without it the whitespace and comments before the first real token would never be attached to the tree and the CST would silently stop being lossless.

Source

pub fn is_eof(&self) -> bool

Source

pub fn origin_text(&self) -> &'a str

Source

pub fn token_count(&self) -> usize

How many tokens the parser was handed, trivia included.

The bound for a scan that walks the tokens ahead of the cursor by index rather than by peek_token_kind_at, which counts significant tokens and so cannot step past an unknown amount of trivia one position at a time.

Source

pub fn current_token(&self) -> CppTokenKind

Source

pub fn current_token_index(&self) -> usize

Source

pub fn anchor(&self) -> ParseAnchor

Mark the current position for a later question about what has been parsed here.

Two positions describe a moment in a parse and they are not interchangeable: CppParser::open_marks is a length of the marker stack, for closing the nodes opened since, while Checkpoint is where the event stream and cursor stood. Neither can answer a question about tokens already consumed — a marker records an event index and the marker stack is not one — and passing the wrong one is a silent mistake rather than a loud one, because all three are small integers. That is what this type exists to prevent.

Source

pub fn current_token_range(&self) -> SourceRange

Source

pub fn crossed_a_file_boundary_since(&self, since: usize) -> bool

Has the cursor crossed a file boundary since since?

A scope may not cross a file boundary — see ParserConfig::with_file_boundaries. A block’s body loop asks this with the offset its own { (or its own start) sits at, and stops where that file’s text stopped, so that the { a header left open is closed at the end of that header rather than paired with a } written later, or holding every file spliced after it.

§Why it takes the block’s own start

“Is the cursor past some boundary” is not the question: a block that begins inside file n has every token of file n past the boundaries of the files before it, and would stop immediately. What matters is a boundary strictly after the block began and at or before the cursor — a boundary the block has crossed.

[since, at] rather than (since, at] is deliberate. A block that begins exactly on a boundary is a block that begins in a new file, which is the state after a header’s leaked scope was closed: the boundary is the cursor’s own position, and treating it as crossed would close the new file’s block at its own first token.

Source

pub fn current_token_text(&self) -> &str

Source

pub fn checkpoint(&self) -> Checkpoint

Record a checkpoint that CppParser::rollback can restore.

Source

pub fn rollback(&mut self, checkpoint: Checkpoint)

Rewind to checkpoint, discarding every event, token and diagnostic produced since.

Any markers opened after the checkpoint are dropped along with their events, so callers must not hold on to a Marker created inside a speculative region.

§Why the diagnostics go too

A rollback is always “that reading did not fit, try another” — a.b < c > d against a.b<c> d, a cast against a parenthesised expression, an init-declarator against an expression statement. Whatever the discarded reading complained about is not a fact about the file: it is a fact about a guess. Keeping it reports code the user did not write, which is worse than saying nothing — the file that comes out is valid C++ and the editor would underline it. The reading that is kept reports its own problems normally.

Source

pub fn try_parse<T>( &mut self, f: impl FnOnce(&mut Self) -> Option<T>, ) -> Option<T>

Run f speculatively: if it returns None, everything it consumed is rolled back.

This is the primitive that makes C++’s declaration/expression ambiguity tractable without a symbol table.

Source

pub fn try_lex_header_name(&mut self) -> bool

Try to consume a header name at the cursor, folding the <, name and > tokens the ordinary sweep produced into a single CppTokenKind::HeaderName.

Only the parser knows when a header name is expected (#include and friends), and only the parser can rewrite the token stream, so the two halves of header-name lexing live on either side of this call. Returns whether a header name was consumed; on false the caller should lex the token normally.

Source

pub fn events_contain_any( &self, from_event: usize, kinds: &[CppSyntaxKind], ) -> bool

Does any node opened at or after from_event have one of these kinds?

Used by the grammar to answer “what did I just parse?” without threading a return value through every level. from_event is an event index from CppParser::current_event_count.

Source

pub fn joined_operator_at_the_cursor(&self) -> (CppTokenKind, usize)

Source

pub fn current_event_count(&self) -> usize

Number of events recorded so far, for use as a from_event bound.

Source

pub fn brace_balance_since(&self, from_event: usize) -> isize

How many { tokens were consumed at or after from_event without a } to match them?

The question a recovery has to ask about a construct it gave up on. Closing the markers of a failed sub-parse ([MarkerEventContainer::finish_marks_to]) keeps the abandonment from swallowing what follows, but it cannot un-consume the tokens: a member that failed after eating a { — a requires-expression’s body, a function body, a block — leaves a brace behind that the enclosing body then spends its own } on. The class body ends early, every member after it is read at file scope, and the only diagnostic in the file lands on the leftover brace at the end (which is exactly what bits/alloc_traits.h did before the brace debt was paid).

Counted from the events, not the token stream: a rollback truncates the events, so a token read, thrown away and read again counts once — the same reason CppParser::events_contain_any reads them.

Source

pub fn events_contain_any_between( &self, from_event: usize, to_event: usize, kinds: &[CppSyntaxKind], ) -> bool

Has a node of one of these kinds been opened between from_event and the cursor?

The counterpart to CppParser::events_contain_any, which asks about everything from from_event onwards. This one is bounded on both sides, which is what lets a rule ask a question about the declaration it is in the middle of: “has this declarator named anything yet?” is answered by a CppSyntaxKind::NameExpr between where the declarator started and where the cursor stands, and the lower bound is what keeps the type’s own name — a NameExpr inside the specifier sequence, recorded before that point — from being mistaken for the declarator’s.

This matters because the two questions are asked by the same function for two different statements: Widget w(1, 2, 3); names w and must be a declaration, while g(1, 2, 3); names nothing and must stay a call. Both have a name in type position; only the first has one after it.

Source

pub fn greater_than_is_an_operator(&self) -> bool

Does a > at the cursor compare, or does it close the list the cursor is in?

The one question the expression grammar asks about template arguments, and the answer is the standard’s: a > closes the list only when it is not nested in a bracketed group opened since that list began. See CppParser::innermost_template_argument_list for the mechanism and the input it was measured on.

Outside every list the answer is true, which is what keeps a > b an ordinary comparison everywhere else in the file. >= and >>= are never this question’s business — they are different tokens and stay operators either way — and >> is, which is why a >> b inside C<(a >> b)> reads as a shift.

Source

pub fn dialect(&self) -> Dialect

Which compiler’s own reserved spellings mean what — see Dialect.

Read by the grammar in exactly one place (a_type_the_compiler_spells), which is the point: the question this answers is not “how is this file written” but “what does the compiler reading it mean by __int128”, and only the caller knows which compiler that is.

Source

pub fn language(&self) -> LanguageConfig

Which language the file is, as the grammar’s feature switches — see crate::ParserConfig::with_language.

Read where a construct is a language feature rather than a dialect’s spelling: _Generic is C11’s, so a C++17 parse must see the identifier and only a C11-and-later one may read the associations. The lexer already asks the same configuration (crate::lexer::LexerConfig::language), so a rule that asks here cannot disagree with it about what the file is.

Source

pub fn enter_template_arguments(&mut self) -> Option<usize>

Enter a template argument list, recording the bracket depth it began at.

Returns what was in force before, for the caller to hand back to CppParser::leave_template_arguments. Option<usize> rather than usize because the two states are not the same question: None is “no list is open, so every > compares”, while Some(0) is “a list began at the outermost bracket depth”. The version before this one spelled both as the number zero, which is how C<(1 > 2)> came to be read as a mis-nested list rather than as a comparison inside parentheses.

Source

pub fn leave_template_arguments(&mut self, previous: Option<usize>)

Leave a template argument list, restoring what CppParser::enter_template_arguments returned.

Source

pub fn is_in_a_constraint(&self) -> bool

Is the cursor inside a constraint, where a { is not a braced initialiser?

See the field’s documentation. Read by the postfix rule, which is the one that would otherwise take the { after a requires-clause for list-initialisation and swallow the body of the definition.

Source

pub fn enter_constraint(&mut self) -> usize

Enter a constraint. Returns the previous depth so the caller can restore it.

Source

pub fn leave_constraint(&mut self, previous: usize)

Leave a constraint, restoring the depth enter_constraint returned.

Source

pub fn a_template_id_may_be_the_name(&self) -> bool

May the declaration being parsed be named by a bare template-id? See the field’s documentation.

Read by the declarator rule, which is the one that would otherwise refuse the template-id that names an instantiation or a specialization.

Source

pub fn set_a_template_id_may_be_the_name(&mut self, value: bool) -> bool

Record whether this declaration may be named by a bare template-id, handing back the previous answer.

Source

pub fn set_last_declarator_is_function(&mut self, is_function: bool)

Record whether the declarator just parsed declared a function. See the field’s docs.

Source

pub fn last_declarator_is_function(&self) -> bool

Did the declarator parsed most recently declare a function?

Source

pub fn declare_type_name(&mut self, name: &str)

Record that name was declared to be a type, at the current scope depth.

Called by the grammar where a name sits in type position: class Widget, typedef ... Integer, using Alias = .... See TypeNames for why the parser needs this at all.

Source

pub fn is_a_known_type_name(&self, name: &str) -> bool

Is name a type as far as this file’s own declarations say?

The question that separates Widget w(1, 2); from g(1, 2);. A false is not “not a type” but “this file does not say it is one” — a type from an included header, a template parameter, or a builtin this table never saw. Callers must therefore treat it as one signal among several rather than as the answer.

A template parameter counts, and the two sources are kept apart because they are scoped differently: see CppParser::template_parameters.

Source

pub fn declare_template_parameter(&mut self, name: &str)

Record a template parameter that declares a type — typename T, class C.

Called by the template parameter rule, which is the only place that knows which spelling of a parameter this is: int N and auto N declare values, and N[4] is not an array of anything.

Source

pub fn template_parameter_count(&self) -> usize

How many template parameters are in scope, so that CppParser::truncate_template_parameters can cut the list back to a caller’s own state.

Source

pub fn truncate_template_parameters(&mut self, count: usize)

Forget every template parameter recorded since count names were in scope.

The list is appended as heads are read and cut back by the declaration that owns them, which is what keeps a parameter from being a type name for the rest of the file. Nested declarations save and restore around themselves, so a class template’s members still see the class’s parameters: the length the member’s own parse_declaration saved already includes them.

Source

pub fn enter_type_name_scope(&mut self)

Enter a braced body, for the table’s scope approximation.

Source

pub fn leave_type_name_scope(&mut self)

Leave a braced body.

Source

pub fn last_consumed_token_kind(&self) -> Option<CppTokenKind>

The kind of the token the parser consumed most recently, skipping the trivia after it.

The answer to “what did the rule that just ran end on?”, which the cursor cannot give: after a bump the cursor is on the next token, so a rule that has to describe its own last token has to walk back. Used by the specifier sequence, which asks the question of the specifier it just consumed.

None at the start of the file, where nothing has been consumed.

Source

pub fn token_text_at(&self, index: usize) -> &str

The text of the token at index, or empty when there is none.

The companion to CppParser::token_kind_at, for the walks that go backwards over the tokens already consumed — which is how the grammar asks what a declaration began with. peek_token_text_at cannot answer that: it walks forward from the cursor.

Source

pub fn the_name_a_declarator_introduced( &self, from_event: usize, ) -> Option<String>

The unqualified name the declarator that began at from_event introduced, if it named anything.

§Why the event stream and not a return value

A declarator’s name is not always where a rule can see it: typedef WCHAR *PWCHAR; has the name after a *, typedef void (*F)(int); has it inside parentheses, and ns::Widget *w has a qualified spelling whose last component is the name. Threading a Option<String> back through every return of parse_declarator and its six helpers would touch rules that have no interest in the answer, and the grammar already answers “what did I just parse?” from the events in CppParser::events_contain_any_between. This is the same question with the text attached.

§How the name is picked

The first NameExpr at the declarator’s own depth, and then the last identifier inside it. Both halves are load-bearing:

typedef WCHAR *PWCHAR;        the NameExpr is one level in: `PWCHAR`
typedef void (*F)(int);       the declarator's is two levels in: `F`
typedef void (*F)(Widget w);  …and the *parameter's* NameExpr is deeper still, so `w` cannot win
ns::Widget *w;                the NameExpr holds `ns :: Widget`, whose last identifier is the name

A declarator that names nothing — an abstract declarator in a type-id, a structured binding — answers None, which is the honest answer and the one the callers act on.

Source

pub fn type_names(&self) -> &TypeNames

The table itself, for a consumer that wants to audit what the parse recorded.

Source

pub fn remembered_follower(&self, index: usize) -> Option<bool>

What was remembered for CppParser::follower_memo, if anything.

Source

pub fn remember_follower(&self, index: usize, answer: bool)

Remember an answer for CppParser::follower_memo.

Source

pub fn token_range_at(&self, index: usize) -> Option<SourceRange>

The source range of the token at index, or None when there is none.

Beside CppParser::token_text_at for the rules that ask the evidence about a token that is not at the cursor: a macro’s body is in force from an offset, so “what does this name stand for here” needs to know where here is.

Source

pub fn symbol_kind(&self, name: &str) -> Option<SymbolKind>

What the caller’s table says about name, if the caller supplied one.

None is “no evidence from outside” — never “not a type”. See [crate::symbols].

Source

pub fn is_a_macro(&self, name: &str) -> Option<bool>

Is name a macro where the cursor stands? — the caller’s environment, when there is one.

None is “nobody says”, and every rule that asks must fall back to the reading it had before the question could be asked at all. See ParserConfig::is_a_macro_at for what the three answers mean, and CppParser::token_range_at for why the position is part of the question: a macro’s body is in force from an offset, so _STD is a macro below the #include that defines it and an identifier above.

Source

pub fn begin_declaration_type(&mut self)

Start recording the declaration’s leading type name; see the field’s documentation.

The name currently in force is parked rather than dropped, because a speculative region may run this and then be rolled back — see Checkpoint.

Source

pub fn note_declaration_type_name(&mut self, name: String)

Record a name seen in type position, if none has been recorded for this declaration yet.

Only the first: unsigned int has two type words and the declaration’s type is the sequence, not either word — but what the reader needs is only “is the leading name a type this file declared”, and the first name is the one a qualifier would precede.

Source

pub fn note_qualified_declaration_type(&mut self)

Record that the declaration’s type was written as a qualified name.

Called by the specifier sequence as it walks a name’s ::-separated segments. Read once, by the declarator, to tell void ns::C::method() — where the last segment is the function being defined — from Foo(1, 2);, where the name is a callee and the parentheses are an argument list.

Source

pub fn has_qualified_declaration_type_name(&self) -> bool

Was the declaration’s type written as a qualified name? See the field’s documentation.

Source

pub fn declaration_type_record(&self) -> (Option<String>, bool)

The recorded declaration type name, owned, for a caller that has to hold it across a nested parse.

The pair to CppParser::set_declaration_type_record; see that function for what is being saved and why the saving happens at the call site rather than inside the specifier sequence.

Source

pub fn set_declaration_type_record(&mut self, record: (Option<String>, bool))

Put back a record saved by CppParser::declaration_type_record.

A nested parse must not be allowed to answer for the declaration that contains it. A template argument is read with [crate::grammar::cpp::types::parse_type_id], a type-id opens a specifier sequence, and that sequence begins by calling CppParser::begin_declaration_type — which parks the enclosing name and clears the field. Nothing restored it, so G<double> d(…) lost the "G" as soon as its argument was read, and the declarator afterwards found whatever the argument had left: "S" for G<S>, and nothing at all for G<double>, whose argument is a builtin keyword that no name-recording branch covers.

The field answers “did this declaration write a name in type position” — the question that decides whether the ( after the declarator is an initializer or a parameter list ([crate::grammar::cpp::decls::a_declaration_is_the_better_reading]) — so both answers were wrong for a declaration that had written G in plain sight. Measured: G<double> d(static_cast<double>(x)); was refused while S s(static_cast<double>(x)); parsed.

A local, not a parser field. Giving the specifier sequence a matching “end” that restored a shared park slot was tried and is not safe: the slot is written on entry and was only read on the success path, so every sequence that returned an error left it occupied, and the next successful sequence restored a stale name. Fifty headers went from 5 diagnostics to 399, and the parser’s own tests from 21 green binaries to 11. A local around one call has no state to leak, and the error paths need it least — a failed argument fails the whole template-id, and the rollback that follows restores the field anyway.

Source

pub fn enter_class_body(&mut self)

Enter a class body, for the declaration rule that distinguishes a bit-field from a member initializer.

Pushed on a stack rather than counted, because what the rule needs is the innermost brace: see CppParser::is_at_class_member_level. Deliberately not part of Checkpoint: no speculative region opens a brace and then rewinds past it, since the braces it would be rewinding over are its own rather than a guess.

Source

pub fn enter_block_body(&mut self)

Enter a statement or declaration block — a function body, a nested { … }, an extern "C" { … }.

The other half of the stack above: a : inside one of these is never a bit-field’s width, whatever class body encloses it.

Source

pub fn note_declaration_ended_inside_specifiers(&mut self)

Record that the specifier sequence finished a whole declaration — which only friend does.

Called by the specifier rule for friend, whose payload is the declaration that follows it, including its ;. See the field’s documentation.

Source

pub fn take_declaration_ended_inside_specifiers(&mut self) -> bool

Take the flag above: was the statement already over when the specifiers ended?

A take rather than a read, so the answer cannot leak into the next declaration: the flag describes the sequence just parsed, and the caller that asks is the one that parses the declaration around it.

Source

pub fn note_the_terminator_came_from_a_branch(&mut self)

Record that this rule consumed the ; that ends the declaration, from inside a conditional branch.

See the field’s documentation: the readers that call this are the ones that read a construct’s tail once per branch, and the ; each branch writes is part of the tail they read.

Source

pub fn take_the_terminator_came_from_a_branch(&mut self) -> bool

Take the flag above: did the construct just parsed already consume the declaration’s ;?

A take for the reason every flag here is: the answer describes one construct, and the caller that asks (the rule that would otherwise require a ;) is the one that parses around it.

Source

pub fn open_conditionals(&self) -> isize

How many conditionals are open at the cursor, as far as the directives read so far say.

A construct whose tail is written once per branch has to tell the #else that closes a conditional it opened itself from the #else of a conditional opened around it — the two spellings are otherwise identical (void f(int a); / #else / void g(long a); reads as parameters just as well as a per-branch tail does). Counting the directives a construct read does not answer that: a nested reading may have consumed one on its way past, which is exactly what happens in type_traits:2269, where the expression grammar’s operator-position seam takes the # if before the conditional can see it.

So the count is kept by the directive reader itself, and a construct records the value when it starts and compares it where the question is asked.

Source

pub fn note_a_directive_name(&mut self)

Record the directive name at the cursor, for CppParser::open_conditionals.

Called by the directive rule, which is the one place a directive is read — including the ones read by rules that are not that rule’s caller. It is called with the cursor on the name (not on the #), and only when the directive has one: the null directive (# alone on a line) names nothing.

Source

pub fn leave_class_body(&mut self)

Leave a class body.

Source

pub fn leave_block_body(&mut self)

Leave a statement or declaration block.

Source

pub fn is_at_class_member_level(&self) -> bool

Is the cursor at the member level of a class body — the innermost brace being the class’s own?

The question finish_init_declarator puts to it: int bits : 3; and S() : a(1) {} have a : in the same position, and only the enclosing construct says which they are — a member declarator’s : introduces a width, while a function declarator’s introduces the member-initializer list of a constructor. The function case is decided first and does not need this; what does is the member that names no function.

The innermost brace is what matters, not “is a class body somewhere above”. Asking the weaker question meant that everything inside a member function’s body counted as member level, which turned for (auto &v: vec) in a header’s inline member into a bit-field of width vec — and the same for a label (again:) or any other statement whose first name a declaration reading could take.

Source

pub fn declaration_type_name(&self) -> Option<&str>

The declaration’s leading type name, when it has one.

Source

pub fn note_class_head_name(&mut self, name: &str)

The name of the class body the cursor is in, when it is in one and that class has a name.

What a constructor’s name has to equal: \Widget(int);\ inside \class Widget\ is a constructor, and the same two tokens inside \class Other\ are a member function whose return type was written somewhere the parser could not see it (an #if\ branch, a macro). Measured: without this the second shape was read as the first. Record the name a class head declared, for the body that follows it.

Source

pub fn current_class_name(&self) -> Option<&str>

Source

pub fn has_declaration_type_name(&self) -> bool

Has a name been recorded in type position for the declaration being parsed?

The weaker form of CppParser::declaration_type_name, for a caller that only needs to know whether the specifier sequence saw a name at all — a declarator left without one is then not an abstract declarator but the mark of a name that was taken for a type.

Source

pub fn is_inside_a_body(&self) -> bool

Is the cursor inside a braced body?

The signal a_declaration_is_the_better_reading uses: a function declaration inside a function body is vanishingly rare, while a local variable with constructor arguments is everywhere, so a known type name inside a body is read as a declaration.

A parser depth rather than a C++ scope — a class body counts the same as a function body. That is the approximation TypeNames documents, and it is on the safe side here: what the answer licenses is the declaration reading, which is the cheaper of the two mistakes.

What it must not count is a brace that is not a body: a linkage specification’s block (extern "C++" { … }) introduces no scope for names at all — see parse_linkage_block — so the depth does not move inside one, and the rules that ask “am I inside a body” (the macro-from-a-header rules of stats.rs and decls.rs) keep answering no there. Counting it was a bug with wide reach: most of libstdc++ is written inside extern "C++" { namespace std { … } }, so _GLIBCXX_BEGIN_NAMESPACE_VERSION was read as a name instead of as the macro it is, and the declaration behind it came out as an expression (expected ; after expression at using ::wint_t; — the first error of cwchar, cstdlib and every other header with that shape).

Source

pub fn is_at_file_scope(&self) -> bool

Is the cursor at file scope, outside every braced body?

Named separately from CppParser::is_inside_a_body because callers read better for it, and because the two together are exhaustive: a statement is either in a body or not.

Source

pub fn peek_token_range_at(&self, offset: usize) -> SourceRange

Source range of the significant token at relative offset offset from the cursor.

The third of the peek_token_*_at family, and it exists for the questions that are about the source rather than about the tokens: whether two things are written together or apart. operator T&&() and operator bool() && are the same three tokens in the same order, and only the spacing separates the type from the ref-qualifier.

A zero-width range at the end of input, mirroring CppParser::current_token_range, so a caller comparing offsets never has to handle absence separately.

Source

pub fn peek_token_text_at(&self, offset: usize) -> &str

Text of the significant token at relative offset offset from the cursor. Empty past the end.

The companion to CppParser::peek_token_kind_at, needed because module and import are contextual keywords: they must be recognised by spelling, not by kind.

Source

pub fn token_kind_at(&self, index: usize) -> CppTokenKind

Kind of the token at index, ignoring trivia. None past the end.

Source

pub fn bump(&mut self)

Source

pub fn peek_next_token(&self) -> CppTokenKind

Source

pub fn peek_token_kind_at(&self, range: Range<usize>) -> Vec<CppTokenKind>

Kinds of the next range.len() significant tokens, starting at the cursor.

Trivia is skipped, so this is “the next few things the grammar will see”. Shorter than range near end of input, and padded with CppTokenKind::None so callers can index it without a length check.

§Comparing the result

This returns a Vec, and Vec<T> == [T; N] is always false in Rust — there is no PartialEq<[T; N]> impl. A comparison written as peek_token_kind_at(1..2) == [X] therefore compiles and silently never matches, which is exactly the kind of bug that reads as “the branch is dead” rather than “the comparison is wrong”. Call .as_slice() or use the single-token helpers (CppParser::peek_next_token, CppParser::peek_token_text_at).

Source

pub fn push_error(&mut self, err: CppParseError)

Source

pub fn consume_current_token(&mut self)

Emit the current token and move on, without attaching the trivia that follows it.

bump is the right call almost everywhere: trivia belongs to the construct it sits inside. A grammar rule that wants to close its node before that trivia is emitted — because the trivia belongs to the enclosing construct — advances with this and then calls CppParser::emit_trivia_after_current_token.

Source

pub fn consume_current_token_if(&mut self, kind: CppTokenKind) -> bool

Consume the current token with CppParser::consume_current_token if it has this kind.

Returns whether it was consumed. Used for tokens a rule requires but wants to treat as optional at the point of consumption, so that a missing one is reported by the caller rather than aborting a construct that is otherwise complete.

Source

pub fn emit_trivia_after_current_token(&mut self)

Emit the trivia the cursor is sitting on, without consuming any significant token.

The counterpart to CppParser::consume_current_token: after a rule has closed its node, the layout it left behind still has to reach the tree — the CST is lossless — and this is what puts it into whichever node is open now.

Source

pub fn emit_layout_after_current_token(&mut self)

Emit the layout the cursor is sitting on and stop at the first comment.

§Why a rule wants this instead of bump

bump attaches everything after a token to whichever node is open, and a rule that is about to close its node is still open — so a comment written below the construct it just read becomes a child of that construct. Measured: struct S { int x; /// The y. int y; }; made /// The y. a child of the declaration of x, and the module note in tests/doc_owner.rs says what the shape should be — the comment between two members belongs to the class body, where a reader sees it.

Consuming only the layout and stopping short of the comment is what puts it there: the caller completes its node, and the comment is emitted afterwards into the node that encloses it both.

§The layout still has to be emitted

Stopping at a comment is not the same as stopping at all trivia. A version that consumed the ; and emitted nothing stranded the layout entirely: five tests in tests/ast.rs failed on the tree no longer reproducing the file, because the newline after a } was never written to the tree by anything. Whitespace and line breaks go in here, as they did before; only the comment is left for later.

Source

pub fn emit_missing_node(&mut self)

Emit a zero-width MissingNode, i.e. “a token was expected here but is not present”.

Zero-width nodes cost nothing in the tree and are what make completion work at a broken position: the cursor is inside a node of the expected kind rather than in an error blob.

Source

pub fn open_marks(&self) -> usize

Snapshot the set of currently open nodes. Pass the result to CppParser::close_marks_above in every ?-using grammar function’s error path.

The snapshot is a count rather than a depth, and it is only meaningful as long as the stack below it is untouched: closing the nodes above it is driven by the open-node stack itself, so a rule that already closed some of its own nodes cannot confuse it.

Source

pub fn close_marks_above(&mut self, base: usize)

Close every node opened after CppParser::open_marks was snapshotted, as if its closing token had been present.

Call this on every early return from a grammar function. A ? return leaves markers open, and an open NodeStart sits before the ancestor that eventually closes, so all following tokens would be swallowed into it — a silent corruption of the whole rest of the file rather than a local error.

Source

pub fn recover_to_level(&mut self, base: usize)

Close any node opened after base, keeping the consumed tokens in the tree.

Unlike CppParser::rollback, which erases events, this keeps the text and only re-balances the node stack. Used by statement-level recovery when the tokens are known to belong to the current block but the statement parser gave up part way through.

The nodes are closed with their end events ([MarkerEventContainer::end_marks_to]), which is the whole difference between a local error and a lost block: a node detached without one is balanced by the tree builder at the end of the stream, so the statement that failed swallows everything written after it — including the } that closes its own block. That is what bits/stl_map.h pays for: __glibcxx_function_requires(…) is a macro written without its ;, the expression statement failed, and the ExpressionStat left behind then took the rest of operator[]’s body and the rest of class map with it (std::map had no find).

Source

pub fn has_error(&self) -> bool

Source

pub fn get_errors(&self) -> Vec<CppParseError>

Source

pub fn events(&self) -> &[MarkEvent]

Audit the event stream for balance. Used by tests to assert that a particular input’s recovery left no node dangling.

This is the check that catches the failure mode the marker stack exists to prevent: an unclosed NodeStart does not make the tree ill-formed, it makes it wrongly nested, and every token after the leak ends up in the wrong node.

The raw event stream, for debugging the parser’s recovery. Tests assert on it; production code should use the tree.

Source

pub fn is_open(&self, kind: CppSyntaxKind) -> bool

Is a node of this kind open at the cursor — one of the nodes being parsed right now?

CppParser::open_marks is the parser’s own record of what is open: the event positions of the nodes that have been started and not finished. The positions are private but the events are not, so the question is answered from both.

It is asked by rules that need to know where they are rather than what they are parsing. A template argument list is the case it exists for: inside a constraint the < is read one way at the constraint’s top level (where a declaration follows the whole constraint) and another way inside parentheses (where the constraint itself continues). See [crate::grammar::cpp::exprs].

An empty node does not count: Marker::complete drops a node with nothing after its NodeStart, which is the same exclusion CppParser::audit_events makes.

Source

pub fn audit_events(&self) -> EventStreamAudit

Note that a NodeStart with no children legitimately has no matching NodeEnd: Marker::complete drops empty nodes so the tree does not fill up with zero-width wrappers. Those are tracked in EventStreamAudit::empty_nodes and excluded from EventStreamAudit::unclosed — everything left in unclosed is a genuine leak.

Auto Trait Implementations§

§

impl<'a> !Freeze for CppParser<'a>

§

impl<'a> !RefUnwindSafe for CppParser<'a>

§

impl<'a> !Send for CppParser<'a>

§

impl<'a> !Sync for CppParser<'a>

§

impl<'a> !UnwindSafe for CppParser<'a>

§

impl<'a> Forget for CppParser<'a>

§

impl<'a> Move for CppParser<'a>

§

impl<'a> Unpin for CppParser<'a>

§

impl<'a> UnsafeUnpin for CppParser<'a>

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.