Skip to main content

Module parser

Module parser 

Source
Expand description

The parse pipeline: lex → parse_expr (Pratt) + structural (recursive descent) → events → build_tree → rowan CST.

Losslessness is the core invariant: reconstruct(text) == text for all inputs. Incremental reparse splicing lives in reparse (a single edit) and reparse_edits (a chain of them); both return None when no strategy applies, and the host’s salsa layer (fatou::incremental) then does a full parse.

Structs§

Edit
The edit currency, defined in the sibling edit module and re-exported here so parser::Edit stays the path the parser layer uses. The edit currency, defined in the sibling edit module and re-exported here so parser::Edit stays the path the parser layer uses. The edit currency, defined in the sibling edit module and re-exported here so parser::Edit stays the path the parser layer uses. A single contiguous text edit: replace range (a byte range in the old text) with insert.
InvalidSyntaxEdition
An explicit edition must name a supported edition. Project metadata instead normalizes a Julia version with SyntaxEdition::normalized.
ParseDiagnostic
A parse-time diagnostic: a classified message anchored to a byte range in the source.
ParseOptions
Resolved parsing options. The parser never discovers projects or invokes Julia.
ParseOutput
The result of a parse: the lossless CST plus any diagnostics.
ReparseBase
The parse options and source from which an incremental reparse starts. green and diagnostics must come from parsing text with options.
Reparsed
The result of a successful incremental reparse: the new whole-file green tree and its parse diagnostics (absolute offsets in the new text).

Enums§

ReparseTier
Which strategy produced a Reparsed. Surfaced for tests and benchmarks. Ordered cheapest-first, so a chain of reparses can report the most expensive tier it had to reach with Ord::max.
StringDecodeError
Why a STRING_CONTENT token’s source could not be reduced to a value. The two failures matter separately to the s-expression projector, which shows them differently; a consumer that just wants the value treats both as “this literal does not denote a string”.
SyntaxEdition
A Julia syntax edition supported by this parser. Editions are separate from the runtime versions a package supports.

Constants§

KEYWORDS
Every Julia keyword, as written. Shared by the lexer’s keyword_kind classification and the language server’s keyword completion.

Functions§

apply_edits
The edit currency, defined in the sibling edit module and re-exported here so parser::Edit stays the path the parser layer uses. The edit currency, defined in the sibling edit module and re-exported here so parser::Edit stays the path the parser layer uses. The edit currency, defined in the sibling edit module and re-exported here so parser::Edit stays the path the parser layer uses. Apply edits to old left-to-right, each expressed against the text its predecessors produced (the shape LSP didChange batches arrive in). Folds in place, so peak memory is one text rather than one per edit.
diff_edit
The edit currency, defined in the sibling edit module and re-exported here so parser::Edit stays the path the parser layer uses. The edit currency, defined in the sibling edit module and re-exported here so parser::Edit stays the path the parser layer uses. The edit currency, defined in the sibling edit module and re-exported here so parser::Edit stays the path the parser layer uses. Recover a single contiguous Edit from a pair of full texts by stripping the common prefix and suffix. Multiple disjoint edits collapse into one spanning edit — still a correct transform, just coarser. Boundaries are clamped to char boundaries of both texts.
is_ident_continue
Whether c may continue an identifier, mirroring JuliaSyntax’s is_identifier_char (Base.is_id_char). ASCII is handled inline; non-ASCII code points defer to the generated unicode_ident tables. The != operator split for a trailing ! is handled by the caller in scan_ident.
is_ident_start
Whether c may begin an identifier, mirroring JuliaSyntax’s is_identifier_start_char (Base.is_id_start_char). ASCII is handled inline; non-ASCII code points defer to the generated unicode_ident tables.
normalize_sexpr
Canonical form of a JuliaSyntax s-expression string. Tokenizes on whitespace and parentheses (preserving "…" string literals as atoms) and rejoins with single-space separation, so pretty-print spacing no longer affects equality.
parse
Parse text into a lossless rowan CST. The drive loop works one logical line at a time: leading trivia (including newlines) is emitted directly at the root, then the line’s statements are parsed with the Pratt parser. A line that carries a top-level ; groups its statements into a TOPLEVEL_SEMICOLON node (mirroring JuliaSyntax’s toplevel-;); a plain line stays bare. An unparseable token is consumed as one error token so the loop always makes progress.
parse_with_options
Parse with an explicitly selected syntax edition.
reconstruct
Round-trip the input through the parser: concatenating every token in the CST must reproduce the original text exactly (the losslessness invariant).
reparse
Incrementally reparse a tree produced with the default parsing options.
reparse_edits
Replay edits against a tree produced with the default parsing options.
reparse_edits_with_options
Replay an edit chain under the selected edition. Edition changes require a full parse, even when the source text has not changed.
reparse_with_options
Reparse under the selected edition, declining a base from another edition.
sexpr_tokens
The token sequence normalize_sexpr joins: ( and ) each on their own, "…" string literals whole (escapes honored), and every other run of non-delimiter bytes as one atom.
string_value
The value a STRING_CONTENT token’s source text denotes: escapes decoded and line continuations dropped.
to_juliasyntax_sexpr
Render the given Fatou CST as a JuliaSyntax-native s-expression string.
to_juliasyntax_sexpr_with_options
Project the encoding of a tree parsed with options, including module and @VERSION edition metadata. This never changes the tree’s grammar.
try_apply_edits
The edit currency, defined in the sibling edit module and re-exported here so parser::Edit stays the path the parser layer uses. The edit currency, defined in the sibling edit module and re-exported here so parser::Edit stays the path the parser layer uses. The edit currency, defined in the sibling edit module and re-exported here so parser::Edit stays the path the parser layer uses. apply_edits for an edit slice of unproven provenance: None when any edit’s range does not fit the text its predecessors produced.