Expand description
The parse pipeline: lex → parse_expr (Pratt) + structural (recursive descent) → events → build_tree → rowan CST.
Losslessness is the core invariant: reconstruct(text) == text for all
inputs. The grammar is a walking skeleton over a Julia subset and grows
incrementally (see TODO.md). Incremental reparse splicing lives in
reparse (a single edit) and reparse_edits (a chain of them); both
return None when no strategy applies, and the host’s salsa layer
(fatou::incremental) then does a full parse.
Structs§
- Edit
- The edit currency, defined in the sibling
editmodule and re-exported here soparser::Editstays the path the parser layer uses. A single contiguous text edit: replacerange(a byte range in the old text) withinsert. - Parse
Diagnostic - A parse-time diagnostic: a classified message anchored to a byte range in the source.
- Parse
Output - The result of a parse: the lossless CST plus any diagnostics.
- Reparsed
- The result of a successful incremental reparse: the new whole-file green tree and its parse diagnostics (absolute offsets in the new text).
Enums§
- Reparse
Tier - Which strategy produced a
Reparsed. Surfaced for tests and benchmarks. Ordered cheapest-first, so a chain of reparses can report the most expensive tier it had to reach withOrd::max.
Constants§
- KEYWORDS
- Every Julia keyword, as written. The single source of truth for the set of
keywords, shared by the lexer’s
keyword_kindclassification and the language server’s keyword completion. A test keeps it in step withkeyword_kind.
Functions§
- apply_
edits - The edit currency, defined in the sibling
editmodule and re-exported here soparser::Editstays the path the parser layer uses. Applyeditstooldleft-to-right, each expressed against the text its predecessors produced (the shape LSPdidChangebatches arrive in). Folds in place, so peak memory is one text rather than one per edit. - diff_
edit - The edit currency, defined in the sibling
editmodule and re-exported here soparser::Editstays the path the parser layer uses. Recover a single contiguousEditfrom a pair of full texts by stripping the common prefix and suffix. Multiple disjoint edits collapse into one spanning edit — still a correct transform, just coarser. Boundaries are clamped to char boundaries of both texts. - fingerprint
- Structural fingerprint of a tree: one line per descendant element with
kind@rangeplus the token text (empty for nodes). Two trees with equal fingerprints are byte-identical. Oracle/debug support: the Tenet-4 assert inreparseand thetests/incremental_reparse.rsharness share this definition so they can never diverge. - is_
ident_ continue - Whether
cmay continue an identifier, mirroring JuliaSyntax’sis_identifier_char(Base.is_id_char). ASCII is handled inline; non-ASCII code points defer to the generatedunicode_identtables. The!=operator split for a trailing!is handled by the caller inscan_ident. - is_
ident_ start - Whether
cmay begin an identifier, mirroring JuliaSyntax’sis_identifier_start_char(Base.is_id_start_char). ASCII is handled inline; non-ASCII code points defer to the generatedunicode_identtables. - normalize_
sexpr - Canonical form of a JuliaSyntax s-expression string. Tokenizes on whitespace
and parentheses (preserving
"…"string literals as atoms) and rejoins with single-space separation, so pretty-print spacing no longer affects equality. - parse
- Parse
textinto a losslessrowanCST. The drive loop works one logical line at a time: leading trivia (including newlines) is emitted directly at the root, then the line’s statements are parsed with the Pratt parser. A line that carries a top-level;groups its statements into aTOPLEVEL_SEMICOLONnode (mirroring JuliaSyntax’stoplevel-;); a plain line stays bare. An unparseable token is consumed as one error token so the loop always makes progress. - reconstruct
- Round-trip the input through the parser: concatenating every token in the CST must reproduce the original text exactly (the losslessness invariant).
- reparse
- Attempt an incremental reparse of the previous parse (
prev_green, parsed fromprev_textwithprev_diags) underedit, which transformsprev_textintonew_text. ReturnsNonewhen no incremental strategy applies — the caller must then do a full parse. - reparse_
edits - Attempt an incremental reparse across a chain of edits, each expressed
against the text its predecessors produced — the shape an LSP
didChangebatch arrives in (seefatou::text::apply_content_changeshost-side). - to_
juliasyntax_ sexpr - Render the given Fatou CST as a JuliaSyntax-native s-expression string.
- try_
apply_ edits - The edit currency, defined in the sibling
editmodule and re-exported here soparser::Editstays the path the parser layer uses.apply_editsfor an edit slice of unproven provenance:Nonewhen any edit’s range does not fit the text its predecessors produced.