fatou-parser
A lossless CST parser, typed AST wrappers, and incremental reparser for the Julia language, built on rowan.
This is the parsing engine of fatou, extracted into its own
crate so other tools can embed it. The tree preserves every byte of the input
(including whitespace and comments), parse errors are recoverable diagnostics
rather than failures, and edited buffers can be reparsed incrementally. The
crate builds for wasm32-unknown-unknown: it uses no filesystem, process,
thread, or clock facilities.
Usage
use parse;
let output = parse;
assert!;
assert_eq!;
parser::reparse tries token, nested-block, and top-level statement splices in
that order; ReparseTier::Block identifies a nested body splice. Block
fragments retain the owner's grammar context, including module public
declarations and struct fields. Block bodies reset enclosing bracket markers,
while indexing expressions inside them still parse begin and end normally.
Edits at body edges or in constructs with parser recovery fall back to an
enclosing tier or a full parse. Every successful splice preserves the tree and
diagnostics of parse(edited_text).
Typed AST navigation is available through fatou_parser::ast. Its
DocAttachment view recognizes Julia's ordinary docstrings and exact
two-argument @doc calls. Standard ordinary and raw string payloads can be
decoded without evaluating Julia, with byte ranges mapped in both directions
between the decoded value and source; interpolated and custom payloads remain
opaque.
Decoded documentation can be parsed as Julia Markdown without invoking Julia:
use ;
let source = "# API\n\nSee [`f`](@ref Main.f).\n";
let output = parse;
assert!;
assert_eq!;
fatou_parser::documentation::ast provides typed navigation for Julia Markdown
blocks and inline content. It also classifies core Documenter links, fences, and
jldoctest info strings locally while preserving unknown extension syntax.
Inline constructs span paragraph continuations, including those in lists and
admonitions. Wrapped labels expose Inline::SoftBreak tokens, while inline code
and link destinations join continuations with spaces. The CST retains the
original newlines and container prefixes. A wrapped payload can contain several
content tokens, each with an exact decoded-text range; a range covering the
whole construct also includes the intervening prefixes.
Fences below list items and inside admonitions retain container prefixes as
whitespace trivia. Their CodeBlock::content() joins the CODE_CONTENT tokens
without those prefixes, preserving code indentation; each token has an exact
decoded-text range. Admonitions accept four-space and tab prefixes, including
mixed prefixes within one body. content_range() covers the whole body,
including prefixes between tokens, so consumers that transform the code should
map individual tokens rather than add offsets to that range.
Status
This crate's API is still early and may change between releases; it is versioned
independently of the fatou CLI.
Documentation
- API documentation
- fatou, the project this crate is extracted from