# Markdown Parser Architecture
This crate implements a Markdown state machine that tokenizes source, resolves constructs, and can either compile events directly to HTML or build an mdast syntax tree.
## Main paths
- `to_html` and `to_html_with_options` stream parser events into the built-in HTML compiler. This path does not construct an AST.
- `to_mdast` builds a `mdast::Node` tree. The `Node` enum includes a transparent `Fragment` variant for detached child lists.
- `Renderer` is the AST-to-output interface. `HtmlRenderer` and `MarkdownRenderer` are built in; a custom renderer controls its own output and escaping behavior.
- AST helpers include traversal, `text_content`, code-fence metadata, `extract_children`, and heading lookup and section operations. Keep these helpers working for fragments as well as document nodes.
The Markdown serializer lives in `src/markdown/`. Keep its construct handlers and safety logic separate from the parser's event-to-HTML compiler. Serializer changes should preserve references, extensions, and formatting options when possible.
`MarkdownOptions::line_wrapping` controls soft source line breaks in text. `Preserve` is the default and keeps round-trip behavior; `Unwrap` joins those breaks with spaces. Explicit hard-break nodes, code, and block boundaries are not unwrapped. The serializer does not currently enforce a line width.
## Code fences
`Node::code_fence()` returns `Option<CodeFence>` with `character`, `length`, and `indent`, following Markly's `Node::Fence` structure at [436de44](https://github.com/socketry/markly/blob/436de44bebc5aeeb3684cf8ac6e625fbb3fb6e7b/ext/markly/markly.c). The corresponding regression examples are in [`test/markly/node.rb`](https://github.com/socketry/markly/blob/436de44bebc5aeeb3684cf8ac6e625fbb3fb6e7b/test/markly/node.rb).
Ordinary `Code` nodes retain their opening fence in `fence: Option<CodeFence>` independently of source positions. Indented code and manually constructed or older deserialized code without metadata use `None`. Fence length describes the opening sequence even if the closer is longer or absent. Indentation is measured in columns relative to the container, excluding list and blockquote prefixes; tabs, including partially consumed container tabs, contribute their expanded width. Markdown serialization continues to choose ordinary code formatting from `MarkdownOptions`, so this metadata describes the source rather than the rendered output.
The Rust helper also exposes frontmatter fences: generic nodes use their existing marker and opening length; legacy YAML/TOML use their fixed three-character, unindented delimiter. Inline code and other nodes return `None`. Keep fence information separate from language and info strings.
## Frontmatter
`Constructs::frontmatter` recognizes closed fences only at the beginning of a document, outside containers and without opening indentation. Untagged `---` and `+++` retain the inherited `Yaml` and `Toml` nodes. Tagged delimiter forms (including `---yaml`) and language-tagged backtick or tilde fences use `Frontmatter`.
The generic node follows Markly/cmarkly's format-agnostic model: `info` is the entire opaque info string, trimmed only at its outer spaces and tabs; `value` is the raw body, including its trailing line ending. Neither is decoded or interpreted. `language()` and `Node::code_language()` return the first word, while `Node::code_info()` returns the full info string. Opening and closing fence lengths are retained. Closing code fences follow the usual marker, minimum length, and indentation rules; the Rust extension requires a closing fence for every form.
Both HTML paths omit frontmatter. Markdown serialization preserves info and body while normalizing fence spacing. It lengthens code fences when edited content could close them, and switches conflicting dash/plus delimiters to tildes. Edited bodies without a trailing line ending gain one before the closing fence. Keep these safety checks in the Markdown handler, separate from the public AST type.
## Tests and generated data
Integration tests are organized by Markdown construct in `tests/`; keep regressions close to the behavior they cover. `tests/commonmark.rs` exercises the CommonMark corpus, and `tests/html_renderer.rs` and `tests/markdown_renderer.rs` cover AST rendering. `tests/markdown_serializer_*.rs` contains the Markdown serializer corpus.
Run formatting, linting, and tests from the workspace root. The standard test workflow requires 100% source-region coverage for the published `socketry-markdown` package across all targets and features, then runs the workspace test task. This runs the package's tests again with default features and covers the private Bake and generator packages, which are outside the coverage gate. The crate exposes optional `json`, `log`, and `serde` features. The `generate/` program refreshes generated CommonMark and Unicode data used by the tests.