# Preserve leading YAML frontmatter while formatting Markdown
This ExecPlan (execution plan) is a living document. The sections `Constraints`,
`Tolerances`, `Risks`, `Progress`, `Surprises & Discoveries`, `Decision Log`,
and `Outcomes & Retrospective` must be kept up to date as work proceeds.
Status: DELIVERED
## Purpose / big picture
After this change, `mdtablefix` must accept a Markdown document that begins
with a YAML frontmatter block and leave that block byte-for-byte unchanged
while continuing to format the Markdown body normally. A user should be able to
run the formatter with flags such as `--wrap`, `--breaks`, or `--in-place` and
still see the opening delimiter, YAML keys, and closing delimiter exactly as
they were written.
The observable success case is a file that starts with:
```plaintext
---
name: weaver
description: Preserve this YAML metadata exactly.
---
```
and ends with the same frontmatter block unchanged after formatting, while the
Markdown that follows is still reflowed, renumbered, or otherwise rewritten
according to the selected options.
## Constraints
- Only a leading document block counts as YAML frontmatter. A `---` line later
in the document must keep its existing Markdown meaning.
- The detected frontmatter block must be copied verbatim, including both
delimiter lines and all interior lines.
- No new external dependencies may be introduced.
- Public CLI flags and existing library entry points in `src/lib.rs` must stay
stable.
- The implementation must keep files below the repository's 400-line limit.
- The change must include both focused unit tests and behavioural CLI tests.
- The user guide must be updated. In this repository that means at minimum
`README.md`, and `docs/architecture.md` should also be updated if the
processing pipeline changes materially.
## Tolerances (exception triggers)
- Scope: if the work requires changes to more than 8 files or roughly 250 net
lines of code, stop and re-evaluate the design.
- Interfaces: if preserving frontmatter requires changing a public function
signature or adding a new CLI flag, stop and escalate.
- Dependencies: if a new crate seems necessary, stop and escalate.
- Ambiguity: if the required delimiter rules are not satisfied by the standard
leading YAML forms (`---` opener with `---` or `...` closer), stop and ask
for clarification before coding.
- Iterations: if the new or updated tests still fail after 3 focused fix
cycles, stop and document the blocker.
- Time: if one milestone takes more than 2 hours, stop and record why.
## Risks
- Risk: frontmatter might still be modified by CLI-only transforms such as
`renumber_lists` or `format_breaks` after the main stream processor returns.
- Severity: high
- Likelihood: medium
- Mitigation: protect the body split at the highest shared pipeline boundary
and add a CLI regression that includes `--breaks`.
- Risk: delimiter detection can become too permissive and accidentally treat a
thematic break or ordinary `---` block as frontmatter.
- Severity: medium
- Likelihood: medium
- Mitigation: only detect frontmatter when the very first line is a
delimiter and require a matching closing delimiter before shielding the
block.
- Risk: `src/process.rs` is already close to the repository's file-length
ceiling.
- Severity: medium
- Likelihood: high
- Mitigation: place the detector and splitter logic in a new small module
instead of extending `src/process.rs` significantly.
## Progress
- [x] (2026-04-05 22:45Z) Reviewed the current processing pipeline, test
layout, and user-facing documentation surfaces.
- [x] (2026-04-09) Add a shared helper for detecting and splitting leading YAML
frontmatter.
- [x] (2026-04-09) Thread the helper through the library and CLI formatting
pipeline
so all transforms skip the frontmatter prefix.
- [x] (2026-04-09) Add unit and behavioural regression tests covering detection,
wrapping, and `--breaks`.
- [x] (2026-04-09) Update `README.md` and `docs/architecture.md`.
- [x] (2026-04-09) Run `make check-fmt`, `make lint`, `make test`,
`make markdownlint`,
and `make nixie` if Mermaid content changes.
## Surprises & discoveries
- Observation: `src/main.rs` applies `renumber_lists` and `format_breaks`
after `process_stream_opts`, so shielding only `process_stream_inner` would
still allow the frontmatter delimiters to be rewritten. Evidence:
`process_lines` in `src/main.rs`. Impact: the plan must protect the body
before or around CLI-only transforms, not just inside `src/process.rs`.
- Observation: `src/process.rs` is 343 lines before this feature.
Evidence: `leta files` output for `src/process.rs`. Impact: new helper logic
should live in its own module to stay within the repository limit and keep
tests readable.
## Decision log
- Decision: use a shared internal splitter for leading YAML frontmatter rather
than adding special cases separately in each transform. Rationale: one
detector keeps the delimiter rules consistent and reduces the chance that a
later pipeline stage mutates the protected prefix. Date/Author: 2026-04-05
22:45Z / Droid.
- Decision: treat unmatched opening delimiters as ordinary Markdown instead of
partially shielding the document. Rationale: this avoids swallowing the
entire file into a special mode and preserves current behaviour for malformed
input. Date/Author: 2026-04-05 22:45Z / Droid.
## Outcomes & retrospective
The frontmatter splitter was successfully implemented in the `frontmatter`
module and integrated through both the `process` module and `main` module. Test
coverage was added covering detection, wrapping, and `--breaks` flags for both
library and CLI paths. All transforms now correctly skip the frontmatter
prefix, preserving the leading YAML block exactly while formatting the Markdown
body.
## Context and orientation
The main formatting pipeline lives in `src/process.rs`. It handles table
reflow, fence tracking, HTML table conversion, heading conversion, wrapping,
ellipsis replacement, and footnote conversion through
`process_stream_inner(lines, opts)`.
The CLI entry point lives in `src/main.rs`. Its `process_lines` function first
calls `process_stream_opts`, then applies ordered-list renumbering and
thematic-break formatting. This is important because the YAML delimiter `---`
looks like a thematic break, so protecting the frontmatter only in
`src/process.rs` is insufficient.
In-place file rewriting is handled by `src/io.rs`, which delegates to the
library functions and therefore benefits automatically once the shared library
pipeline preserves frontmatter correctly.
Focused library tests already live beside the implementation in
`src/process.rs` and `src/io.rs`. Behavioural CLI tests live in `tests/cli.rs`
and `tests/wrap/cli.rs`. The user-facing guide is `README.md`. The processing
pipeline is described in `docs/architecture.md`.
## Plan of work
Stage A is a small, isolated detector module. Add `src/frontmatter.rs` with a
module-level comment and a private helper that splits the input into an
unchanged frontmatter prefix and a Markdown body slice. The helper should only
match when `lines.first()` is exactly the YAML opener and a closing delimiter
is found before the body begins. If no valid closer exists, return an empty
prefix and the original input as the body.
Stage B wires the helper through the library pipeline. Update `src/lib.rs` to
declare the new module. In `src/process.rs`, split the input first, run the
existing processing logic only on the body slice, and then prepend the
unchanged prefix to the processed body. Keep the current ordering of fences,
HTML tables, wrapping, headings, ellipsis, and footnotes for the body.
Stage C wires the same protection through the CLI-only transforms. In
`src/main.rs`, split the original input once in `process_lines`, pass only the
body slice through `process_stream_opts`, `renumber_lists`, and `format_breaks`
as needed, then prepend the original prefix before returning the final lines.
This ensures `--breaks` cannot rewrite the `---` delimiters in the frontmatter
block.
Stage D adds regression coverage. Put detector-specific unit tests in
`src/frontmatter.rs` and a pipeline regression in `src/process.rs` or a small
new test module. Add at least one behavioural CLI test in `tests/cli.rs`
covering a document with leading frontmatter plus a paragraph or table body.
The CLI test should enable `--breaks` and one ordinary formatting option such as
`--wrap` so it proves both preservation and continued formatting.
Stage E updates the docs. Add a short YAML frontmatter note and example to
`README.md` so users know the block is preserved. Update `docs/architecture.md`
to describe the leading-frontmatter split before the rest of the formatting
pipeline.
Each stage ends with focused validation before moving on.
## Concrete steps
Work from the repository root:
```bash
pwd
```
Expected:
```plaintext
/home/leynos/Projects/mdtablefix.worktrees/yaml-frontmatter
```
Add the detector and its focused tests, then run the smallest relevant test set
first:
```bash
cargo test frontmatter --lib
```
Expected:
```plaintext
running <N> tests
test ...frontmatter... ok
```
After wiring the library and CLI paths, run focused regressions:
```bash
cargo test process::tests:: --lib
cargo test --test cli yaml_frontmatter
```
Manually verify the user-visible behaviour with a CLI example:
```bash
printf '%s\n' \
'---' \
'name: weaver' \
'description: short example' \
'---' \
'' \
```
Expected:
```plaintext
---
name: weaver
description: short example
---
```
Finish with repository validators, run sequentially:
```bash
make check-fmt
make lint
make test
make markdownlint
make nixie
```
If `docs/architecture.md` does not change any Mermaid content, `make nixie` may
be skipped.
## Validation and acceptance
Acceptance means all of the following are true:
- A document that starts with a valid YAML frontmatter block keeps that block
exactly unchanged after formatting.
- The Markdown body that follows is still formatted normally, including table
reflow and optional wrapping.
- `--breaks` does not rewrite the frontmatter delimiters.
- The new detector rejects malformed or non-leading cases without changing
existing behaviour elsewhere.
- The README explains the feature clearly enough for a user to discover it.
Quality criteria:
- Tests: the new unit tests and CLI regression tests pass, and `make test`
passes for the full suite.
- Lint: `make lint` passes with no warnings.
- Formatting: `make check-fmt` passes.
- Docs: `make markdownlint` passes, and `make nixie` passes.
## Idempotence and recovery
The planned edits are safe to repeat because the detector only changes control
flow, not persisted state outside the repository. If a step goes wrong, revert
the affected file and rerun the focused tests before continuing. The manual CLI
example is read-only and may be rerun as many times as needed.
## Artifacts and notes
Key repository evidence gathered before implementation:
```plaintext
src/process.rs -> core stream processor, already 343 lines
src/main.rs -> CLI-only post-processing for renumbering and thematic breaks
tests/cli.rs -> behavioural CLI coverage
README.md -> current user guide
docs/architecture.md -> pipeline description
```
The most failure-prone path is `--breaks`, because it can legally rewrite a
plain `---` line outside frontmatter. The tests must therefore include that
flag.
## Interfaces and dependencies
Do not add dependencies.
Add a new internal module at `src/frontmatter.rs` with a helper shaped like:
```rust
pub(crate) mod frontmatter;
```
The helper `split_leading_yaml_frontmatter` returns `(prefix, body)`, where
`prefix` is the untouched leading YAML block, or an empty slice if no valid
block exists. The library keeps the helper internal, and the binary includes
the same source file privately so CLI code can continue to use it without
reopening the public API.
`src/process.rs` calls the helper in `process_stream`,
`process_stream_no_wrap`, and `process_stream_opts` before existing body
processing. `src/main.rs` calls the same helper in `process_lines` before
CLI-only transforms (`--renumber`, `--breaks`).
Interface note: The library now declares `frontmatter` as `pub(crate)` and does
not re-export `split_leading_yaml_frontmatter`. The binary crate (`main.rs`)
remains able to use the helper by including `src/frontmatter.rs` privately with
`#[path = "frontmatter.rs"] mod frontmatter;`. This keeps YAML frontmatter
detection available to internal callers while closing the external library
surface.
Revision note: Delivered. The implementation follows the plan with the
visibility adjustment noted above. All tests pass and the feature is ready for
use.