# Parser backlog & roadmap
**Single entry point** for open work on `sysml-v2-parser` and the Spec42 diagnostics integration. Historical plans remain as references; this document is updated when items open or close.
**Last updated:** 2026-06-08
## How to use this document
| See **all open work** in one place | This file — sections below |
| Understand **what the parser already ships** for Spec42 | [§ Completed — Spec42 parser wave](#completed--spec42-parser-wave-june-2026) |
| Wire **Spec42 graph builders / collectors** | [§ 1 — Spec42 cross-repo](#1-spec42-cross-repo-follow-up) |
| Improve **editor / LSP** behavior | [§ 3 — Language server](#3-language-server--recovery) |
| Go deeper on **grammar fidelity** | [§ 4 — Grammar & compliance](#4-grammar-depth--compliance) |
| Read the **original Spec42 parser spec** | [SPEC42-DIAGNOSTICS-PARSER-IMPROVEMENTS.md](./SPEC42-DIAGNOSTICS-PARSER-IMPROVEMENTS.md) |
### Regression gates (every parser PR)
- `cargo test`
- `cargo test --test validation -- --include-ignored`
- `test_systems_library_strict_no_diagnostics` / `test_full_library_strict_no_diagnostics` (validation suite)
- `ExtendedLibraryDecl = 0` in library node-shape gates
---
## Status snapshot
| CI & library gates | Green |
| BNF coverage map | 640/640 productions classified `implemented` |
| Spec42 diagnostics **parser AST** (P0–P2 wave) | **Largely done** — see completed table; partial items listed in § 2 |
| Spec42 **semantic** diagnostics | **Open** — graph projection + collectors in Spec42 repo |
| Deep body fidelity | **Open** — many `advance_to_closing_brace` call sites remain |
| Full `OwnedExpression` | **Open** — operator enums added; full KerML expression family not modeled |
| Unified definition/usage grammar layer | **Open** — P5+ architectural work |
```mermaid
flowchart TB
subgraph done [Parser done June 2026]
ast[AST fields + spans]
fixtures[spec42_diagnostics_ast tests]
end
subgraph open_parser [Parser open]
partial[Partial items §2]
bodies[Opaque bodies §2.3]
expr[Expression depth §2.4]
end
subgraph open_spec42 [Spec42 open]
graph[Graph builders]
coll[Diagnostic collectors]
end
ast --> graph --> coll
partial --> graph
```
---
## 1. Spec42 cross-repo follow-up
Parser changes unlock diagnostics only after Spec42 projects new AST fields. **Highest ROI** after the June 2026 parser wave.
| `accept_payload_incompatible` on transitions | Yes — `Transition.accept`, `PayloadClause` | Graph: transition trigger `payloadType`; collector | [DIAGNOSTIC-CHECKS-ROADMAP](https://github.com/spec42/spec42/blob/main/docs/engineering/DIAGNOSTIC-CHECKS-ROADMAP.md) |
| `send_payload_incompatible` | Yes — `ActionUsage.send` | Graph: control-node send payload | same |
| Final-state cardinality (`multiple_final_states`, …) | Yes — `FinalState` member | Graph: final-state edges; drop sink heuristics | same |
| `metadata_keyword_unresolved` (`#Tag`) | Yes — `MetadataKeywordUsage` (simple `#name`) | Walk new node in part/state/requirement builders | [AST-SEMANTIC-COVERAGE](https://github.com/spec42/spec42/blob/main/docs/engineering/AST-SEMANTIC-COVERAGE.md) |
| `viewpoint_reference_unresolved` (stakeholder/purpose) | Yes — `StakeholderMember`, `PurposeMember` | Extend collector for new ref spans | same |
| `viewpoint_rep_language_unresolved` | Partial — `rep` in requirement body | Wire `TextualRep` + `language_span` in graph | same |
| `transition_guard_non_boolean`, filters, assignments | Partial — `BinaryOperator` / `UnaryOperator` | Replace string heuristics with classified expr walk | same |
| `assignment_value_incompatible` in case bodies | Partial — `AttributeDef.name_span` | Wire verification-local `AttributeDef` in graph | same |
| Initial state via `first` | Yes — `Transition.is_initial` | Align with `ThenStmt` initial edges | same |
**Release train:** parser release → Spec42 graph_builder PR → collector + catalog entry → move item **Deferred → Done** in Spec42 roadmap.
---
## 2. Parser — open & partial (Spec42 wave)
Items from [SPEC42-DIAGNOSTICS-PARSER-IMPROVEMENTS.md](./SPEC42-DIAGNOSTICS-PARSER-IMPROVEMENTS.md) that are **not fully closed** in the parser.
### 2.1 Metadata & annotations
| `#keyword` in bodies | **Done** (simple `#Tag;`) | — |
| Extended `#refinement dependency …` | **Done** (opaque `Annotation`) | — |
| User-defined **declaration** keywords (`metadata def` short name as header starter) | **Not started** (deferred 1.5b) | Dynamic dispatch in `feature_decl` / `classifier_decl`; package-local metadata def index |
| `MetadataAnnotation` in all bodies | **Partial** | Part **def**, action, constraint bodies still prefer generic `Annotation` for `@` |
| `head_span` on all annotation usages | **Partial** | Wired on parse paths; not all body enums expose `MetadataAnnotation` variant |
### 2.2 TextualRepresentation (`rep`)
| `rep` in requirement / viewpoint body | **Done** | Fixture: `tests/fixtures/requirement-rep-language.sysml` |
| `rep` in frame, concern, package-adjacent bodies | **Partial** | Package-level `TextualRep` exists; frame/concern may need explicit wiring |
| `language_span` | **Done** on parse path | — |
| Parser diagnostics `missing_rep_language` / `invalid_rep_language` | **Catalog only** | Constants in [`diagnostic_catalog.rs`](../src/parser/diagnostic_catalog.rs); not emitted by `textual_representation()` yet |
### 2.3 Opaque brace-body skipping
**Problem:** Unmodeled inner regions are invisible to Spec42 and the LSP.
| `action.rs` | 7 | High — behavior / control nodes |
| `requirement.rs` | 4 | High |
| `state.rs` | 2 | Medium (transition connect bodies unified) |
| `part/usage.rs` | 3 | Medium |
| `interface.rs`, `connection.rs`, `enumeration.rs` | 3–4 each | Lower |
| `usecase.rs`, `import.rs`, `alias.rs`, … | 1–2 each | Lower |
**Direction:** Per construct family, replace silent skip with `ParseErrorNode` + partial member lists ([LANGUAGE_SERVER_BACKLOG.md](./LANGUAGE_SERVER_BACKLOG.md) P0). One family per PR; track remaining sites here.
### 2.4 Expression AST
| Operator classification | **Done** — `BinaryOperator`, `UnaryOperator` | — |
| Literal / call / member-chain spans for LSP | **Partial** | Some inner spans still `Span::dummy()` |
| `if`, `let`, lambda, classify, type cast | **Not started** | Incremental tranches; not full KerML `OwnedExpression` |
| `select` / `collect` / sequence expressions | **Partial** | Present in parser; callee sometimes folded to `FeatureRef` |
### 2.5 Case & verification bodies
| `AttributeDef.name_span` in case bodies | **Done** | — |
| `value_span` on `AttributeDef` | **Not started** | Optional parity with usages |
| Verdict / return forms, `:>>` in analysis bodies | **Partial** | Many library forms still land in `UseCaseDefBodyElement::Other` |
### 2.6 Parser diagnostic contract
| `diagnostic_catalog.rs` | **Done** (registry file) | Wire constants into `diagnostics.rs` / `recovery.rs` instead of string literals |
| Range-text regression tests | **Partial** | `recovery_diagnostics_integration.rs` exists; add transition/import/type range matrix |
| Scope labels (`"state body"`, …) | **Done** in major bodies | Extend to nested families per § 2.3 |
---
## 3. Language server & recovery
Consolidated from [LANGUAGE_SERVER_BACKLOG.md](./LANGUAGE_SERVER_BACKLOG.md). **Not duplicated** — see that file for narrative detail.
| P0 | Tighten recovery diagnostics (`expected` / `suggestion` precision) | Yes |
| P0 | Expand `ParseErrorNode` to view/constraint/calc nested scopes | Yes |
| P0 | Remove silent reshaping on malformed input | Yes |
| P0 | Recovery tests per construct (codes + ranges + siblings) | Partial — good baseline, gaps in views/constraints |
| P1 | Normalize recovery loops across modules | Partial — `parse_structured_brace_members` exists |
| P1 | Finer grammar-aware sync helpers | Yes |
| P1 | Span robustness under recovery | Yes |
| P2 | Strict vs resilient parse path separation (internal) | Yes |
| P2 | Richer error infrastructure (`nom-supreme`, custom state) | Investigate |
---
## 4. Grammar depth & compliance
Consolidated from [SYSML_V2_COMPLIANCE_GAP.md](./SYSML_V2_COMPLIANCE_GAP.md) and [PARSER_TECHNICAL_DEBT.md](./PARSER_TECHNICAL_DEBT.md).
| Unified definition / usage / specialization grammar layer | **P5+** | Largest architectural gap; do not big-bang rewrite |
| `take_until_terminator` header scraping → structured headers | Medium | Per-family as library fixtures expose gaps |
| `part_def` prelude unify with `definition_prefix` | Low | Intentionally local for disambiguation |
| `package_body_element` sub-dispatchers | **Done** (P2) | Maintain when adding keywords |
| AST shape dedup (`DefinitionDecl` internal) | P5+ | Drive from grammar work |
| Semantic conformance (types, resolution) | Out of scope | Spec42 / other tools |
---
## Completed — Spec42 parser wave (June 2026)
Parser-side delivery for [SPEC42-DIAGNOSTICS-PARSER-IMPROVEMENTS.md](./SPEC42-DIAGNOSTICS-PARSER-IMPROVEMENTS.md).
| P0 §1 | Transition `accept` | `TransitionAccept`, `PayloadClause`, [`payload.rs`](../src/parser/payload.rs) | `transition_accept_retained_with_spans` |
| P0 §2 | Final state | `FinalState`, `StateDefBodyElement::FinalState` | `final_state_members_parsed` |
| P0 §3 | Send payload | `ActionUsage.send`, `control_node_action_usage` | `send_payload_on_control_node_action` |
| P0 §4 | `#keyword` (bodies) | `MetadataKeywordUsage` | `metadata_keyword_usage_in_part_body` |
| P0 §5 | Stakeholder / purpose | `StakeholderMember`, `PurposeMember` | `viewpoint_stakeholder_and_purpose_members` |
| P1 §6 | Expression operators | `BinaryOperator`, `UnaryOperator` | `expression_parses_implies_lower_than_or` |
| P1 §7 | Case attribute spans | `AttributeDef.name_span` | `verification_local_attribute_has_name_span` |
| P1 §8 | `rep` in requirement body | `TextualRep`, `language_span` | `requirement_body_rep_language_parsed` |
| P1 §9 | Annotations | `head_span` on `Annotation` / `MetadataAnnotation`; use-case bodies | apollo_regressions (rationale) |
| P2 §10 | Brace skipping (increment) | Transition uses `connect_body` | — |
| P2 §11 | Diagnostic catalog | [`diagnostic_catalog.rs`](../src/parser/diagnostic_catalog.rs) | `diagnostic_catalog_documents_stable_codes` |
| P2 §12 | `first` = initial | `Transition.is_initial` | `transition_first_sets_is_initial_flag` |
**Fixtures:** [tests/fixtures/](../tests/fixtures/) (`transition-accept-typed.sysml`, `final-state.sysml`, `send-payload.sysml`, `metadata-keyword-usage.sysml`, `viewpoint-stakeholder-purpose.sysml`, `verification-local-attribute.sysml`, `requirement-rep-language.sysml`)
**Integration test file:** [tests/spec42_diagnostics_ast.rs](../tests/spec42_diagnostics_ast.rs)
---
## Completed — technical debt tranches (reference)
| P1 definition prefix + opaque bodies | Complete | [PARSER_DEBT_P1_PLAN.md](./PARSER_DEBT_P1_PLAN.md) |
| P2 structured body loops | Complete | [PARSER_DEBT_P2_PLAN.md](./PARSER_DEBT_P2_PLAN.md) |
| P3 AST split, action/requirement bodies | Complete | [PARSER_DEBT_P3_PLAN.md](./PARSER_DEBT_P3_PLAN.md) |
| P4 view/part bodies, implies, part split | Complete | [PARSER_DEBT_P4_PLAN.md](./PARSER_DEBT_P4_PLAN.md) |
---
## Suggested execution order
1. **Spec42 graph builders** for completed P0 AST (§ 1) — unlocks user-visible diagnostics.
2. **Parser partials** that block Spec42 (§ 2.1 declaration keywords, § 2.2 rep diagnostics, § 2.3 action/requirement bodies).
3. **LSP P0** (§ 3) in parallel with body fidelity.
4. **Expression depth** (§ 2.4) and **P5 grammar layer** (§ 4) as longer horizons.
---
## Document map
| **This file** | Open backlog & roadmap (maintain here) |
| [SPEC42-DIAGNOSTICS-PARSER-IMPROVEMENTS.md](./SPEC42-DIAGNOSTICS-PARSER-IMPROVEMENTS.md) | Spec42-facing parser requirements & fixture index |
| [LANGUAGE_SERVER_BACKLOG.md](./LANGUAGE_SERVER_BACKLOG.md) | LSP/recovery detail |
| [SYSML_V2_COMPLIANCE_GAP.md](./SYSML_V2_COMPLIANCE_GAP.md) | Grammar fidelity narrative |
| [PARSER_TECHNICAL_DEBT.md](./PARSER_TECHNICAL_DEBT.md) | Duplication & architecture notes |
| [BNF_COMPLIANCE_MATRIX.md](./BNF_COMPLIANCE_MATRIX.md) | Compact grammar-family snapshot |
| [ERROR_RECOVERY.md](./ERROR_RECOVERY.md) | Recovery behavior reference |
| PARSER_DEBT_P1–P4_PLAN.md | Completed implementation checklists |