1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
//! Prose content: text runs, `{expr}` interpolation, `<>` glue, diverts in
//! content position, and `#`-tags (charter §5/§6/§11).
//!
//! [`content_items_until`] is the shared engine: a loop that recognizes
//! interpolation/glue/the annotated-brace family/diverts and folds
//! everything else into `TEXT` runs, stopping at a caller-chosen set of
//! terminators. Reused by `CONTENT_LINE` (stops at newline), choice-line
//! anatomy (stops at `[`/`]`), and inline alternation bodies (stops at
//! `|`).
use crate;
use Parser;
/// A single line of prose content, terminated by `NEWLINE` or EOF. Never
/// consumes a bare `R_BRACE` (that closes the enclosing body, never the
/// content line itself). May open with a `(label)` — G-1 (RULED
/// 2026-07-20, "label any content line"): extends the choice-line `(name)`
/// label syntax (charter §11) to any content line, giving ink's `-
/// (label)` mid-flow re-entry / backward-loop divert target a native
/// spelling. Trailing `#tag`s are folded in before the line ends.
pub
/// `content_line`'s twin for `family::colon_body`'s per-line dispatch
/// (`family::colon_body_line`, #1254 Gap 1): identical in every respect
/// except the scan also stops at a same-line else-arm boundary
/// (`family::at_else_arm`), so `{if cond: … else: …}` written as one
/// physical line hands control back to `colon_body` instead of the
/// trailing `else: …` getting swallowed into this line's `TEXT`.
pub
/// `(ident)` at the very start of a content line — G-1's label prefix.
/// Guarded by the same positive-lookahead discipline `decl.rs`'s Finding
/// #5 uses for keyword-headed declarations (see that module's doc
/// comment): only the exact `L_PAREN IDENT R_PAREN` shape commits to a
/// `LABEL`; anything else (an empty `()`, a multi-word parenthetical, an
/// unclosed paren) falls through to ordinary prose text, same as before
/// this fix. This does not fully resolve the syntax's inherent ambiguity
/// with a single-word parenthetical remark opening a line (`(Sighing) I
/// trudge on.` is indistinguishable from a real label by construction) —
/// that is the exact tradeoff the choice-line `(name)` syntax already
/// accepts (charter §11); noted as a finding rather than papered over.
/// `(name)` — a label. Shared by `content_line` (G-1) and `choice.rs`'s
/// choice-line label (charter §11, "kept in the one label syntax"): one
/// grammar rule, two call sites.
pub
/// The shared prose-scanning loop: recognizes interpolation/glue/the
/// annotated-brace family/diverts, folds everything else into `TEXT` runs,
/// and stops as soon as the current (trivia-skipped) token is EOF or
/// appears in `stop`. Does not consume the stopping token.
pub
/// [`content_items_until`]'s twin for `family::colon_body`'s per-line
/// dispatch (#1254 Gap 1): identical, except the scan also stops at a
/// same-line else-arm boundary (`family::at_else_arm`) — the two are kept
/// as separate entry points (rather than adding an `else`-boundary kind to
/// every caller's `stop` slice) because `at_else_arm` needs two-token
/// lookahead (`KW_ELSE` immediately followed by `{`/`:`/`if`), not the
/// plain single-token membership test `stop` gives every other kind, and
/// this behavior must stay opt-in: an ordinary content line's bare word
/// "else" (unfollowed by one of those three tokens) is just prose and
/// must keep its leading whitespace like any other `TEXT`-run word, never
/// treated as a structural boundary the way a real else-arm opener is.
pub
/// The shared engine's actual body. `expected_close` is `Some(name)` only
/// when this call is [`super::markup::span`] scanning a span's own body
/// looking for `</name>` — every other caller passes `None`. Forwarding the
/// exact same `stop`/`stop_at_else_arm` a span's caller was given, one
/// level down, into the recursive call `markup::span` makes for its body,
/// is the entire nesting-doctrine enforcement mechanism (see
/// `markup`'s module doc); this function does not need to know that, it
/// only needs to treat a close tag matching `expected_close` as a stop
/// condition like any other.
///
/// `pub(crate)`, not private: `markup::span` (a sibling module) is the one
/// other caller, recursing back in for a span's body.
pub
/// True when the next non-trivia token (`cur`, already trivia-skipped by
/// `current()`) begins a prose `TEXT` run — i.e. it would fall to
/// `content_items_until`'s `_` dispatch arm rather than a structural item
/// (`{…}` family/interpolation, glue, divert, doc-comment) or a stop/break
/// token (`EOF`/`HASH`/a caller-supplied stop). When it does, the outer
/// loop must NOT `skip_ws` first, so `text_run_until` can fold the leading
/// whitespace into the `TEXT` node and preserve significant inter-token
/// prose whitespace. Every branch this returns `false` for either breaks
/// the loop or dispatches to an item whose own `bump`/`expect` (or the
/// explicit `skip_ws` the loop still performs) consumes the leading trivia,
/// so structural lookahead and node-boundary placement are unchanged.
/// True when the `L_BRACE` at the parser's current (unconsumed) position
/// is a genuine body-opener (e.g. a choice's `CHOICE_BODY`) rather than
/// bare `{expr}` interpolation — G-2's disambiguation. Only meaningful
/// when the caller listed `L_BRACE` in its stop set (choice-text scanning,
/// `choice.rs`'s `CHOICE_START_CONTENT`/`CHOICE_INNER_CONTENT`); a plain
/// `content_line` never does, so this is never consulted there — a `{` mid
/// content-line has no competing "body" grammar that could start there, so
/// it's uniformly family-or-interpolation, unchanged from before this fix.
/// Number of raw trivia tokens (`WHITESPACE`/comments) sitting at the
/// parser's current raw position before the next real token — `0` when
/// none are pending. Lets [`is_body_open_brace`] give the same answer
/// whether or not its caller has already `skip_ws`'d.
/// True when the `L_BRACE` at the parser's (possibly trivia-pending)
/// position is going to resolve to a bare `{expr}` interpolation — neither
/// the choice/conditional/alternation family nor (when the caller lists
/// `L_BRACE` as a stop kind) a body-opener. Mirrors exactly the condition
/// under which `content_items_until`'s dispatch match reaches
/// [`interpolation`]; used solely to gate the inter-interpolation
/// whitespace fix, which must never fire for the family/body-open cases —
/// those still need their leading trivia `skip_ws`'d bare, unchanged.
/// `{expr}` — bare-brace interpolation, and nothing else, ever (charter
/// §6).
pub
/// A run of literal text: every raw token up to the next breaking
/// construct (`{`, `<>`, `->` (N-1), a doc-comment token (B0.6b), a
/// caller-supplied stop kind, an else-arm boundary when `stop_at_else_arm`
/// (#1254 Gap 1), or EOF), including any interior whitespace/plain-comments
/// — those are literal prose here, not trivia to discard (so trailing
/// whitespace already absorbed into an in-progress run before one of these
/// breaking constructs — e.g. the space before a trailing `#tag`/`->`/
/// `else:` — stays part of this `TEXT` node; only a FRESH item's leading
/// whitespace is ever trimmed, by the outer loop, before this function is
/// even called). `HASH`/`DIVERT`/doc-comment tokens always break a text run
/// (tags and diverts are recognized structurally by every caller; a
/// doc-comment token must never fold into visible prose), even if the
/// caller didn't ask for it — mirrors `content_items_until`'s own
/// unconditional-break agreement for `HASH`; without this, a `->` (or a
/// stray `//!`) reached mid-run would get bumped as literal text before the
/// outer loop ever saw it, and its dedicated match arm there would be dead
/// code. The else-arm boundary needs the same treatment for the same
/// reason: `family::at_else_arm`'s two-token lookahead only fires reliably
/// when checked from the OUTER loop between items, so a genuine
/// `else:`/`else{`/`else if` reached mid-run must hand control back here
/// too, or it would get swallowed as literal text before that check ever
/// runs (`colon_form_else_on_the_same_line_is_recognized_as_an_else_arm`).
/// `# tag text # another tag\n` — a standalone tag-line body item (charter
/// §11: tags kept).
pub
/// One or more `#`-tags in a row, without a `TAG_LINE` wrapper — used by
/// `tag_line`, `content_line`'s trailing-tags tail, and any other prose
/// loop (e.g. inline alternation bodies) that hits a `HASH` it must
/// consume to keep making forward progress.
pub
/// [`tag_line_tail`]'s twin for a *declaration header* line
/// (`decl::header_tags`, §8b.4's `flow x #tag { … }`): identical, except
/// each tag's text also stops at the body opener that follows it on the
/// same line — a bare `{`, or a `~`/`>` body-dialect selector (charter §4).
/// Without the extra stops the last tag's free-text run would swallow the
/// brace and the header would lose its body.
///
/// The two entry points are kept separate rather than always stopping at
/// `L_BRACE`/`TILDE`/`GT`, because on a *content* line those characters are
/// ordinary tag text with no body opener anywhere in sight, and narrowing
/// the general tag grammar to serve the header case would silently change
/// what an existing `#tag` means.
pub
/// Spec'd in `docs/prose-dialect-spec.md` §4.7.
///
/// #1728: a tag's body is raw text (`tag_text_runs_raw_to_end_of_line`) —
/// it never re-parses `{…}` as a real interpolation/alternation/choice
/// node — but it still must not mistake a `}` that merely *echoes* a `{`
/// the tag's own text already contains for its own terminator. `depth`
/// counts literal, unpaired `{`s bumped so far: a `}` only stops the scan
/// when depth is back to zero, i.e. it isn't balanced by an earlier `{`
/// within this same tag. This is pure raw-character balancing, not
/// interpolation awareness — `#tag {gold` (never closed) still runs to
/// end of line exactly as before.
///
/// The real tradeoff, stated plainly (review of #1728, not "no
/// regression"): a *balanced* brace inside a tag no longer terminates it
/// early (the bug this fixes), but a genuinely *unbalanced*, unescaped `{`
/// left open inside a tag now eats the enclosing single-line block's own
/// same-line `}` closer instead of stopping there — the mirror image of
/// the original bug, inherent to depth-based balancing over raw text with
/// no real grammar to bound it. Pinned by
/// `an_unbalanced_open_brace_in_a_tag_eats_the_enclosing_blocks_own_closer`.
///
/// `\{` is different: #1716/PR #1732 made it the literal-brace escape, so
/// it is text, not a metacharacter, and counting it as a depth-opener
/// would be the surprising reading — a `\{` with no real interpolation
/// nearby would then let a later unescaped `}` swallow the enclosing
/// closer, converting previously clean source into a parse error. So an
/// `L_BRACE` is excluded from the counter when it is preceded by an *odd*
/// number of consecutive raw `BACKSLASH`es (#1852: `\\{` is an escaped
/// backslash followed by a real, depth-counted brace, not an escaped
/// brace — an even backslash count means the backslashes escape each
/// other and the brace stands unescaped) (an `R_BRACE` is still
/// unconditionally significant to the depth check either way — `\}` alone,
/// with no preceding unescaped `{`, already terminated a tag before this
/// fix and still does). Pinned by
/// `a_tag_with_an_escaped_open_brace_does_not_swallow_the_enclosing_blocks_own_closer`.
///
/// **CONFIRMED (issue #1883, item 2): `\}`'s unconditional significance is
/// intentional, not a residual asymmetry to close.** `\{`'s
/// backslash-parity carve-out exists *because* `\{` is one of the ruled,
/// final four-character inline escape set (§8d.6: `\< \{ \# \\`) —
/// #1716/PR #1732 ruled it the literal-brace escape, so a depth counter
/// that treated it as a real opener would be reading a ruled escape
/// wrong. `}` is **not** a member of that set — there is no equivalent
/// "`\}` is a literal, non-metacharacter close-brace" ruling anywhere to
/// protect. So an `R_BRACE` preceded by a `BACKSLASH` is exactly what it
/// looks like: an ordinary backslash character followed by an ordinary,
/// structurally significant `}` — giving it the same backslash-parity
/// carve-out `{` has would *invent* a new escape meaning for a character
/// the ruled grammar has never assigned one to, not close a parity gap.
/// Pinned by
/// `a_tags_own_unescaped_closing_brace_remains_the_terminator_even_when_preceded_by_a_backslash`.
/// See `docs/prose-dialect-spec.md` §4.7b for the durable spec-level home.
///
/// **`\#` escapes the tag-boundary role of `#` (issue #1738).** A bare
/// unescaped `HASH` always ends a tag — that is the `TAG_LINE`/trailing-tag
/// grammar's own separator (`#a#b` is two sibling tags,
/// `tags_with_no_space_between_are_two_separate_tag_nodes`), and this
/// function must keep honoring that for the common case. But `#` is one of
/// the four members of the ruled, final inline escape set (§8d.6: `\< \{ \#
/// \\`), and before this fix `tag()` gave it **zero** escape treatment: a
/// `\#` inside a tag's own text still split the tag in two at the `#`,
/// leaving a dangling, meaningless backslash in the first half — the exact
/// "escape/markup layer coverage inconsistent across prose scanners" gap
/// #1738 tracks (the *ordinary* content-line scanner already turns `\#` into
/// a literal `#` via `markup::escape`'s `ESCAPE` node; `tag()`'s free-text
/// scan didn't). The fix mirrors the *existing* `\{` carve-out immediately
/// below, not `markup::escape`'s node-producing shape: an `HASH` is only
/// treated as the tag's terminator when NOT preceded by an odd number of
/// consecutive raw `BACKSLASH`es — same `backslash_count` parity tracking,
/// same "even means the backslashes escape each other" reading (#1852). Like
/// `\{`, the backslash is **not stripped** from this raw `TAG` node's own
/// CST text — it stays a lossless, unstripped copy of the source, exactly
/// like every other raw-text scanner in this file — this stays
/// self-consistent with `\{`'s established "structural role only" treatment
/// inside these two raw-text scanners, not a claim that `tag()` now runs the
/// full `markup::escape` layer (it still doesn't: `<ident>` stays inert
/// literal text here too, per the separate, already-ruled #1783 "markup is
/// literal in a `#` tag" decision — untouched by this fix).
///
/// **Superseded in part by issue #2045:** the CST-level claim above still
/// holds, but `ast::Tag::text()` (`ast/nodes.rs`) is a *later* materialization
/// point that now strips a recognized escape's backslash from the tag's
/// rendered text (parity with `markup::escape`'s stripping for ordinary
/// content), and `hir::lower_native::body::lower_tag` was changed to funnel
/// through it instead of hand-rolling its own HASH-skip + concatenation —
/// so "not stripped" is only true of the raw CST node, not of every reader
/// of a tag's text. Pinned by
/// `a_tag_with_an_escaped_hash_does_not_end_the_tag_early` (raw CST) and
/// `a_tags_text_accessor_strips_a_recognized_escapes_backslash` (the
/// stripping accessor).
///
/// **RULED (review of #1777, issue #1787): `depth` is scoped per-tag, not
/// per-line, and that is the intended contract, not a gap.** `tag_line_tail`
/// calls this function fresh for each `HASH` it sees, so `depth` always
/// starts at zero for a new tag regardless of what an earlier sibling tag
/// on the same line left unbalanced. In `#a {x #b}` (two trailing tags —
/// `content_line`'s own doc comment: "Trailing `#tag`s are folded in
/// before the line ends", so this shape only ever arises between
/// *sibling* tags, never between a tag and following prose), tag `a`'s
/// scan is cut short by the (unescaped — see the `\#` note just above) `HASH`
/// starting `b` — before the brace-depth check ever runs, exactly
/// like `NEWLINE`/`EOF` — so `a`'s in-progress depth of 1 is simply
/// discarded, not carried into `b`'s scan. `b` starts its own scan at
/// depth zero and immediately meets the `}`, stopping there without
/// consuming it, so that brace is left for whatever the line's own
/// enclosing block expects it to close. Carrying depth across the `HASH`
/// boundary instead (a per-line scope) would make one tag's own unbalanced
/// text reach *through* a syntactically distinct sibling tag and swallow
/// that sibling's — or the enclosing block's — own closer, a strictly
/// worse and less local failure mode than today's already-accepted
/// per-tag tradeoff above: a `HASH` is a real, tokenized boundary (each one
/// starts its own `TAG` node), unlike the raw, grammar-blind `{`/`}`
/// characters this scan balances, so treating it as anything other than a
/// hard reset would blur a structural boundary the CST already treats as
/// absolute. Pinned by
/// `a_tags_own_unbalanced_brace_does_not_leak_depth_into_a_sibling_tag`.