mdtablefix 0.5.0

`mdtablefix` unb0rks and reflows Markdown tables so that each column has a uniform width. When the `--wrap` option is used, it also wraps paragraphs and list items to 80 columns.
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
# Architecture

## Contents

- [Markdown stream processor]#markdown-stream-processor
- [Table reflow pipeline]#table-reflow-pipeline
- [Footnote conversion]#footnote-conversion
- [HTML table support]#html-table-support-in-mdtablefix
- [Module relationships]#module-relationships
- [Concurrency with `rayon`]#concurrency-with-rayon
- [Unicode width handling]#unicode-width-handling

## Markdown stream processor

`process_stream_inner` orchestrates line-by-line rewriting. The full
implementation lives in [src/process.rs](../src/process.rs). Its signature is:

```rust
pub fn process_stream_inner(lines: &[String], opts: Options) -> Vec<String>
```

The public stream entry points call `process_with_frontmatter` before invoking
this function. It is the canonical boundary for leading YAML frontmatter: it
passes only the post-frontmatter body to a caller-provided closure and restores
the prefix verbatim after the closure returns. The library pipeline and
`process_lines` in the binary both use this boundary, with CLI-only transforms
such as `renumber_lists` and `format_breaks` inside the binary's closure.

The function combines several helpers documented in `docs/`:

- `frontmatter::split_leading_yaml_frontmatter` detects a valid leading YAML
  frontmatter block for `process_with_frontmatter`. A block starts with `---`
  on the first line and ends with `---` or `...` before any body content.
- `fences::compress_fences` and `attach_orphan_specifiers` normalize code block
  delimiters. Fence normalization uses the same `FenceTracker` semantics as
  wrapping, so fence-like lines inside an already open fenced block remain
  literal content. Outer delimiters are only compressed when doing so cannot
  make a nested literal fence look structural. `attach_orphan_specifiers`
  preserves and propagates indentation from the language line when the target
  fence lacks indentation. Language specifiers explicitly set to `null`
  (case-insensitive) or consisting solely of whitespace are treated as absent.
  `compress_fences` also tolerates spaces within comma-separated specifiers,
  e.g. `TOML, Ini` becomes `toml,ini`.
- `html::convert_html_tables` transforms basic HTML tables into Markdown so \
  they can be reflowed like regular tables. See \
  [HTML table support]#html-table-support-in-mdtablefix.
- `wrap::wrap_text` applies optional line wrapping. It classifies Markdown
  block structure locally and delegates greedy line fitting to the `textwrap`
  crate over Markdown-aware fragments measured with `unicode-width`.
- `wrap::tokenize_markdown` emits `Token` values for custom processing.
- `headings::convert_setext_headings` rewrites Setext headings with underline
  markers into ATX headings when the CLI `--headings` flag is provided. The
  underline must contain at least three identical `=` or `-` characters, so the
  converter can distinguish headings from thematic breaks or list markers. The
  helper only rewrites lines whose shared prefix is whitespace or `>` so nested
  lists continue to behave normally.

Heading conversion runs after fence/table processing and before wrapping, so
the wrapping stage observes ATX headings and leaves them untouched.

The function maintains a small state machine that tracks whether it is inside a
Markdown table, an HTML table, or a fenced code block. The state determines how
incoming lines are buffered or emitted. Once the end of a table or fence is
reached, buffered lines are flushed and possibly reformatted. The simplified
behaviour is illustrated below.

State-machine abstraction options for this and the wrapping continuation
machines are evaluated in
[Architecture Decision Record (ADR) 0004](adrs/0004-state-machine-abstractions.md).

```mermaid
stateDiagram-v2

    [*] --> Streaming: Start

    Streaming: Default state—processing lines individually

    InMarkdownTable: Buffering lines of a Markdown table

    InHtmlTable: Buffering lines of an HTML table

    InCodeFence: Passing through lines within a fenced code block

    Streaming --> InMarkdownTable: Line starts with "|"
    Streaming --> InHtmlTable: Line contains table HTML tag
    Streaming --> InCodeFence: Line is a fence delimiter ("```" or "~~~")

    InMarkdownTable --> Streaming: Flush buffer and reflow table on non-table line (e.g., blank, heading)
    InMarkdownTable --> InMarkdownTable: Line contains "|" or separator pattern

    InHtmlTable --> Streaming: Flush buffer and convert table on final table HTML closing tag
    InHtmlTable --> InHtmlTable: Line inside table tag

    InCodeFence --> Streaming: Line is a fence delimiter
```

Before:

```markdown
|A|B|
|---|---|
|1|22|
<table><tr><td>3</td><td>4</td></tr></table>
```

After:

```markdown
| A | B  |
| --- | --- |
| 1 | 22 |
| 3 | 4  |
```

Code fences are passed through verbatim:

```rust
| not | a | table |
```

After scanning all lines, the processor performs optional post-processing steps
such as ellipsis replacement and footnote conversion. See \
[footnote conversion](#footnote-conversion) for details. The function then
returns the updated stream for writing to disk or further manipulation.

## Table reflow pipeline

`reflow_table` aligns Markdown tables in four stages:

1. `extract_indent_and_trim` records any leading indentation and removes table
   escape lines such as `\-`.
2. `parse_rows` preserves physical source-line boundaries. When a row starts
   with empty cells, `protect_leading_empty_cells` replaces those cells with a
   private marker. The parser uses the inferred table width to recover only
   complete legacy rows concatenated on one line, so row boundaries remain
   structural and cannot collide with cell data.
3. `clean_rows`, `detect_separator`, and `calculate_widths` rebuild the logical
   table. Explicit separator lines are preferred, but the second parsed row can
   be promoted when the source embeds the separator in the body. Widths are
   measured with `UnicodeWidthStr::width`, so Chinese, Japanese, and Korean
   (CJK) text, emoji, and accented characters align by display width rather
   than byte count.
4. `format_rows` and `insert_separator` emit the final table. Separator cells
   preserve alignment markers, and each separator column is widened to at least
   three dashes to keep Markdown linters satisfied.

Continuation-row protection has one extra constraint: once the protected row is
rebuilt, literal pipe characters inside the non-leading cells are re-escaped as
`\|`. Without that step, a second parse would treat the restored pipe as a new
column delimiter and split the row incorrectly.

When `process_stream_inner` flushes a buffered table with `Options::ellipsis`
enabled, it applies ellipsis replacement before calling `reflow_table`. This
ordering ensures the width calculation sees the final glyphs, rather than
aligning for `...` and shrinking the rendered column after the fact.

Outside table buffering, `replace_ellipsis` maintains fence and indented-code
state while it walks the original lines. Its private indented-code tracker is
owned solely by the ellipsis pass: it preserves top-level code blocks that
begin at the document boundary or after a blank line, heading, closed fenced
block, link reference definition, or markdownlint directive. It also preserves
interior blank lines without changing the public tokenizer's token variants.
Lines classified as code are forwarded byte-for-byte, while prose lines
continue through the shared inline tokenizer, so code spans remain opaque.

Within prose tokens, the ellipsis-owned literal-region scanner protects inline
links and images, autolinks, bare URLs, and filesystem-like tokens. It reuses
the wrapping parser's balanced link-destination boundaries, then returns merged
source ranges, so protected bytes are copied directly and only the intervening
prose is normalized. Link reference definition lines are classified by the
shared block classifier and preserved in full. The pass also uses the wrapping
parser's `LinkTitleWindow` state machine to preserve split destination and
title continuations. The scanner is private to the ellipsis feature; other
transforms may reuse the balanced link span helper but must define their own
literal-region policy rather than inheriting typography rules implicitly. The
rationale is recorded in [ADR 0005](adrs/0005-ellipsis-literal-regions.md).

`ProcessBuffer` owns the active table run during stream processing. It flushes
that buffer before lines that open a new Markdown block, including blockquote,
list-item, link-reference, and footnote-definition lines that themselves
contain pipe characters. Those block-opening lines are then handled by the
ordinary block pipeline rather than being absorbed as continuation rows.

The rationale for these choices is captured in
[Architecture Decision Record (ADR) 0001](adrs/0001-table-reflow-pipeline.md).

## Footnote Conversion

`mdtablefix` can optionally convert bare numeric references into
GitHub-flavoured Markdown footnotes. A bare numeric reference is a number that
appears after punctuation or before a colon with no footnote formatting, for
example:

```markdown
An example of a bare numeric reference.1
The official docs page showcases several types 7:
```

`convert_footnotes` performs this operation and is exposed via the higher-level
`process_stream_opts` helper. Set
`Options { footnotes: true, ..Default::default() }` when calling
`process_stream_opts` to enable the conversion logic. The parameter defaults to
`false`.

Inline references that appear after punctuation or before a colon are rewritten
as footnote links.

Before:

```markdown
A useful tip.1
Core types 7:
```

After:

```markdown
A useful tip.[^1]
Core types[^7]:
```

Numbers inside inline code or parentheses are ignored.

ATX heading lines (including those nested in blockquotes and list items) are
not processed for footnote conversion, so identifiers like "A.2" remain
verbatim. Setext-style headings are not detected unless the `--headings`
conversion already rewrote them into ATX headings earlier in the pipeline.

Before:

```markdown
Look at `code 1` for details.
Refer to equation (1) for context.
```

After:

```markdown
Look at `code 1` for details.
Refer to equation (1) for context.
```

When the final lines of a document form a numbered list that is preceded by an
H2 heading (the heading text is not inspected), and the document contains no
existing footnote definitions outside fenced code blocks, they are replaced
with footnote definitions. Blank lines and indentation within the list are
tolerated; blank-only trailing blocks are ignored. Lines beginning with inline
footnote references at the start of a line do not count as existing
definitions, allowing references before the final footnote definition block.
Definitions prefixed by blockquote markers (`>`) still count as existing
blocks, but those inside fenced code blocks are ignored.

Once inline references and trailing lists are normalized, `renumber_footnotes`
walks the document in the order readers encounter references. It assigns
sequential identifiers starting from one, rewrites every reference to use its
new identifier, and updates footnote definitions to match. Trailing numeric
lists are converted into definitions when the document already contains at
least one footnote reference or definition, ensuring unrelated lists are left
untouched. The rewritten definitions are then sorted numerically so the
rendered footnote block mirrors the logical ordering of references in the text.

Before:

```markdown
Text.

## Footnotes

 1. First note

 2. Second note

10. Final note
```

After:

```markdown
Text.

## Footnotes

 [^1]: First note

 [^2]: Second note

[^10]: Final note
```

`convert_footnotes` only processes the final contiguous numeric list that
immediately follows an H2 heading when these conditions are met.

## HTML table support in mdtablefix

`mdtablefix` can format simple HTML `<table>` elements embedded in Markdown.
These HTML tables are transformed into Markdown before the main table reflow
logic runs. That preprocessing is handled by the `convert_html_tables`
function. The parser path uses `html5ever` to build a temporary `RcDom` tree
through `markup5ever_rcdom`, then walks that tree to extract table rows and
cells.

Only straightforward tables with `<tr>`, `<th>` and `<td>` tags are detected.
Attributes and tag casing are ignored, and complex nested or styled tables are
not supported. After conversion, each HTML table is represented as a Markdown
table, so the usual reflow algorithm can align its columns consistently with
the rest of the document.

```html
<table>
  <tr><th>A</th><th>B</th></tr>
  <tr><td>1</td><td>2</td></tr>
</table>
```

The converter checks the first table row for `<th>` cells or for `<strong>` or
`<b>` tags inside `<td>` elements to decide whether it is a header. If no such
markers exist and the table contains multiple rows, the first row is still
treated as the header, so the Markdown output includes a separator line. This
last-resort behaviour keeps simple tables readable after conversion.

## Module Relationships

This diagram illustrates the connections between the crate's modules.

```mermaid
classDiagram
    class lib {
        <<module>>
    }
    class html {
        <<module>>
        +convert_html_tables()
        +html_table_to_markdown() %% deprecated
    }
    class table {
        <<module>>
        +reflow_table()
        +split_cells()
        +SEP_RE
    }
    class wrap {
        <<module>>
        +wrap_text()
        +is_fence()
        +Token
        +tokenize_markdown()
    }
    class lists {
        <<module>>
        +renumber_lists()
    }
    class breaks {
        <<module>>
        +format_breaks()
        +THEMATIC_BREAK_LEN
    }
    class ellipsis {
        <<module>>
        +replace_ellipsis()
    }
    class fences {
        <<module>>
        +compress_fences()
        +attach_orphan_specifiers()
    }
    class footnotes {
        <<module>>
        +convert_footnotes()
    }
    class footnotes_renumber_definitions {
        <<module>>
        +DefinitionScanState
    }
    class footnotes_renumber_reorder {
        <<module>>
        +reorder_definition_block()
    }
    class textproc {
        <<module>>
        +process_tokens()
    }
    class process {
        <<module>>
        +process_stream()
        +process_stream_no_wrap()
    }
    class process_buffer {
        <<module>>
        +ProcessBuffer
    }
    class io {
        <<module>>
        +rewrite()
        +rewrite_no_wrap()
    }
    lib --> html
    lib --> table
    lib --> wrap
    lib --> lists
    lib --> breaks
    lib --> ellipsis
    lib --> fences
    lib --> process
    lib --> io
    html ..> wrap : uses is_fence
    table ..> reflow : uses parse_rows, etc.
    lists ..> wrap : uses is_fence
    breaks ..> wrap : uses is_fence
    ellipsis ..> textproc : uses process_tokens
    process ..> html : uses convert_html_tables
    process ..> table : uses reflow_table
    process ..> wrap : uses wrap_text, is_fence
    process ..> fences : uses compress_fences, attach_orphan_specifiers
    process ..> ellipsis : uses replace_ellipsis
    process ..> footnotes : uses convert_footnotes
    process ..> process_buffer : buffers active table run
    footnotes --> footnotes_renumber_definitions
    footnotes_renumber_definitions --> footnotes_renumber_reorder : final definition block
    footnotes_renumber_reorder --> footnotes_renumber_definitions : DefinitionLine, definition_segment_end
    footnotes ..> wrap : uses tokenize_markdown
    footnotes ..> textproc : uses push_original_token
    io ..> process : uses process_stream, process_stream_no_wrap
```

The `lib` module is re-exported as the public API from the other modules. The
`wrap` module exposes the `Token` enum and `tokenize_markdown` function for
custom processing. The `ellipsis` module performs text normalization, while
`footnotes` converts bare references. The `textproc` module contains shared
token-processing helpers used by both the `ellipsis` and `footnotes` modules.
Tokenization is handled by `wrap::tokenize_markdown`, replacing the small state
machine that previously resided in `process_tokens`. The `process` module
provides streaming helpers that combine the lower-level functions. The `io`
module handles filesystem operations, delegating the text processing to
`process`.

### Stateful helpers

`BlockquotePrefix` owns the wrap module's interpretation of leading blockquote
structure. It borrows the source line, preserves the prefix's exact spacing,
and exposes both nesting depth and stripped inner content. The type is the only
blockquote-prefix parser used by the wrapping pipeline; downstream
classification, fence detection, and prefix-aware emission consume its parsed
view rather than matching blockquote syntax independently. It does not parse
general Markdown containers or mutate content, so callers compose it with the
existing list, fence, and inline parsers after stripping the outer prefix.

`ParagraphWriter` owns paragraph buffering and flush boundaries for wrapping.
It keeps the current indent, emits wrapped or verbatim lines into the caller's
output buffer, and leaves inline fitting to the wrapping helpers.

Prefixed paragraphs whose inline-code span crosses a source line remain pending
after the closing fence until the list, blockquote, or footnote continuation
ends. This lets the closing-line tail and later continuation text participate
in one greedy fit instead of requiring a second formatter pass. The pending
state and prefix helpers live in `wrap::paragraph::pending`; they are private
to the wrapping pipeline and are not a general Markdown-prefix API.

Plain paragraph buffers retain their source segments until flush. The
`wrap::paragraph::spanning_code` fallback inspects only source boundaries
inside matched inline-code spans. If joining such a span would exceed the width
while every source line already conforms, it emits those source lines
unchanged. Ordinary prose and code spans that fit remain owned by the standard
greedy wrapper.

`HtmlTableState` buffers candidate HTML table lines until the surrounding table
closes. Its depth counter tracks nested `<table>` blocks, so only the outermost
table is converted at once, while incomplete input can still be flushed back
verbatim.

`ProcessBuffer` owns the active table run during stream processing. It flushes
that buffer before lines that open a new Markdown block, including blockquote,
list-item, link-reference, and footnote-definition lines that themselves
contain pipe characters, so those lines continue through the regular block
pipeline rather than becoming continuation rows.

The `footnotes::renumber::definitions` submodule owns definition scanning and
rewriting. `DefinitionScanState` coordinates the number mapping, collects
already-parsed definitions, and stages numeric candidates for later conversion
without cluttering the top-level renumber flow.

The sibling `footnotes::renumber::reorder` submodule consumes the
`DefinitionLine` rewrite plan once scanning is complete, then reorders the
final definition block by ascending new number, breaking ties by original
index. It uses `definitions::definition_segment_end` so scanning and reordering
compute identical segment boundaries, keeping definition headers and
continuation lines grouped. Block prefixes remain in place, and leading
separator blanks migrate at the first-segment boundary. If recomposing the
block would change its row count, the reorder is skipped with a warning.

`ListState` tracks the active indentation stack and per-indent counters for
ordered list renumbering. It resets on headings and thematic breaks, and it
uses paragraph boundaries to decide when numbering should restart.

### Tokenizer flow

The inline tokenizer still iterates over the source string lazily, so no
duplicate `Vec<char>` representation is required. The resulting tokens are then
grouped into Markdown-aware fragments and passed to
`textwrap::wrap_algorithms::wrap_first_fit`, which chooses the breakpoints
without splitting code spans, links, or punctuation groups.

```mermaid
flowchart TD
    A["Input text (&str)"] --> B["Tokenize into whitespace and inline Markdown tokens"]
    B --> C["Normalize inline footnote reference spacing"]
    C --> D["Group tokens into Markdown-aware fragments"]
    D --> E["Measure fragment widths with unicode-width"]
    E --> F["Run textwrap wrap_first_fit over current fragments"]
    F --> G["Merge whitespace-only continuation lines forward"]
    G --> H["Render wrapped lines, trimming only a single trailing separator space"]
```

Figure: Wrap-tokenizer flow. Starting from an input string, the wrapper emits
whitespace and inline Markdown tokens, normalizes inline footnote references
with `normalize_footnote_ref_spacing`, groups tokens into fragments, measures
their display widths with `unicode-width`, feeds them through
`textwrap::wrap_algorithms::wrap_first_fit`, and then reconstructs wrapped
lines while preserving Markdown-aware spacing rules.

### Wrap flow

The higher-level `wrap_text` entry point combines block classification,
paragraph buffering, prefix-aware wrapping, and inline line fitting. The
following flow shows how a line moves through those stages before it is either
preserved verbatim or emitted as wrapped output.

```mermaid
flowchart TD
    A[Start: wrap_text called with lines and width] --> P[Parse blockquote prefix and depth]
    P --> Q{handle_fence_line recognizes a marker at the current depth}
    Q -->|Yes| C[Preserve line verbatim]
    Q -->|No| R{FenceTracker active at the current depth}
    R -->|Yes| C
    R -->|No| B{Classify stripped inner content}

    B -->|Indented code block| C
    B -->|Table or heading or directive| C
    B -->|Blank line| D[Flush active paragraph and emit blank]
    B -->|Paragraph or prefixed line| E[Send to ParagraphWriter]

    E --> F{Has prefix such as bullet, blockquote, footnote}
    F -->|Yes| G[wrap_with_prefix computes display width using unicode-width]
    F -->|No| H[wrap_preserving_code wraps inline content]

    G --> I[fragment-building / post-process helpers]
    H --> I

    I --> J[textwrap::wrap_algorithms::wrap_first_fit performs line breaking]
    J --> K[Reconstruct wrapped lines with prefixes and preserved spans]
    K --> L[Emit wrapped lines to wrap_text]

    C --> M[Append line to output]
    D --> M
    L --> M

    M --> N{More input lines?}
    N -->|Yes| P
    N -->|No| O[Flush remaining paragraph and finish]
```

_Figure 2: `wrap_text` control flow. The wrapper first extracts blockquote
depth and inner content, then applies depth-aware fence handling before
classifying that inner content. It passes fenced blocks, tables, headings,
directives, and indented code through unchanged, flushes paragraphs on blanks,
routes prose and prefixed lines through `ParagraphWriter`, computes visible
widths with `unicode-width`, and delegates inline line fitting to `textwrap`
before reconstructing the emitted Markdown lines with their original blockquote
container._

### Wrap sequence

The following sequence diagram focuses on the runtime collaboration between the
CLI entry point, `wrap_text`, `ParagraphWriter`, the inline wrapper, and
`textwrap` while a paragraph is being processed.

```mermaid
sequenceDiagram
    participant CLI as mdtablefix_CLI
    participant WT as wrap_text
    participant PW as ParagraphWriter
    participant WP as wrap_preserving_code
    participant IH as inline.rs_helpers
    participant TW as textwrap::wrap_first_fit

    CLI->>WT: wrap_text(lines, width)
    loop For each classified paragraph line
        WT->>PW: handle_prefix_line / flush_paragraph
        alt Prefixed or plain paragraph content
            PW->>WP: wrap_preserving_code(text, width)
            WP->>IH: normalize_footnote_ref_spacing
            IH->>IH: build_fragments + merge/rebalance
            IH->>TW: wrap_first_fit(fragments, line_widths)
            TW-->>IH: wrapped_fragment_groups
            IH-->>WP: wrapped_lines_with_spans
            WP-->>PW: wrapped_lines_with_prefixes
            PW-->>WT: wrapped_lines
            WT-->>CLI: append wrapped output
        else Nonwrappable line
            PW-->>WT: push_verbatim / original_line
            WT-->>CLI: append original output
        end
    end
    WT-->>CLI: return final wrapped text
```

Figure: `wrap_text` sequence flow. The CLI calls `wrap_text`, which delegates
paragraph handling to `ParagraphWriter`; wrappable paragraph content then flows
through `wrap_preserving_code`, the fragment-building and post-processing
helpers in `src/wrap/inline.rs`, and the underlying `textwrap` engine before
wrapped lines return through the same stack to the CLI, while nonwrappable
lines bypass the inline wrapping path and are emitted unchanged.

The helper `html_table_to_markdown` is retained for backward compatibility but
is deprecated. New code should call `convert_html_tables` instead.

## Concurrency with `rayon`

`mdtablefix` uses the `rayon` crate to process multiple files concurrently.
`rayon` provides a work-stealing thread pool and simple parallel iterators. The
tool relies on Rayon's global thread pool so that no manual setup is required.
The dependency is specified as `1.0` in `Cargo.toml` to track stable API
changes within the same major release.

Parallelism is enabled automatically whenever more than one file path is
provided on the command line. Each worker gathers its output before printing,
so results appear in the original order. This buffering increases memory usage
and may reduce performance if many tiny files are processed.

```mermaid
sequenceDiagram
    participant User as actor User
    participant CLI as CLI Main
    participant Formatter as format_to_string
    participant Rewriter as rewrite_in_place
    participant Stdout as Stdout
    participant Stderr as Stderr

    User->>CLI: Run CLI with multiple files
    alt Stdout mode
        CLI->>Formatter: format_to_string(file1)
        CLI->>Formatter: format_to_string(file2)
        CLI->>Formatter: format_to_string(file3)
        Note over CLI,Formatter: Files processed in parallel
        Formatter-->>CLI: Result<String> or Err(error)
        loop For each file in input order
            CLI->>Stdout: Print text (if Ok)
            CLI->>Stderr: Print error (if Err)
        end
    else In-place mode
        CLI->>Rewriter: rewrite_in_place(file1)
        CLI->>Rewriter: rewrite_in_place(file2)
        CLI->>Rewriter: rewrite_in_place(file3)
        Note over CLI,Rewriter: Files processed in parallel
        Rewriter-->>CLI: Result<()> or Err(error)
        loop For each file in input order
            CLI->>Stderr: Print error (if Err)
        end
    end
    CLI-->>User: Exit (with error if any file errored)
```

_Figure 4: The CLI processes file inputs in parallel, then reports results in
their original order: formatted text goes to stdout, while in-place processing
writes files directly and both modes report errors on stderr._

## Unicode Width Handling

`mdtablefix` wraps paragraphs and list items while respecting the display width
of Unicode characters. The `unicode-width` crate is used to compute the width
of prefixes and Markdown-aware wrapping fragments before `textwrap` performs
line fitting. This prevents emojis or other multibyte characters from causing
unexpected wraps or truncation.

Whenever wrapping logic examines the length of a token, it relies on
`UnicodeWidthStr::width` to measure visible columns rather than byte length.

## Link punctuation handling

Trailing punctuation immediately following a Markdown link or image is
tokenized separately and grouped with the link when wrapping. This keeps
sentences like:

```markdown
[link](path).
```

on a single line, rather than splitting the punctuation onto the next line when
wrapping occurs.

## Inline code punctuation handling

Trailing punctuation that follows an inline code span is grouped with the code
when wrapping. This prevents sentences such as:

```markdown
`useState`.
```

from splitting the full stop onto a new line, preserving the code span's
readability.

This grouping is deliberately narrow. Whitespace between separate inline code
spans remains a valid break opportunity, so sequences such as `.toml`, `.json`,
`.json5`, `.yaml`, and `.yml` can wrap between spans when required. The
coupling rule only keeps immediately trailing punctuation with the preceding
code span.

Inflectional affixes and possessive markers (`s`, `'s`, `ed`, `ing`) and
hyphenated compounds that appear immediately after a closing backtick fence are
absorbed into the code token during tokenization by `scan_code_suffix_end` in
`src/wrap/tokenize/scanning.rs`. The combined code-and-suffix token is then
classified as atomic by `has_inline_code_structure` in
`src/wrap/inline/fragment.rs`, so wrapping treats the full string — for example,
`` `VarGuard`s ``, `` `class`'s ``, `` `fetch`ed ``, or `` `run`ning `` — as
an unbreakable unit. No line break is inserted between the closing backtick and
the following letters.