Skip to main content

docling_core/
pandoc.rs

1//! Pandoc AST output (`--to pandoc`, #515): the document as the JSON
2//! serialization of Pandoc's `Pandoc` type (`pandoc -f json`), which hands
3//! docling.rs's parsing to every Pandoc writer — DOCX, ODT, EPUB, RST, Org,
4//! Typst, AsciiDoc, … — through `docling-rs in.pdf --to pandoc | pandoc -f
5//! json -t docx -o out.docx`.
6//!
7//! Like [`crate::html`] and [`crate::latex`] this walks the *JSON* document
8//! model ([`export_to_json_value`](crate::DoclingDocument::export_to_json_value))
9//! rather than the flat [`Node`](crate::Node) stream: the JSON export already
10//! carries docling's item structure for every backend — the backend-built
11//! item tree where one exists (HTML, DOCX: heading nesting, inline groups of
12//! formatted runs, rich table cells), docling's generic grouping rules
13//! otherwise — so the AST inherits those decisions instead of re-deriving
14//! them. The walk is the HTML serializer's (`_iterate_items` with groups,
15//! pictures not traversed, captions rendered by the item that owns them,
16//! content layers filtered with their children still walked); only the
17//! node it emits differs.
18//!
19//! Mapping (Pandoc constructor ← docling item):
20//!
21//! | docling | Pandoc |
22//! |---|---|
23//! | `title` | `Header 1` |
24//! | `section_header` (level *n*) | `Header (n+1)`, capped at 6 — docling's title-reserving depth; under [`HeadingDepth::Source`] (#848) `Header n`, the depth the DOCX / ODT author wrote |
25//! | `text`, `paragraph` | `Para` (`Plain` inside lists and table cells) |
26//! | `inline` group | the runs' inlines joined by `Space` |
27//! | formatting / hyperlink | `Strong`, `Emph`, `Underline`, `Strikeout`, `Subscript`, `Superscript`, then `Link` around it all — an internal link's target is `#id` (#849); a Markdown `[label](target)` inside an item's text, the flat node chain's hyperlink (`.doc`, AbiWord, …), is a `Link` too (#852) |
28//! | a numbered heading (#850: a DOCX heading numbered through `w:numPr`, docling's `1.2 Title` text with `_number`) | the `Header` holds the authored text alone, `Title` — Pandoc numbers sections itself (`--number-sections`), and its docx reader writes no number, so a numbered AST would show `1.2 1.2 Title`; the Markdown and JSON keep docling's numbered text |
29//! | a link destination (#849: a DOCX bookmark, `_anchors`) | on a heading, its `Header` identifier (Pandoc's own auto-identifier of the heading text, `target-section`, so `#target-section` resolves as it does from `pandoc -f docx`); anywhere else an empty `Span` with the identifier and the class `anchor`, Pandoc's docx reader's shape, at the start of the item's inlines |
30//! | `list` group | `OrderedList` (first item enumerated; start, number style and delimiter from the items' markers — `a)` LowerAlpha OneParen, `iii.` LowerRoman from 3, `(3)` Decimal TwoParens, #835) / `BulletList`, nested lists inside their item |
31//! | `code` | `CodeBlock` with the language as its class (`Code` inline) |
32//! | `formula` | `Para [Math DisplayMath]` (`Math InlineMath` inline) |
33//! | `checkbox_selected` / `_unselected` | the text after `☒` / `☐` — Pandoc's own task-list convention |
34//! | `table` | `Table`: leading all-header rows as `TableHead`, `rowspan` / `colspan`, rich cells as their blocks, captions as the caption |
35//! | `picture` | `Para [Image]` (Pandoc's readers' shape), or a `Figure` holding it when there is a caption or a tabular chart's data `Table`; the `Image`'s alt text is the picture's description — the alt text a DOCX / PPTX / XLSX / ODT author typed (#834) — else its caption; its `width` / `height` attributes the size the document displays it at; the target per [`ImageMode`] |
36//! | footnotes of a table / picture | `Note` at the end of its caption (the float is the call site) |
37//! | key-value / form graph | `Div .key-value-region` / `.form-container` holding a `DefinitionList` (or the nested `BulletList` of a hierarchical graph) |
38//! | form `field_region` | `Div .field-region` holding a `DefinitionList`: `marker` + `field_key` as the term, each `field_value` a definition |
39//! | any other text label (`caption` without an owner, `footnote` without one, `reference`, `handwritten_text`, `page_header`, …) | `Div .docling-<label> [Para]` |
40//! | the document's metadata (#836) | `meta`: `title`, `subject`, `description`, `date` (the document's own, else its creation date), `publisher`, `rights`, `identifier` as `MetaInlines`, `author` and `keywords` as `MetaList`s of `MetaInlines`, `lang` as `MetaString` |
41//! | a tracked change kept under `track_changes=all` (#851: a DOCX `w:ins` / `w:del` / `w:moveTo` / `w:moveFrom` run, `_revision`) | `Span .insertion` / `Span .deletion` with `author` and `date` around the run's inlines — Pandoc's docx reader's `--track-changes=all` shape; under an explicit `accept` / `reject` the document holds that view's text alone and the comments below are left out, as Pandoc leaves them |
42//! | a reviewer comment on an item (#836) | Pandoc's docx reader's `--track-changes=all` shape: a `Span .comment-start` with `comment-id`, `author` and `date` holding the comment's text before the item's inlines, and an empty `Span .comment-end` with the `comment-id` after them; a thread's replies follow as further pairs |
43//!
44//! No Pandoc equivalent, so not written: provenance (pages, bounding boxes),
45//! confidence and classification meta, form field geometry; furniture
46//! (headers/footers) is off unless its [`ContentLayers`] are asked for,
47//! exactly as in the HTML export, and the notes layer's comment sections
48//! are written as the spans above, never as blocks. Images
49//! are targets, not bytes, in Pandoc: the default [`ImageMode::Embedded`]
50//! writes `data:` URIs, so the AST is self-contained and `pandoc -f json -t
51//! docx` embeds every picture (#537); [`ImageMode::Referenced`] links files
52//! under the artifacts directory (relative paths, resolved from where
53//! `pandoc` runs). A picture without a target — [`ImageMode::Placeholder`],
54//! or one whose payload docling cannot decode (EMF/WMF) — is still an
55//! `Image`, classed `docling-placeholder` with an empty target: it survives
56//! into every writer (which reports the missing resource and prints the alt
57//! text) instead of vanishing as raw HTML.
58//!
59//! Version: the output is stamped [`PANDOC_API_VERSION`] — the latest
60//! `pandoc-types` (Pandoc 3.x). Pandoc accepts a document whose
61//! `pandoc-api-version` agrees on the first two components; a caller asking
62//! for another API ([`PandocExportOptions::api_version`]) gets
63//! [`PandocError::UnsupportedApiVersion`] rather than a document Pandoc would
64//! reject. Every node is built through the [`ast`] constructors, the one
65//! place a future API change touches.
66
67use std::collections::{BTreeMap, HashMap, HashSet};
68
69use serde_json::Value;
70
71use crate::document::{ContentLayers, DoclingDocument};
72use crate::markdown::ImageMode;
73use crate::TrackChanges;
74
75/// The `pandoc-api-version` written — `pandoc-types` 1.23.1.1 (Pandoc 3.x).
76pub const PANDOC_API_VERSION: [u32; 4] = [1, 23, 1, 1];
77
78/// A Pandoc export: the JSON AST, and — for [`ImageMode::Referenced`] — the
79/// image files it links to, as `(path under artifacts_dir, bytes)`.
80pub type PandocOutput = (String, Vec<(String, Vec<u8>)>);
81
82/// How a `section_header`'s `level` becomes a Pandoc `Header` depth (#848).
83#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
84pub enum HeadingDepth {
85    /// docling's convention, the default: `Header 1` is the document title
86    /// and a `section_header` of level *n* is `Header (n+1)` (capped at 6),
87    /// the depth docling's Markdown and HTML serializers give it (`##` for
88    /// level 1). The flat `Node::Heading` round trip relies on it — a
89    /// level-2 heading is stored as a level-1 `section_header` and comes
90    /// back as `Header 2` — as do consumers of docling's JSON, so it is not
91    /// changed under them.
92    #[default]
93    Docling,
94    /// The source's own depth: a `section_header` of level *n* is `Header n`
95    /// (clamped to 1–6; a missing or zero level is 1) and the `title` stays
96    /// `Header 1` — what Pandoc's own docx / odt readers give (a DOCX
97    /// `Heading 1` is `Header 1`), so `--number-sections`, TOC depths and
98    /// `--shift-heading-level-by` behave as on `pandoc -f docx`. A document
99    /// whose backend reserves level 1 for the title (HTML, Markdown: an
100    /// `<h2>` is a level-1 `section_header`) comes out one depth shallower
101    /// under it.
102    Source,
103}
104
105impl HeadingDepth {
106    /// The spellings every surface accepts (`docling` = the default).
107    pub const NAMES: [&'static str; 2] = ["docling", "source"];
108
109    pub fn as_str(self) -> &'static str {
110        match self {
111            HeadingDepth::Docling => "docling",
112            HeadingDepth::Source => "source",
113        }
114    }
115}
116
117impl std::str::FromStr for HeadingDepth {
118    type Err = String;
119
120    fn from_str(s: &str) -> Result<Self, String> {
121        match s.trim().to_ascii_lowercase().as_str() {
122            "docling" | "" => Ok(HeadingDepth::Docling),
123            "source" => Ok(HeadingDepth::Source),
124            other => Err(format!(
125                "unknown heading depth {other:?} (expected docling or source)"
126            )),
127        }
128    }
129}
130
131/// Options for [`to_pandoc`].
132#[derive(Debug, Clone)]
133pub struct PandocExportOptions {
134    /// How pictures carry their image (see the module docs); `Embedded` by
135    /// default, so the AST alone rebuilds a document with its pictures.
136    pub image_mode: ImageMode,
137    /// The directory referenced images are written under
138    /// (`<artifacts_dir>/image_NNNNNN.<ext>`, the Markdown export's names).
139    pub artifacts_dir: String,
140    /// The content layers written; body only by default.
141    pub layers: ContentLayers,
142    /// The Pandoc API version the caller needs (`"1.23"`, `"1.23.1.1"`, …);
143    /// `None` = [`PANDOC_API_VERSION`]. Only that API is supported.
144    pub api_version: Option<String>,
145    /// The `Header` depth of a `section_header` (#848): docling's
146    /// title-reserving `n + 1` by default, or the source's own `n`.
147    pub heading_depth: HeadingDepth,
148    /// The tracked-changes policy the document was read under (#851).
149    /// `None` (no policy asked for) and `Some(All)` write the reviewer
150    /// comments as `Span .comment-start` / `.comment-end` pairs; an
151    /// explicit `Accept` / `Reject` leaves them out, as Pandoc's
152    /// `--track-changes=accept|reject` do.
153    pub track_changes: Option<TrackChanges>,
154}
155
156impl Default for PandocExportOptions {
157    fn default() -> Self {
158        Self {
159            image_mode: ImageMode::Embedded,
160            artifacts_dir: "artifacts".to_string(),
161            layers: ContentLayers::BODY,
162            api_version: None,
163            heading_depth: HeadingDepth::Docling,
164            track_changes: None,
165        }
166    }
167}
168
169/// Why a Pandoc export was refused.
170#[derive(Debug, Clone, PartialEq, Eq)]
171pub enum PandocError {
172    /// The requested `pandoc-api-version` is not the one supported.
173    UnsupportedApiVersion { requested: String },
174}
175
176impl std::fmt::Display for PandocError {
177    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
178        match self {
179            PandocError::UnsupportedApiVersion { requested } => write!(
180                f,
181                "unsupported Pandoc API version '{requested}': only {} (pandoc-types {}, Pandoc 3.x) is supported",
182                version_string(&PANDOC_API_VERSION[..2]),
183                version_string(&PANDOC_API_VERSION),
184            ),
185        }
186    }
187}
188
189impl std::error::Error for PandocError {}
190
191fn version_string(parts: &[u32]) -> String {
192    parts
193        .iter()
194        .map(u32::to_string)
195        .collect::<Vec<_>>()
196        .join(".")
197}
198
199/// Check a requested API version: Pandoc reads a document whose version
200/// shares its first two components (major API, minor API), so `1.23`,
201/// `1.23.1` and `1.23.1.1` are all this export's; anything else is refused.
202pub fn check_api_version(requested: &str) -> Result<(), PandocError> {
203    let err = || PandocError::UnsupportedApiVersion {
204        requested: requested.to_string(),
205    };
206    let parts: Vec<u32> = requested
207        .trim()
208        .split('.')
209        .map(|p| p.parse::<u32>().map_err(|_| err()))
210        .collect::<Result<_, _>>()?;
211    if parts.len() < 2 || parts[..2] != PANDOC_API_VERSION[..2] {
212        return Err(err());
213    }
214    if parts.len() > PANDOC_API_VERSION.len()
215        || parts
216            .iter()
217            .zip(PANDOC_API_VERSION.iter())
218            .skip(2)
219            .any(|(a, b)| a > b)
220    {
221        // A newer patch than this export knows.
222        return Err(err());
223    }
224    Ok(())
225}
226
227/// The document as Pandoc JSON (compact, one line, like `pandoc -t json`),
228/// and for [`ImageMode::Referenced`] the `(path, bytes)` image files to
229/// write.
230pub fn to_pandoc(
231    doc: &DoclingDocument,
232    options: &PandocExportOptions,
233) -> Result<PandocOutput, PandocError> {
234    // The JSON with the note calls docling's model has no field for (#538).
235    from_docling_json(&crate::json::to_json_with_notes(doc), options)
236}
237
238/// [`to_pandoc`] for a document already in docling's JSON wire format —
239/// e.g. one produced by Python docling, or `DoclingDocument.export_to_dict()`
240/// in the Python bindings.
241pub fn from_docling_json(
242    json: &Value,
243    options: &PandocExportOptions,
244) -> Result<PandocOutput, PandocError> {
245    if let Some(v) = &options.api_version {
246        check_api_version(v)?;
247    }
248    // The walk recurses per nesting level; an XBRL instance nests thousands
249    // deep, so it runs on a big-stack thread like the HTML serializer.
250    let render = || {
251        let mut ser = Serializer::new(json, options);
252        let blocks = ser.body();
253        let doc = ast::document_with_meta(meta_of(json), blocks);
254        (
255            serde_json::to_string(&doc).expect("Pandoc JSON is always serializable"),
256            ser.artifacts,
257        )
258    };
259    #[cfg(not(target_arch = "wasm32"))]
260    {
261        Ok(std::thread::scope(|scope| {
262            std::thread::Builder::new()
263                .name("docling-pandoc".into())
264                .stack_size(256 << 20)
265                .spawn_scoped(scope, render)
266                .expect("spawn the pandoc serializer thread")
267                .join()
268                .expect("pandoc serializer thread panicked")
269        }))
270    }
271    #[cfg(target_arch = "wasm32")]
272    {
273        Ok(render())
274    }
275}
276
277/// Pandoc's auto-identifier of a heading (`Text.Pandoc.Shared.
278/// stringToIdentifier`, the default `auto_identifiers` rule, #849): the
279/// text lower-cased, everything but letters, digits, `_`, `-`, `.` and
280/// spaces removed, the words joined by `-`, anything up to the first letter
281/// dropped; `section` when nothing is left. `Target section` →
282/// `target-section`, `2026 Results` → `results`.
283pub fn auto_identifier(text: &str) -> String {
284    let kept: String = text
285        .chars()
286        .filter(|c| c.is_alphanumeric() || c.is_whitespace() || matches!(c, '_' | '-' | '.'))
287        .flat_map(char::to_lowercase)
288        .collect();
289    let joined = kept.split_whitespace().collect::<Vec<_>>().join("-");
290    let id = joined.trim_start_matches(|c: char| !c.is_alphabetic());
291    if id.is_empty() {
292        "section".to_string()
293    } else {
294        id.to_string()
295    }
296}
297
298/// The Pandoc AST constructors (`Text.Pandoc.Definition`, JSON per
299/// `Text.Pandoc.JSON`): every node this module writes is built here.
300pub mod ast {
301    use serde_json::{json, Value};
302
303    use super::PANDOC_API_VERSION;
304
305    pub fn document(blocks: Vec<Value>) -> Value {
306        document_with_meta(json!({}), blocks)
307    }
308    /// A document with its `Meta` (#836).
309    pub fn document_with_meta(meta: Value, blocks: Vec<Value>) -> Value {
310        json!({ "pandoc-api-version": PANDOC_API_VERSION, "meta": meta, "blocks": blocks })
311    }
312    pub fn meta_inlines(inlines: Vec<Value>) -> Value {
313        node("MetaInlines", json!(inlines))
314    }
315    pub fn meta_string(s: &str) -> Value {
316        node("MetaString", json!(s))
317    }
318    pub fn meta_list(items: Vec<Value>) -> Value {
319        node("MetaList", json!(items))
320    }
321    /// `Span Attr [Inline]` with key-value attributes.
322    pub fn span(classes: &[&str], attrs: &[(String, String)], inlines: Vec<Value>) -> Value {
323        let kv: Vec<Value> = attrs.iter().map(|(k, v)| json!([k, v])).collect();
324        node("Span", json!([["", classes, kv], inlines]))
325    }
326
327    /// `Attr`: identifier, classes, key-value pairs.
328    pub fn attr(classes: &[&str]) -> Value {
329        json!(["", classes, []])
330    }
331
332    fn node(t: &str, c: Value) -> Value {
333        json!({ "t": t, "c": c })
334    }
335
336    // --- blocks -------------------------------------------------------------
337
338    pub fn para(inlines: Vec<Value>) -> Value {
339        node("Para", json!(inlines))
340    }
341    pub fn plain(inlines: Vec<Value>) -> Value {
342        node("Plain", json!(inlines))
343    }
344    pub fn header(level: usize, inlines: Vec<Value>) -> Value {
345        node("Header", json!([level, attr(&[]), inlines]))
346    }
347    /// A `Header` with an identifier (#849: a bookmarked heading).
348    pub fn header_with_id(level: usize, id: &str, inlines: Vec<Value>) -> Value {
349        node("Header", json!([level, [id, [], []], inlines]))
350    }
351    /// An empty `Span` carrying an identifier and the class `anchor` — the
352    /// link destination Pandoc's docx reader writes for a bookmark (#849).
353    pub fn anchor(id: &str) -> Value {
354        node("Span", json!([[id, ["anchor"], []], []]))
355    }
356    pub fn code_block(language: Option<&str>, text: &str) -> Value {
357        let classes: Vec<&str> = language.into_iter().collect();
358        node("CodeBlock", json!([attr(&classes), text]))
359    }
360    pub fn bullet_list(items: Vec<Vec<Value>>) -> Value {
361        node("BulletList", json!(items))
362    }
363    /// `OrderedList ListAttributes [[Block]]`: `style` a `ListNumberStyle`
364    /// (`Decimal`, `LowerAlpha`, `UpperRoman`, …), `delim` a
365    /// `ListNumberDelim` (`Period`, `OneParen`, `TwoParens`, `DefaultDelim`).
366    pub fn ordered_list(start: u64, style: &str, delim: &str, items: Vec<Vec<Value>>) -> Value {
367        node(
368            "OrderedList",
369            json!([[start, { "t": style }, { "t": delim }], items]),
370        )
371    }
372    pub fn definition_list(entries: Vec<(Vec<Value>, Vec<Vec<Value>>)>) -> Value {
373        node(
374            "DefinitionList",
375            Value::Array(
376                entries
377                    .into_iter()
378                    .map(|(term, defs)| json!([term, defs]))
379                    .collect(),
380            ),
381        )
382    }
383    /// `RawBlock Format Text` — kept by writers of that format, dropped by
384    /// the others.
385    pub fn raw_block(format: &str, text: &str) -> Value {
386        node("RawBlock", json!([format, text]))
387    }
388    pub fn div(classes: &[&str], blocks: Vec<Value>) -> Value {
389        node("Div", json!([attr(classes), blocks]))
390    }
391    /// `Caption`: no short caption, the long one as blocks.
392    pub fn caption(blocks: Vec<Value>) -> Value {
393        json!([null, blocks])
394    }
395    pub fn figure(classes: &[&str], caption: Value, blocks: Vec<Value>) -> Value {
396        node("Figure", json!([attr(classes), caption, blocks]))
397    }
398    /// A table cell: `Cell Attr Alignment RowSpan ColSpan [Block]`.
399    pub fn cell(row_span: usize, col_span: usize, blocks: Vec<Value>) -> Value {
400        json!([attr(&[]), { "t": "AlignDefault" }, row_span, col_span, blocks])
401    }
402    pub fn row(cells: Vec<Value>) -> Value {
403        json!([attr(&[]), cells])
404    }
405    /// `Table Attr Caption [ColSpec] TableHead [TableBody] TableFoot`.
406    pub fn table(caption: Value, num_cols: usize, head: Vec<Value>, body: Vec<Value>) -> Value {
407        let colspecs: Vec<Value> = (0..num_cols)
408            .map(|_| json!([{ "t": "AlignDefault" }, { "t": "ColWidthDefault" }]))
409            .collect();
410        node(
411            "Table",
412            json!([
413                attr(&[]),
414                caption,
415                colspecs,
416                [attr(&[]), head],
417                [[attr(&[]), 0, [], body]],
418                [attr(&[]), []]
419            ]),
420        )
421    }
422
423    // --- inlines ------------------------------------------------------------
424
425    pub fn str_(s: &str) -> Value {
426        node("Str", json!(s))
427    }
428    pub fn space() -> Value {
429        json!({ "t": "Space" })
430    }
431    pub fn line_break() -> Value {
432        json!({ "t": "LineBreak" })
433    }
434    /// `Strong`, `Emph`, `Underline`, `Strikeout`, `Subscript`, `Superscript`.
435    pub fn wrap(t: &str, inlines: Vec<Value>) -> Value {
436        node(t, json!(inlines))
437    }
438    pub fn code(text: &str) -> Value {
439        node("Code", json!([attr(&[]), text]))
440    }
441    pub fn math(display: bool, tex: &str) -> Value {
442        let kind = if display { "DisplayMath" } else { "InlineMath" };
443        node("Math", json!([{ "t": kind }, tex]))
444    }
445    pub fn link(inlines: Vec<Value>, url: &str) -> Value {
446        node("Link", json!([attr(&[]), inlines, [url, ""]]))
447    }
448    pub fn image(classes: &[&str], alt: Vec<Value>, src: &str) -> Value {
449        image_sized(classes, &[], alt, src)
450    }
451    /// An `Image` with key-value attributes (`width` / `height`, #834).
452    pub fn image_sized(
453        classes: &[&str],
454        attrs: &[(String, String)],
455        alt: Vec<Value>,
456        src: &str,
457    ) -> Value {
458        let kv: Vec<Value> = attrs.iter().map(|(k, v)| json!([k, v])).collect();
459        node("Image", json!([["", classes, kv], alt, [src, ""]]))
460    }
461    pub fn note(blocks: Vec<Value>) -> Value {
462        node("Note", json!(blocks))
463    }
464}
465
466/// Text as `Str` words and `Space`s, `\n` as `LineBreak`; surrounding
467/// whitespace trimmed (a no-break space stays inside its word, as Pandoc's
468/// readers keep it). A Markdown link `[label](target)` in the text — how
469/// the flat node chain (the `.doc`, AbiWord, … readers) carries a
470/// hyperlink, #852 — is a `Link`.
471fn text_inlines(text: &str) -> Vec<Value> {
472    let mut out = Vec::new();
473    for (i, line) in text.trim().split('\n').enumerate() {
474        if i > 0 {
475            out.push(ast::line_break());
476        }
477        let is_ws = |c: char| c.is_whitespace() && c != '\u{a0}';
478        let line_start = out.len();
479        // Whitespace seen since the last inline of this line: one `Space`
480        // before the next.
481        let mut pending_space = false;
482        let mut rest = line;
483        while !rest.is_empty() {
484            let (plain, link) = match markdown_link(rest) {
485                Some((before, label, target, after)) => {
486                    rest = after;
487                    (before, Some((label, target)))
488                }
489                None => (std::mem::take(&mut rest), None),
490            };
491            let words: Vec<&str> = plain.split(is_ws).filter(|w| !w.is_empty()).collect();
492            if words.is_empty() {
493                pending_space |= !plain.is_empty();
494            } else {
495                if out.len() > line_start && (pending_space || plain.starts_with(is_ws)) {
496                    out.push(ast::space());
497                }
498                for (k, word) in words.iter().enumerate() {
499                    if k > 0 {
500                        out.push(ast::space());
501                    }
502                    out.push(ast::str_(word));
503                }
504                pending_space = plain.ends_with(is_ws);
505            }
506            if let Some((label, target)) = link {
507                if out.len() > line_start && pending_space {
508                    out.push(ast::space());
509                }
510                let mut label_inlines = Vec::new();
511                for (k, word) in label.split_whitespace().enumerate() {
512                    if k > 0 {
513                        label_inlines.push(ast::space());
514                    }
515                    label_inlines.push(ast::str_(word));
516                }
517                out.push(ast::link(label_inlines, target));
518                pending_space = false;
519            }
520        }
521    }
522    out
523}
524
525/// The first Markdown link of `text` as `(before, label, target, after)`:
526/// `[label](target)` with a label free of brackets and newlines and a
527/// target free of whitespace and parentheses — the shape the flat readers
528/// write; an image's `![alt](src)` is not one.
529fn markdown_link(text: &str) -> Option<(&str, &str, &str, &str)> {
530    let mut from = 0;
531    while let Some(open) = text[from..].find('[') {
532        let open = from + open;
533        from = open + 1;
534        if text[..open].ends_with('!') {
535            continue;
536        }
537        let close = text[open + 1..].find(']')?;
538        let close = open + 1 + close;
539        let label = &text[open + 1..close];
540        if label.is_empty() || label.contains(['[', '\n']) || !text[close + 1..].starts_with('(') {
541            continue;
542        }
543        let Some(end) = text[close + 2..].find(')') else {
544            continue;
545        };
546        let end = close + 2 + end;
547        let target = &text[close + 2..end];
548        if target.is_empty() || target.contains(|c: char| c.is_whitespace() || c == '(') {
549            continue;
550        }
551        return Some((&text[..open], label, target, &text[end + 1..]));
552    }
553    None
554}
555
556/// [`text_inlines`] with a `Note` spliced in at each call: `notes` is
557/// `[[offset, text], …]` (chars into `text`), the JSON's internal `_notes`.
558/// Whitespace around a call becomes a `Space`, as Pandoc's docx reader
559/// writes `word¹ next` → `Str "word", Note, Space, Str "next"`.
560fn text_with_notes(text: &str, notes: &[Value]) -> Vec<Value> {
561    let mut calls: Vec<(usize, &str)> = notes
562        .iter()
563        .filter_map(|n| Some((n.get(0)?.as_u64()? as usize, n.get(1)?.as_str()?)))
564        .collect();
565    calls.sort_by_key(|&(offset, _)| offset);
566    let chars: Vec<char> = text.chars().collect();
567    let mut out: Vec<Value> = Vec::new();
568    let push_segment = |out: &mut Vec<Value>, seg: &str| {
569        let inlines = text_inlines(seg);
570        if inlines.is_empty() {
571            if !out.is_empty() && seg.chars().any(char::is_whitespace) {
572                out.push(ast::space());
573            }
574            return;
575        }
576        if !out.is_empty() && seg.starts_with(char::is_whitespace) {
577            out.push(ast::space());
578        }
579        out.extend(inlines);
580        if seg.ends_with(char::is_whitespace) {
581            out.push(ast::space());
582        }
583    };
584    let mut prev = 0usize;
585    for (offset, note) in calls {
586        let at = offset.clamp(prev, chars.len());
587        let seg: String = chars[prev..at].iter().collect();
588        push_segment(&mut out, &seg);
589        out.push(ast::note(vec![ast::para(text_inlines(note))]));
590        prev = at;
591    }
592    let rest: String = chars[prev..].iter().collect();
593    push_segment(&mut out, &rest);
594    while out
595        .last()
596        .is_some_and(|v| v.get("t").and_then(Value::as_str) == Some("Space"))
597    {
598        out.pop();
599    }
600    out
601}
602
603/// Inline lists joined by single `Space`s (an inline group's runs) — none
604/// before a run that opens with closing punctuation (`.`, `,`, `)`, …) or
605/// after one that ends with opening punctuation: docling keeps each
606/// formatting run as its own item and its serializers join them with a
607/// space, which would put `link .` into every Pandoc output.
608fn join_inlines(parts: Vec<Vec<Value>>) -> Vec<Value> {
609    fn edge_char(inlines: &[Value], first: bool) -> Option<char> {
610        let node = if first {
611            inlines.first()
612        } else {
613            inlines.last()
614        }?;
615        match node.get("t").and_then(Value::as_str)? {
616            "Str" => {
617                let s = node.get("c")?.as_str()?;
618                if first {
619                    s.chars().next()
620                } else {
621                    s.chars().last()
622                }
623            }
624            // Formatting / links: look inside.
625            "Strong" | "Emph" | "Underline" | "Strikeout" | "Subscript" | "Superscript" => {
626                edge_char(node.get("c")?.as_array()?, first)
627            }
628            "Link" => edge_char(node.get("c")?.get(1)?.as_array()?, first),
629            _ => None,
630        }
631    }
632    let mut out: Vec<Value> = Vec::new();
633    for part in parts.into_iter().filter(|p| !p.is_empty()) {
634        if !out.is_empty() {
635            let closes = edge_char(&part, true)
636                .is_some_and(|c| ".,;:!?)]}%\u{bb}\u{201d}\u{2019}".contains(c));
637            let opens = edge_char(&out, false).is_some_and(|c| "([{\u{ab}\u{201c}".contains(c));
638            if !closes && !opens {
639                out.push(ast::space());
640            }
641        }
642        out.extend(part);
643    }
644    out
645}
646
647/// Pandoc's `Meta` from the document's `metadata` (#836, the JSON export's
648/// `--metadata json` object; Python docling's JSON has none): `title`,
649/// `subject`, `description`, `publisher`, `rights`, `identifier` and `date`
650/// — the document's own date, else when it was created — as `MetaInlines`,
651/// `author` and `keywords` as `MetaList`s of `MetaInlines` (Pandoc's own
652/// shape for several authors), `lang` as `MetaString`, so `pandoc -t docx`
653/// sets the title and author and the templates see them. Empty when the
654/// document has none.
655fn meta_of(json: &Value) -> Value {
656    let mut meta = serde_json::Map::new();
657    let Some(m) = json.get("metadata").and_then(Value::as_object) else {
658        return Value::Object(meta);
659    };
660    let text = |key: &str| {
661        m.get(key)
662            .and_then(Value::as_str)
663            .map(text_inlines)
664            .filter(|t| !t.is_empty())
665    };
666    let list = |key: &str| -> Vec<Value> {
667        m.get(key)
668            .and_then(Value::as_array)
669            .into_iter()
670            .flatten()
671            .filter_map(Value::as_str)
672            .map(text_inlines)
673            .filter(|t| !t.is_empty())
674            .map(ast::meta_inlines)
675            .collect()
676    };
677    for (key, field) in [
678        ("title", "title"),
679        ("subject", "subject"),
680        ("description", "description"),
681        ("publisher", "publisher"),
682        ("rights", "rights"),
683        ("identifier", "identifier"),
684    ] {
685        if let Some(t) = text(field) {
686            meta.insert(key.into(), ast::meta_inlines(t));
687        }
688    }
689    let authors = list("authors");
690    if !authors.is_empty() {
691        meta.insert("author".into(), ast::meta_list(authors));
692    }
693    if let Some(date) = text("date").or_else(|| text("created")) {
694        meta.insert("date".into(), ast::meta_inlines(date));
695    }
696    let keywords = list("keywords");
697    if !keywords.is_empty() {
698        meta.insert("keywords".into(), ast::meta_list(keywords));
699    }
700    if let Some(lang) = m.get("language").and_then(Value::as_str).map(str::trim) {
701        if !lang.is_empty() {
702            meta.insert("lang".into(), ast::meta_string(lang));
703        }
704    }
705    Value::Object(meta)
706}
707
708/// A docling comment note — `[author: Name (initials), time: date]: text`,
709/// the DOCX, `.doc`, ODT and XLSX backends' shape — split into its author,
710/// date and text; a note without the head is all text.
711fn split_comment(note: &str) -> (Option<String>, Option<String>, String) {
712    let Some(rest) = note.strip_prefix('[') else {
713        return (None, None, note.to_string());
714    };
715    let Some((head, body)) = rest.split_once(']') else {
716        return (None, None, note.to_string());
717    };
718    let body = body.strip_prefix(':').unwrap_or(body).trim().to_string();
719    let mut author = None;
720    let mut date = None;
721    for part in head.split(", ") {
722        if let Some(a) = part.strip_prefix("author: ") {
723            author = Some(a.trim().to_string());
724        } else if let Some(t) = part.strip_prefix("time: ") {
725            date = Some(t.trim().to_string());
726        }
727    }
728    (author, date, body)
729}
730
731/// A length in inches as Pandoc's readers write it: `2.5in`, `1.25in` —
732/// up to four decimals, trailing zeros dropped.
733fn inches(v: f64) -> String {
734    let s = format!("{v:.4}");
735    let s = s.trim_end_matches('0').trim_end_matches('.');
736    format!("{}in", if s.is_empty() { "0" } else { s })
737}
738
739/// Pandoc's `ListAttributes` — start, number style, delimiter — from a
740/// list's item markers (#835): `a)` → `(1, LowerAlpha, OneParen)`, `iii.` →
741/// `(3, LowerRoman, Period)`, `(3)` → `(3, Decimal, TwoParens)`, a bare
742/// `12` → `DefaultDelim`. Every list used to be `Decimal` / `Period`, and a
743/// non-decimal start fell back to 1. A single letter reads as Roman only
744/// when the list does — `i` / `I` (what Pandoc's Markdown reader assumes),
745/// or another item's marker a multi-letter numeral — else as alpha, so a
746/// list starting at `c)` is the third letter, not 100. Letters repeat past
747/// `z` (`aa` = 27, Pandoc's own extension). A marker nothing parses
748/// (`1.2.`) keeps the old `(1, Decimal, Period)`.
749fn list_attributes(markers: &[&str]) -> (u64, &'static str, &'static str) {
750    fn split(marker: &str) -> (&str, &'static str) {
751        let m = marker.trim();
752        if let Some(inner) = m.strip_prefix('(').and_then(|r| r.strip_suffix(')')) {
753            return (inner.trim(), "TwoParens");
754        }
755        if let Some(inner) = m.strip_suffix(')') {
756            return (inner.trim(), "OneParen");
757        }
758        if let Some(inner) = m.strip_suffix('.') {
759            return (inner.trim(), "Period");
760        }
761        (m, "DefaultDelim")
762    }
763    fn roman(token: &str) -> Option<u64> {
764        let value = |c: char| match c.to_ascii_lowercase() {
765            'i' => Some(1),
766            'v' => Some(5),
767            'x' => Some(10),
768            'l' => Some(50),
769            'c' => Some(100),
770            'd' => Some(500),
771            'm' => Some(1000),
772            _ => None,
773        };
774        let values: Vec<i64> = token.chars().map(value).collect::<Option<_>>()?;
775        if values.is_empty() {
776            return None;
777        }
778        let mut total = 0i64;
779        for (i, &v) in values.iter().enumerate() {
780            // Subtractive notation: `IV` is 5 − 1.
781            if values.get(i + 1).is_some_and(|&next| next > v) {
782                total -= v;
783            } else {
784                total += v;
785            }
786        }
787        (total > 0).then_some(total as u64)
788    }
789    fn alpha(token: &str) -> Option<u64> {
790        let first = token.chars().next()?;
791        if !first.is_ascii_alphabetic() || !token.chars().all(|c| c == first) {
792            return None;
793        }
794        let ordinal = (first.to_ascii_lowercase() as u64) - ('a' as u64) + 1;
795        Some((token.chars().count() as u64 - 1) * 26 + ordinal)
796    }
797    const DEFAULT: (u64, &str, &str) = (1, "Decimal", "Period");
798    let Some(first) = markers.first() else {
799        return DEFAULT;
800    };
801    let (token, delim) = split(first);
802    if token.is_empty() {
803        return DEFAULT;
804    }
805    if token.chars().all(|c| c.is_ascii_digit()) {
806        return token
807            .parse()
808            .map(|n| (n, "Decimal", delim))
809            .unwrap_or(DEFAULT);
810    }
811    if !token.chars().all(|c| c.is_ascii_alphabetic()) {
812        return DEFAULT;
813    }
814    let upper = token.chars().next().is_some_and(|c| c.is_ascii_uppercase());
815    let letters = |t: &str| t.chars().all(|c| c.is_ascii_alphabetic());
816    let reads_roman = roman(token).is_some()
817        && (token.len() > 1
818            || token.eq_ignore_ascii_case("i")
819            || markers[1..].iter().any(|m| {
820                let (t, _) = split(m);
821                t.len() > 1 && letters(t) && roman(t).is_some()
822            }));
823    if reads_roman {
824        let style = if upper { "UpperRoman" } else { "LowerRoman" };
825        return (roman(token).unwrap_or(1), style, delim);
826    }
827    match alpha(token) {
828        Some(n) => (n, if upper { "UpperAlpha" } else { "LowerAlpha" }, delim),
829        None => DEFAULT,
830    }
831}
832
833struct GridCell<'a> {
834    cell: &'a Value,
835    start_row: usize,
836    start_col: usize,
837    row_span: usize,
838    col_span: usize,
839}
840
841struct Serializer<'a> {
842    json: &'a Value,
843    image_mode: ImageMode,
844    artifacts_dir: String,
845    layers: ContentLayers,
846    heading_depth: HeadingDepth,
847    track_changes: Option<TrackChanges>,
848    artifacts: Vec<(String, Vec<u8>)>,
849    pic_index: usize,
850    visited: HashSet<String>,
851    /// Items some table / picture / graph holds as a caption or footnote:
852    /// rendered by their owner, skipped where the walk meets them.
853    owned_captions: HashSet<String>,
854    owned_footnotes: HashSet<String>,
855}
856
857impl<'a> Serializer<'a> {
858    fn new(json: &'a Value, options: &PandocExportOptions) -> Self {
859        let mut owned_captions = HashSet::new();
860        let mut owned_footnotes = HashSet::new();
861        for bucket in [
862            "pictures",
863            "tables",
864            "key_value_items",
865            "form_items",
866            "texts",
867        ] {
868            for item in Self::array(json.get(bucket)) {
869                for (key, set) in [
870                    ("captions", &mut owned_captions),
871                    ("footnotes", &mut owned_footnotes),
872                ] {
873                    for r in Self::array(item.get(key)) {
874                        if let Some(cref) = r.get("$ref").and_then(Value::as_str) {
875                            set.insert(cref.to_string());
876                        }
877                    }
878                }
879            }
880        }
881        Self {
882            json,
883            image_mode: options.image_mode,
884            artifacts_dir: options.artifacts_dir.clone(),
885            layers: options.layers,
886            heading_depth: options.heading_depth,
887            track_changes: options.track_changes,
888            artifacts: Vec::new(),
889            pic_index: 0,
890            visited: HashSet::new(),
891            owned_captions,
892            owned_footnotes,
893        }
894    }
895
896    fn array(v: Option<&'a Value>) -> impl Iterator<Item = &'a Value> {
897        v.and_then(Value::as_array).into_iter().flatten()
898    }
899
900    fn resolve(&self, cref: &str) -> Option<&'a Value> {
901        let rest = cref.strip_prefix("#/")?;
902        if rest == "body" {
903            return self.json.get("body");
904        }
905        let (bucket, idx) = rest.split_once('/')?;
906        self.json.get(bucket)?.get(idx.parse::<usize>().ok()?)
907    }
908
909    fn refs(v: Option<&'a Value>) -> Vec<&'a str> {
910        Self::array(v)
911            .filter_map(|r| r.get("$ref").and_then(Value::as_str))
912            .collect()
913    }
914
915    fn children(item: &'a Value) -> Vec<&'a str> {
916        Self::refs(item.get("children"))
917    }
918
919    fn self_ref(item: &Value) -> &str {
920        item.get("self_ref").and_then(Value::as_str).unwrap_or("")
921    }
922
923    fn label(item: &Value) -> &str {
924        item.get("label").and_then(Value::as_str).unwrap_or("")
925    }
926
927    fn text(item: &Value) -> &str {
928        item.get("text").and_then(Value::as_str).unwrap_or("")
929    }
930
931    /// The item's link destinations (`_anchors`, #849).
932    fn anchors(item: &Value) -> Vec<String> {
933        item.get("_anchors")
934            .and_then(Value::as_array)
935            .map(|a| {
936                a.iter()
937                    .filter_map(Value::as_str)
938                    .map(str::to_string)
939                    .collect()
940            })
941            .unwrap_or_default()
942    }
943
944    fn is_group(item: &Value) -> bool {
945        Self::self_ref(item).starts_with("#/groups/")
946    }
947
948    fn excluded(&self, item: &Value) -> bool {
949        let layer = item
950            .get("content_layer")
951            .and_then(Value::as_str)
952            .unwrap_or("body");
953        !self.layers.contains_name(layer)
954    }
955
956    /// docling's `_iterate_items(with_groups=True, traverse_pictures=False)`
957    /// below `root`, depth-first pre-order; items off the chosen layers are
958    /// not yielded but their subtrees are walked; under a picture only its
959    /// captions are visited.
960    fn descendants(&self, root: &'a Value, out: &mut Vec<&'a Value>) {
961        let picture = Self::self_ref(root).starts_with("#/pictures/");
962        let captions = if picture {
963            Self::refs(root.get("captions"))
964        } else {
965            Vec::new()
966        };
967        for cref in Self::children(root) {
968            if picture && !captions.contains(&cref) {
969                continue;
970            }
971            // A caption or footnote some table / picture / graph owns is
972            // rendered by that owner, inline group and all (#833): neither
973            // it nor what nests under it is a block of the walk.
974            if self.owned_captions.contains(cref) || self.owned_footnotes.contains(cref) {
975                continue;
976            }
977            let Some(child) = self.resolve(cref) else {
978                continue;
979            };
980            if !self.excluded(child) {
981                out.push(child);
982            }
983            self.descendants(child, out);
984        }
985    }
986
987    /// Every not-yet-visited descendant of `item`, serialized in order —
988    /// a container marks what it renders as visited, so the flat walk
989    /// skips it.
990    fn blocks_of(&mut self, item: &'a Value) -> Vec<Value> {
991        let mut nodes = Vec::new();
992        self.descendants(item, &mut nodes);
993        let mut out = Vec::new();
994        for node in nodes {
995            if self.visited.insert(Self::self_ref(node).to_string()) {
996                out.extend(self.serialize(node));
997            }
998        }
999        out
1000    }
1001
1002    fn body(&mut self) -> Vec<Value> {
1003        let Some(body) = self.json.get("body") else {
1004            return Vec::new();
1005        };
1006        self.visited.insert("#/body".to_string());
1007        let mut blocks = self.blocks_of(body);
1008        let notes = self.unplaced_notes();
1009        blocks.extend(notes);
1010        blocks
1011    }
1012
1013    /// Furniture `footnote` items no text item calls — docling's DOCX / ODT
1014    /// note bodies when the JSON carries no call sites (one exported by
1015    /// Python docling), or a call that could not be anchored — written while
1016    /// the furniture layer itself is off: each as a trailing `Note`, so a
1017    /// document rebuilt from the AST still has its notes (#538).
1018    fn unplaced_notes(&mut self) -> Vec<Value> {
1019        let notes: Vec<&'a Value> = Self::array(self.json.get("texts"))
1020            .filter(|t| Self::label(t) == "footnote" && !Self::is_note_body(t))
1021            .filter(|t| t.get("content_layer").and_then(Value::as_str) == Some("furniture"))
1022            .filter(|t| self.excluded(t) && !self.owned_footnotes.contains(Self::self_ref(t)))
1023            .collect();
1024        let mut out = Vec::new();
1025        for t in notes {
1026            // The note's formatting and links too (#833).
1027            let (text, _, _) = self.own_inlines(t);
1028            if !text.is_empty() {
1029                out.push(ast::para(vec![ast::note(vec![ast::para(text)])]));
1030            }
1031        }
1032        out
1033    }
1034
1035    /// One item's blocks.
1036    fn serialize(&mut self, item: &'a Value) -> Vec<Value> {
1037        let sref = Self::self_ref(item);
1038        if Self::is_group(item) {
1039            return match Self::label(item) {
1040                // A comment thread is written as spans on the item it
1041                // annotates (#836), never as blocks; one no item refers to
1042                // (a plain cell's, an orphan's) has no place in the AST.
1043                "comment_section" => Vec::new(),
1044                "list" => self.list_group(item),
1045                "inline" => {
1046                    let (inlines, mut rest) = self.inline_group(item);
1047                    let mut out = Vec::new();
1048                    if !inlines.is_empty() {
1049                        out.push(ast::para(inlines));
1050                    }
1051                    out.append(&mut rest);
1052                    out
1053                }
1054                _ => self.blocks_of(item),
1055            };
1056        }
1057        if sref.starts_with("#/texts/") {
1058            if self.owned_captions.contains(sref) || self.owned_footnotes.contains(sref) {
1059                return Vec::new();
1060            }
1061            // A note body some text item calls is written there, as a `Note`.
1062            if self.excluded(item) || Self::is_note_body(item) {
1063                return Vec::new();
1064            }
1065            return self.text_item(item);
1066        }
1067        if sref.starts_with("#/tables/") {
1068            return self.table(item);
1069        }
1070        if sref.starts_with("#/pictures/") {
1071            return self.picture(item);
1072        }
1073        if sref.starts_with("#/key_value_items/") {
1074            return self.graph_item(item, "key-value-region");
1075        }
1076        if sref.starts_with("#/form_items/") {
1077            return self.graph_item(item, "form-container");
1078        }
1079        if sref.starts_with("#/field_regions/") {
1080            return self.field_region(item);
1081        }
1082        if sref.starts_with("#/field_items/") {
1083            let entries = vec![self.field_item(item)];
1084            return Self::field_blocks(entries);
1085        }
1086        Vec::new()
1087    }
1088
1089    /// A form's field region (#515): its fields as one `DefinitionList` —
1090    /// key → value(s) — in a `Div .field-region`; a field without a key
1091    /// contributes its blocks in place.
1092    fn field_region(&mut self, region: &'a Value) -> Vec<Value> {
1093        let mut entries = Vec::new();
1094        for cref in Self::children(region) {
1095            let Some(child) = self.resolve(cref) else {
1096                continue;
1097            };
1098            if !self.visited.insert(cref.to_string()) {
1099                continue;
1100            }
1101            if cref.starts_with("#/field_items/") {
1102                entries.push(self.field_item(child));
1103            } else {
1104                let mut blocks = if self.excluded(child) {
1105                    Vec::new()
1106                } else {
1107                    self.serialize(child)
1108                };
1109                blocks.extend(self.blocks_of(child));
1110                entries.push((Vec::new(), blocks));
1111            }
1112        }
1113        let blocks = Self::field_blocks(entries);
1114        if blocks.is_empty() {
1115            Vec::new()
1116        } else {
1117            vec![ast::div(&["field-region"], blocks)]
1118        }
1119    }
1120
1121    /// One field: its `marker` and `field_key` texts as the term, each
1122    /// `field_value` (and anything else under it) as a definition.
1123    fn field_item(&mut self, field: &'a Value) -> (Vec<Value>, Vec<Value>) {
1124        let mut nodes = Vec::new();
1125        self.descendants(field, &mut nodes);
1126        let (mut term, mut defs) = (Vec::new(), Vec::new());
1127        for node in nodes {
1128            if !self.visited.insert(Self::self_ref(node).to_string()) {
1129                continue;
1130            }
1131            match Self::label(node) {
1132                "marker" | "field_key" => {
1133                    let inlines = self.formatted(node);
1134                    if !inlines.is_empty() {
1135                        term = join_inlines(vec![term, inlines]);
1136                    }
1137                }
1138                "field_value" => {
1139                    let inlines = self.formatted(node);
1140                    if !inlines.is_empty() {
1141                        defs.push(ast::plain(inlines));
1142                    }
1143                }
1144                _ => defs.extend(self.serialize(node)),
1145            }
1146        }
1147        (term, defs)
1148    }
1149
1150    /// Fields as blocks: runs of keyed fields become `DefinitionList`s, a
1151    /// key-less field's blocks stand on their own.
1152    fn field_blocks(entries: Vec<(Vec<Value>, Vec<Value>)>) -> Vec<Value> {
1153        let mut out = Vec::new();
1154        let mut run: Vec<(Vec<Value>, Vec<Vec<Value>>)> = Vec::new();
1155        for (term, defs) in entries {
1156            if term.is_empty() {
1157                if !run.is_empty() {
1158                    out.push(ast::definition_list(std::mem::take(&mut run)));
1159                }
1160                out.extend(defs);
1161            } else {
1162                let defs = defs.into_iter().map(|d| vec![d]).collect();
1163                run.push((term, defs));
1164            }
1165        }
1166        if !run.is_empty() {
1167            out.push(ast::definition_list(run));
1168        }
1169        out
1170    }
1171
1172    /// A text item's own inlines: its formatted text, or — for an empty
1173    /// item whose first child is an inline group (a list item, heading or
1174    /// caption whose content is a run of formatted spans; whatever else
1175    /// nests under it — a sub-list, a section's paragraphs — follows as
1176    /// blocks of its own, #833) — that group's, with any blocks the group
1177    /// held besides.
1178    fn own_inlines(&mut self, item: &'a Value) -> (Vec<Value>, Vec<Value>, bool) {
1179        let children = Self::children(item);
1180        if Self::text(item).is_empty() {
1181            if let Some(group) = children
1182                .first()
1183                .and_then(|c| self.resolve(c))
1184                .filter(|c| Self::is_group(c) && Self::label(c) == "inline")
1185            {
1186                self.visited.insert(Self::self_ref(group).to_string());
1187                let (inlines, rest) = self.inline_group(group);
1188                return (inlines, rest, true);
1189            }
1190        }
1191        (self.formatted(item), Vec::new(), false)
1192    }
1193
1194    /// `inlines` between the comment spans of the item's reviewer comments
1195    /// (#836): for each `comment_section` group the item's `comments` refer
1196    /// to, one `Span .comment-start` per comment text — `comment-id` (the
1197    /// group's number, `-n` for a thread's replies), `author`, `date` from
1198    /// docling's note head, the comment's text inside — before the inlines
1199    /// and an empty `Span .comment-end` after them, Pandoc's docx reader's
1200    /// `--track-changes=all` shape. The groups are then visited, so a notes
1201    /// layer that is on does not write them again as blocks.
1202    fn with_comments(&mut self, item: &'a Value, inlines: Vec<Value>) -> Vec<Value> {
1203        let refs = Self::refs(item.get("comments"));
1204        if refs.is_empty() {
1205            return inlines;
1206        }
1207        // An explicit accept / reject view carries no comments (#851),
1208        // as Pandoc's `--track-changes=accept|reject` write none.
1209        if matches!(
1210            self.track_changes,
1211            Some(TrackChanges::Accept | TrackChanges::Reject)
1212        ) {
1213            return inlines;
1214        }
1215        let mut starts = Vec::new();
1216        let mut ends = Vec::new();
1217        for cref in refs {
1218            let Some(group) = self.resolve(cref) else {
1219                continue;
1220            };
1221            if !Self::is_group(group) || Self::label(group) != "comment_section" {
1222                continue;
1223            }
1224            self.visited.insert(cref.to_string());
1225            let id = group
1226                .get("name")
1227                .and_then(Value::as_str)
1228                .and_then(|n| n.strip_prefix("comment-"))
1229                .unwrap_or("0")
1230                .to_string();
1231            // The group's texts sit on the notes layer, which the layer
1232            // filter would drop: read the children directly.
1233            let mut k = 0;
1234            for r in Self::children(group) {
1235                self.visited.insert(r.to_string());
1236                let Some(node) = self.resolve(r).filter(|_| r.starts_with("#/texts/")) else {
1237                    continue;
1238                };
1239                let (author, date, text) = split_comment(Self::text(node));
1240                let comment_id = if k == 0 {
1241                    id.clone()
1242                } else {
1243                    format!("{id}-{k}")
1244                };
1245                k += 1;
1246                let mut attrs = vec![("comment-id".to_string(), comment_id.clone())];
1247                if let Some(a) = author {
1248                    attrs.push(("author".to_string(), a));
1249                }
1250                if let Some(d) = date {
1251                    attrs.push(("date".to_string(), d));
1252                }
1253                starts.push(ast::span(&["comment-start"], &attrs, text_inlines(&text)));
1254                ends.push(ast::span(
1255                    &["comment-end"],
1256                    &[("comment-id".to_string(), comment_id)],
1257                    Vec::new(),
1258                ));
1259            }
1260        }
1261        if starts.is_empty() {
1262            return inlines;
1263        }
1264        let mut out = starts;
1265        out.extend(inlines);
1266        out.extend(ends);
1267        out
1268    }
1269
1270    /// The item's text with its formatting and hyperlink applied, and the
1271    /// notes it calls as `Note`s at their call sites (#538).
1272    fn formatted(&self, item: &Value) -> Vec<Value> {
1273        let base = match Self::label(item) {
1274            "code" => vec![ast::code(Self::text(item))],
1275            "formula" if !Self::text(item).is_empty() => {
1276                vec![ast::math(false, Self::text(item).trim())]
1277            }
1278            _ => {
1279                let (text, skipped) = Self::authored_text(item);
1280                match item.get("_notes").and_then(Value::as_array) {
1281                    Some(notes) if !notes.is_empty() => {
1282                        if skipped == 0 {
1283                            text_with_notes(text, notes)
1284                        } else {
1285                            // The note offsets count the numbered text.
1286                            let shifted: Vec<Value> = notes
1287                                .iter()
1288                                .map(|n| {
1289                                    let off = n[0].as_u64().unwrap_or(0).saturating_sub(skipped);
1290                                    serde_json::json!([off, n[1]])
1291                                })
1292                                .collect();
1293                            text_with_notes(text, &shifted)
1294                        }
1295                    }
1296                    _ => text_inlines(text),
1297                }
1298            }
1299        };
1300        Self::decorate(base, item)
1301    }
1302
1303    /// The item's text without the generated numbering it opens with
1304    /// (`_number`, #850), and how many chars were dropped.
1305    fn authored_text(item: &Value) -> (&str, u64) {
1306        let text = Self::text(item);
1307        if let Some(number) = item.get("_number").and_then(Value::as_str) {
1308            if let Some(rest) = text
1309                .strip_prefix(number)
1310                .and_then(|rest| rest.strip_prefix(' '))
1311            {
1312                return (rest, (number.chars().count() + 1) as u64);
1313            }
1314        }
1315        (text, 0)
1316    }
1317
1318    /// A furniture `footnote` item that is the body of a note a text item
1319    /// calls ([`crate::tree::TreeItem::note_body`]).
1320    fn is_note_body(item: &Value) -> bool {
1321        item.get("_note_body").and_then(Value::as_bool) == Some(true)
1322    }
1323
1324    fn decorate(mut inlines: Vec<Value>, item: &Value) -> Vec<Value> {
1325        if inlines.is_empty() {
1326            return inlines;
1327        }
1328        if let Some(f) = item.get("formatting") {
1329            let on = |k: &str| f.get(k).and_then(Value::as_bool).unwrap_or(false);
1330            if on("bold") {
1331                inlines = vec![ast::wrap("Strong", inlines)];
1332            }
1333            if on("italic") {
1334                inlines = vec![ast::wrap("Emph", inlines)];
1335            }
1336            if on("underline") {
1337                inlines = vec![ast::wrap("Underline", inlines)];
1338            }
1339            if on("strikethrough") {
1340                inlines = vec![ast::wrap("Strikeout", inlines)];
1341            }
1342            match f.get("script").and_then(Value::as_str) {
1343                Some("sub") => inlines = vec![ast::wrap("Subscript", inlines)],
1344                Some("super") => inlines = vec![ast::wrap("Superscript", inlines)],
1345                _ => {}
1346            }
1347        }
1348        if let Some(url) = item.get("hyperlink").and_then(Value::as_str) {
1349            inlines = vec![ast::link(inlines, url)];
1350        }
1351        // A tracked change kept under `all` (#851): Pandoc's docx reader's
1352        // span, outermost.
1353        if let Some(rev) = item.get("_revision") {
1354            if let Some(kind) = rev.get("kind").and_then(Value::as_str) {
1355                let mut attrs = Vec::new();
1356                for key in ["author", "date"] {
1357                    if let Some(v) = rev.get(key).and_then(Value::as_str) {
1358                        attrs.push((key.to_string(), v.to_string()));
1359                    }
1360                }
1361                inlines = vec![ast::span(&[kind], &attrs, inlines)];
1362            }
1363        }
1364        inlines
1365    }
1366
1367    /// An inline group's runs as inlines joined by `Space`; anything in it
1368    /// that has no inline form (a nested list, a table) comes back as
1369    /// blocks for after the paragraph.
1370    fn inline_group(&mut self, group: &'a Value) -> (Vec<Value>, Vec<Value>) {
1371        let mut nodes = Vec::new();
1372        self.descendants(group, &mut nodes);
1373        let mut runs: Vec<Vec<Value>> = Vec::new();
1374        let mut rest: Vec<Value> = Vec::new();
1375        for node in nodes {
1376            let r = Self::self_ref(node).to_string();
1377            if self.visited.contains(&r) {
1378                continue;
1379            }
1380            self.visited.insert(r.clone());
1381            if r.starts_with("#/texts/") {
1382                if self.owned_captions.contains(&r)
1383                    || self.owned_footnotes.contains(&r)
1384                    || Self::is_note_body(node)
1385                {
1386                    continue;
1387                }
1388                let mut run: Vec<Value> = Self::anchors(node)
1389                    .iter()
1390                    .map(|id| ast::anchor(id))
1391                    .collect();
1392                run.append(&mut self.formatted(node));
1393                runs.push(run);
1394                continue;
1395            }
1396            if Self::is_group(node) && Self::label(node) == "inline" {
1397                let (inl, mut more) = self.inline_group(node);
1398                runs.push(inl);
1399                rest.append(&mut more);
1400                continue;
1401            }
1402            rest.extend(self.serialize(node));
1403        }
1404        (join_inlines(runs), rest)
1405    }
1406
1407    fn text_item(&mut self, item: &'a Value) -> Vec<Value> {
1408        let label = Self::label(item);
1409        let mut out = Vec::new();
1410        match label {
1411            "list_item" => {
1412                // A list item met outside a list group: a one-item list.
1413                let blocks = self.list_item_blocks(item);
1414                if !blocks.is_empty() {
1415                    out.push(ast::bullet_list(vec![blocks]));
1416                }
1417                return out;
1418            }
1419            "code" => {
1420                let lang = item
1421                    .get("code_language")
1422                    .and_then(Value::as_str)
1423                    .filter(|l| !l.is_empty() && *l != "unknown")
1424                    .map(str::to_ascii_lowercase);
1425                out.push(ast::code_block(lang.as_deref(), Self::text(item)));
1426            }
1427            "formula" => {
1428                let tex = Self::text(item).trim();
1429                if !tex.is_empty() {
1430                    out.push(ast::para(vec![ast::math(true, tex)]));
1431                }
1432            }
1433            _ => {
1434                let (inlines, rest, _) = self.own_inlines(item);
1435                let mut inlines = self.with_comments(item, inlines);
1436                // The item's link destinations (#849): a heading takes
1437                // the first as its identifier, the rest open its inlines
1438                // as anchor spans; any other item opens with them all.
1439                let mut anchors = Self::anchors(item);
1440                let header_id =
1441                    if matches!(label, "title" | "section_header") && !anchors.is_empty() {
1442                        Some(anchors.remove(0))
1443                    } else {
1444                        None
1445                    };
1446                if !anchors.is_empty() && (header_id.is_some() || !inlines.is_empty()) {
1447                    let mut with_anchors: Vec<Value> =
1448                        anchors.iter().map(|id| ast::anchor(id)).collect();
1449                    with_anchors.append(&mut inlines);
1450                    inlines = with_anchors;
1451                }
1452                match label {
1453                    "title" | "section_header" => {
1454                        let level = if label == "title" {
1455                            1
1456                        } else {
1457                            let n = item.get("level").and_then(Value::as_u64).unwrap_or(1) as usize;
1458                            match self.heading_depth {
1459                                // docling's depth: level 1 is `##`, under
1460                                // the title's `#`.
1461                                HeadingDepth::Docling => (n + 1).min(6),
1462                                // The source's depth (#848): a DOCX
1463                                // `Heading 1` is `Header 1`, as Pandoc's
1464                                // docx reader writes it.
1465                                HeadingDepth::Source => n.clamp(1, 6),
1466                            }
1467                        };
1468                        if !inlines.is_empty() {
1469                            out.push(match &header_id {
1470                                Some(id) => ast::header_with_id(level, id, inlines),
1471                                None => ast::header(level, inlines),
1472                            });
1473                        }
1474                    }
1475                    "text" | "paragraph" => {
1476                        if !inlines.is_empty() {
1477                            out.push(ast::para(inlines));
1478                        }
1479                    }
1480                    "checkbox_selected" | "checkbox_unselected" => {
1481                        let mark = if label == "checkbox_selected" {
1482                            "☒"
1483                        } else {
1484                            "☐"
1485                        };
1486                        let mut with_mark = vec![ast::str_(mark)];
1487                        if !inlines.is_empty() {
1488                            with_mark.push(ast::space());
1489                            with_mark.append(&mut inlines);
1490                        }
1491                        out.push(ast::para(with_mark));
1492                    }
1493                    other => {
1494                        if !inlines.is_empty() {
1495                            let class = format!("docling-{other}");
1496                            out.push(ast::div(&[&class], vec![ast::para(inlines)]));
1497                        }
1498                    }
1499                }
1500                out.extend(rest);
1501            }
1502        }
1503        // Content nested under the item (an HTML / DOCX heading's section).
1504        out.extend(self.blocks_of(item));
1505        out
1506    }
1507
1508    /// A list item's blocks: its own text as `Plain`, then whatever is
1509    /// nested under it (sub-lists, paragraphs).
1510    fn list_item_blocks(&mut self, item: &'a Value) -> Vec<Value> {
1511        let (inlines, rest, _) = self.own_inlines(item);
1512        let inlines = self.with_comments(item, inlines);
1513        let mut blocks = Vec::new();
1514        if !inlines.is_empty() {
1515            blocks.push(ast::plain(inlines));
1516        }
1517        blocks.extend(rest);
1518        blocks.extend(self.blocks_of(item));
1519        blocks
1520    }
1521
1522    fn list_group(&mut self, group: &'a Value) -> Vec<Value> {
1523        let first = Self::children(group)
1524            .first()
1525            .and_then(|r| self.resolve(r))
1526            .filter(|c| Self::label(c) == "list_item");
1527        let enumerated = first.is_some_and(|c| {
1528            c.get("enumerated")
1529                .and_then(Value::as_bool)
1530                .unwrap_or(false)
1531        });
1532        // The items' markers decide the list's start, number style and
1533        // delimiter (#835).
1534        let markers: Vec<&str> = Self::children(group)
1535            .iter()
1536            .filter_map(|r| self.resolve(r))
1537            .filter(|c| Self::label(c) == "list_item")
1538            .filter_map(|c| c.get("marker").and_then(Value::as_str))
1539            .collect();
1540        let (start, style, delim) = list_attributes(&markers);
1541
1542        let mut nodes = Vec::new();
1543        self.descendants(group, &mut nodes);
1544        let mut items: Vec<Vec<Value>> = Vec::new();
1545        for node in nodes {
1546            let r = Self::self_ref(node).to_string();
1547            if !self.visited.insert(r.clone()) {
1548                continue;
1549            }
1550            if Self::label(node) == "list_item"
1551                && r.starts_with("#/texts/")
1552                && !self.owned_captions.contains(&r)
1553            {
1554                let blocks = self.list_item_blocks(node);
1555                items.push(blocks);
1556                continue;
1557            }
1558            // Something that is not a list item directly in the list (a
1559            // paragraph between items): it belongs to the item before it.
1560            let blocks = self.serialize(node);
1561            if blocks.is_empty() {
1562                continue;
1563            }
1564            match items.last_mut() {
1565                Some(last) => last.extend(blocks),
1566                None => items.push(blocks),
1567            }
1568        }
1569        if items.is_empty() {
1570            return Vec::new();
1571        }
1572        if enumerated {
1573            vec![ast::ordered_list(start, style, delim, items)]
1574        } else {
1575            vec![ast::bullet_list(items)]
1576        }
1577    }
1578
1579    /// The owner's caption texts as one `Caption`, with its footnotes as
1580    /// `Note`s at the end — the float is their call site. Each caption and
1581    /// note keeps its formatting, links and inline group (#833: a bold,
1582    /// linked `<caption>` used to come out as bare words, a multi-run one
1583    /// as an empty caption plus a stray `Para`).
1584    fn caption(&mut self, item: &'a Value) -> Value {
1585        let mut inlines = self.caption_text(item);
1586        for cref in Self::refs(item.get("footnotes")) {
1587            if let Some(note) = self.resolve(cref) {
1588                if cref.starts_with("#/texts/") && !self.excluded(note) {
1589                    let (text, _, _) = self.own_inlines(note);
1590                    if !text.is_empty() {
1591                        inlines.push(ast::note(vec![ast::para(text)]));
1592                    }
1593                }
1594            }
1595        }
1596        if inlines.is_empty() {
1597            ast::caption(Vec::new())
1598        } else {
1599            ast::caption(vec![ast::plain(inlines)])
1600        }
1601    }
1602
1603    /// The owner's caption texts alone (no notes) — a picture's alt text,
1604    /// as Pandoc's readers give a captioned image.
1605    fn caption_text(&mut self, item: &'a Value) -> Vec<Value> {
1606        let caps: Vec<&'a Value> = Self::refs(item.get("captions"))
1607            .into_iter()
1608            .filter(|cref| cref.starts_with("#/texts/"))
1609            .filter_map(|cref| self.resolve(cref))
1610            .filter(|cap| !self.excluded(cap))
1611            .collect();
1612        let mut parts = Vec::new();
1613        for cap in caps {
1614            let (inlines, _, _) = self.own_inlines(cap);
1615            parts.push(inlines);
1616        }
1617        join_inlines(parts)
1618    }
1619
1620    /// docling's `TableData.grid`: each position → the cell covering it.
1621    fn grid(data: &'a Value) -> (usize, Vec<Vec<Option<GridCell<'a>>>>) {
1622        let num_rows = data.get("num_rows").and_then(Value::as_u64).unwrap_or(0) as usize;
1623        let num_cols = data.get("num_cols").and_then(Value::as_u64).unwrap_or(0) as usize;
1624        let mut grid: Vec<Vec<Option<GridCell<'a>>>> = (0..num_rows)
1625            .map(|_| (0..num_cols).map(|_| None).collect())
1626            .collect();
1627        for cell in Self::array(data.get("table_cells")) {
1628            let get = |k: &str| cell.get(k).and_then(Value::as_u64).unwrap_or(0) as usize;
1629            let (r0, c0) = (get("start_row_offset_idx"), get("start_col_offset_idx"));
1630            let (r1, c1) = (get("end_row_offset_idx"), get("end_col_offset_idx"));
1631            if r0 >= num_rows || c0 >= num_cols {
1632                continue;
1633            }
1634            let (r1, c1) = (r1.clamp(r0 + 1, num_rows), c1.clamp(c0 + 1, num_cols));
1635            for row in grid.iter_mut().take(r1).skip(r0) {
1636                for slot in row.iter_mut().take(c1).skip(c0) {
1637                    *slot = Some(GridCell {
1638                        cell,
1639                        start_row: r0,
1640                        start_col: c0,
1641                        row_span: r1 - r0,
1642                        col_span: c1 - c0,
1643                    });
1644                }
1645            }
1646        }
1647        (num_cols, grid)
1648    }
1649
1650    fn table_from_data(&mut self, data: &'a Value, caption: Value) -> Option<Value> {
1651        let (num_cols, grid) = Self::grid(data);
1652        if grid.is_empty() || num_cols == 0 {
1653            return None;
1654        }
1655        let flag = |c: &Value, k: &str| c.get(k).and_then(Value::as_bool).unwrap_or(false);
1656        let mut rows: Vec<(bool, Value)> = Vec::new();
1657        for (i, row) in grid.iter().enumerate() {
1658            let mut cells = Vec::new();
1659            let mut all_header = true;
1660            let mut anchored = 0;
1661            for (j, slot) in row.iter().enumerate() {
1662                let Some(g) = slot else {
1663                    // A position no cell covers (a table's empty corner) is
1664                    // neutral to the header test.
1665                    cells.push(ast::cell(1, 1, Vec::new()));
1666                    continue;
1667                };
1668                if g.start_row != i || g.start_col != j {
1669                    continue;
1670                }
1671                anchored += 1;
1672                all_header &= flag(g.cell, "column_header");
1673                let blocks = match g
1674                    .cell
1675                    .get("ref")
1676                    .and_then(|r| r.get("$ref"))
1677                    .and_then(Value::as_str)
1678                {
1679                    Some(cref) => match self.resolve(cref) {
1680                        Some(target) => {
1681                            self.visited.insert(cref.to_string());
1682                            self.serialize(target)
1683                        }
1684                        None => Vec::new(),
1685                    },
1686                    None => {
1687                        let inl =
1688                            text_inlines(g.cell.get("text").and_then(Value::as_str).unwrap_or(""));
1689                        if inl.is_empty() {
1690                            Vec::new()
1691                        } else {
1692                            vec![ast::plain(inl)]
1693                        }
1694                    }
1695                };
1696                cells.push(ast::cell(g.row_span, g.col_span, blocks));
1697            }
1698            rows.push((all_header && anchored > 0, ast::row(cells)));
1699        }
1700        // Leading all-header rows form the head; the rest is the body.
1701        let head_len = rows.iter().take_while(|(h, _)| *h).count();
1702        let head_len = if head_len == rows.len() { 0 } else { head_len };
1703        let mut head = Vec::new();
1704        let mut body = Vec::new();
1705        for (i, (_, row)) in rows.into_iter().enumerate() {
1706            if i < head_len {
1707                head.push(row);
1708            } else {
1709                body.push(row);
1710            }
1711        }
1712        Some(ast::table(caption, num_cols, head, body))
1713    }
1714
1715    fn table(&mut self, item: &'a Value) -> Vec<Value> {
1716        let caption = self.caption(item);
1717        if self.excluded(item) {
1718            return Vec::new();
1719        }
1720        match item.get("data") {
1721            Some(data) => self.table_from_data(data, caption).into_iter().collect(),
1722            None => Vec::new(),
1723        }
1724    }
1725
1726    fn picture(&mut self, item: &'a Value) -> Vec<Value> {
1727        if self.excluded(item) {
1728            return Vec::new();
1729        }
1730        let caption = self.caption(item);
1731        let has_caption = caption
1732            .get(1)
1733            .and_then(Value::as_array)
1734            .is_some_and(|b| !b.is_empty());
1735        let uri = item
1736            .get("image")
1737            .and_then(|i| i.get("uri"))
1738            .and_then(Value::as_str);
1739        let src = match (self.image_mode, uri) {
1740            (ImageMode::Embedded, Some(uri)) if uri.starts_with("data:") => Some(uri.to_string()),
1741            (ImageMode::Referenced, Some(uri)) => self
1742                .reference_image(uri)
1743                .or_else(|| (!uri.starts_with("data:")).then(|| uri.to_string())),
1744            _ => None,
1745        };
1746        // #537: every picture is an `Image`, so it survives into whatever
1747        // Pandoc writes and can be counted — one without a target
1748        // (placeholder mode, or a payload docling could not decode: EMF/WMF)
1749        // carries the `docling-placeholder` class and an empty target, which
1750        // a writer reports as a missing resource and renders as the alt text.
1751        // The alt text is the author's (`meta.description`, #834) — what
1752        // Pandoc's docx reader takes from `wp:docPr descr` — and only
1753        // without one the caption, as before.
1754        let description = item
1755            .get("meta")
1756            .and_then(|m| m.get("description"))
1757            .and_then(|d| d.get("text"))
1758            .and_then(Value::as_str)
1759            .map(text_inlines)
1760            .filter(|t| !t.is_empty());
1761        let alt = match description {
1762            Some(alt) => alt,
1763            None => self.caption_text(item),
1764        };
1765        // The displayed size (`_display`, inches) as `width` / `height`
1766        // attributes, Pandoc's readers' shape for a sized image (#834).
1767        let attrs: Vec<(String, String)> = item
1768            .get("image")
1769            .and_then(|i| i.get("_display"))
1770            .and_then(Value::as_array)
1771            .and_then(|d| Some((d.first()?.as_f64()?, d.get(1)?.as_f64()?)))
1772            .map(|(w, h)| {
1773                vec![
1774                    ("width".to_string(), inches(w)),
1775                    ("height".to_string(), inches(h)),
1776                ]
1777            })
1778            .unwrap_or_default();
1779        let image = match &src {
1780            Some(src) => ast::image_sized(&[], &attrs, alt, src),
1781            None => ast::image_sized(&["docling-placeholder"], &attrs, alt, ""),
1782        };
1783        // A native chart's data grid (`meta.tabular_chart.chart_data`).
1784        let chart = item
1785            .get("meta")
1786            .and_then(|m| m.get("tabular_chart"))
1787            .and_then(|t| t.get("chart_data"))
1788            .and_then(|chart| self.table_from_data(chart, ast::caption(Vec::new())));
1789        if chart.is_none() && !has_caption {
1790            // Pandoc's own readers put a caption-less image in a paragraph.
1791            return vec![ast::para(vec![image])];
1792        }
1793        let mut blocks = vec![ast::plain(vec![image])];
1794        blocks.extend(chart);
1795        vec![ast::figure(&[], caption, blocks)]
1796    }
1797
1798    /// Referenced mode: a `data:` URI becomes an artifact file named like
1799    /// the Markdown export's; the path is the image target.
1800    fn reference_image(&mut self, uri: &str) -> Option<String> {
1801        let rest = uri.strip_prefix("data:")?;
1802        let (mime, payload) = rest.split_once(";base64,")?;
1803        let bytes = crate::base64::decode(payload)?;
1804        let path = format!(
1805            "{}/image_{:06}.{}",
1806            self.artifacts_dir,
1807            self.pic_index,
1808            crate::markdown::ext_for(mime)
1809        );
1810        self.pic_index += 1;
1811        self.artifacts.push((path.clone(), bytes));
1812        Some(path)
1813    }
1814
1815    /// A key-value / form graph: a `DefinitionList` of keys and their
1816    /// values, or — when `to_child` links make it a hierarchy — the nested
1817    /// `BulletList` of it (the HTML export's two shapes), in a classed `Div`
1818    /// with the item's caption after it.
1819    fn graph_item(&mut self, item: &'a Value, class: &str) -> Vec<Value> {
1820        let mut out = Vec::new();
1821        if !self.excluded(item) {
1822            if let Some(graph) = item.get("graph") {
1823                if let Some(block) = Self::graph(graph) {
1824                    out.push(ast::div(&[class], vec![block]));
1825                }
1826            }
1827        }
1828        // The caption (and footnote notes) as a paragraph after the region.
1829        let caption = self.caption(item);
1830        for block in caption
1831            .get(1)
1832            .and_then(Value::as_array)
1833            .into_iter()
1834            .flatten()
1835        {
1836            if let Some(inlines) = block.get("c").and_then(Value::as_array) {
1837                out.push(ast::para(inlines.clone()));
1838            }
1839        }
1840        out
1841    }
1842
1843    fn graph(graph: &'a Value) -> Option<Value> {
1844        let cells: HashMap<u64, &Value> = Self::array(graph.get("cells"))
1845            .filter_map(|c| Some((c.get("cell_id")?.as_u64()?, c)))
1846            .collect();
1847        let order: Vec<u64> = Self::array(graph.get("cells"))
1848            .filter_map(|c| c.get("cell_id")?.as_u64())
1849            .collect();
1850        let mut child_links: HashMap<u64, Vec<u64>> = HashMap::new();
1851        let mut value_links: BTreeMap<u64, Vec<u64>> = BTreeMap::new();
1852        let mut value_order: Vec<u64> = Vec::new();
1853        let mut parents: HashSet<u64> = HashSet::new();
1854        for link in Self::array(graph.get("links")) {
1855            let (Some(s), Some(t)) = (
1856                link.get("source_cell_id").and_then(Value::as_u64),
1857                link.get("target_cell_id").and_then(Value::as_u64),
1858            ) else {
1859                continue;
1860            };
1861            if !cells.contains_key(&s) || !cells.contains_key(&t) {
1862                continue;
1863            }
1864            match link.get("label").and_then(Value::as_str) {
1865                Some("to_child") => {
1866                    child_links.entry(s).or_default().push(t);
1867                    parents.insert(t);
1868                }
1869                Some("to_value") => {
1870                    if !value_links.contains_key(&s) {
1871                        value_order.push(s);
1872                    }
1873                    value_links.entry(s).or_default().push(t);
1874                }
1875                _ => {}
1876            }
1877        }
1878        let cell_text =
1879            |id: u64| text_inlines(cells[&id].get("text").and_then(Value::as_str).unwrap_or(""));
1880        if child_links.is_empty() {
1881            if value_order.is_empty() {
1882                return None;
1883            }
1884            let entries = value_order
1885                .iter()
1886                .map(|&k| {
1887                    let defs = value_links[&k]
1888                        .iter()
1889                        .map(|&v| vec![ast::plain(cell_text(v))])
1890                        .collect();
1891                    (cell_text(k), defs)
1892                })
1893                .collect();
1894            return Some(ast::definition_list(entries));
1895        }
1896        // Hierarchical: a nested bullet list, cycles cut on the descent path.
1897        fn node(
1898            id: u64,
1899            cell_text: &dyn Fn(u64) -> Vec<Value>,
1900            child_links: &HashMap<u64, Vec<u64>>,
1901            value_links: &BTreeMap<u64, Vec<u64>>,
1902            path: &mut HashSet<u64>,
1903        ) -> Option<Vec<Value>> {
1904            if !path.insert(id) {
1905                return None;
1906            }
1907            let mut inlines = cell_text(id);
1908            if let Some(values) = value_links.get(&id) {
1909                let vals = join_inlines(values.iter().map(|&v| cell_text(v)).collect());
1910                inlines = vec![ast::wrap("Strong", inlines)];
1911                inlines.push(ast::str_(":"));
1912                if !vals.is_empty() {
1913                    inlines.push(ast::space());
1914                    inlines.extend(vals);
1915                }
1916            }
1917            let mut blocks = vec![ast::plain(inlines)];
1918            let kids: Vec<Vec<Value>> = child_links
1919                .get(&id)
1920                .into_iter()
1921                .flatten()
1922                .filter_map(|&c| node(c, cell_text, child_links, value_links, path))
1923                .collect();
1924            if !kids.is_empty() {
1925                blocks.push(ast::bullet_list(kids));
1926            }
1927            path.remove(&id);
1928            Some(blocks)
1929        }
1930        let mut path = HashSet::new();
1931        let items: Vec<Vec<Value>> = order
1932            .iter()
1933            .filter(|id| !parents.contains(id))
1934            .filter_map(|&id| node(id, &cell_text, &child_links, &value_links, &mut path))
1935            .collect();
1936        (!items.is_empty()).then(|| ast::bullet_list(items))
1937    }
1938}
1939
1940#[cfg(test)]
1941mod tests {
1942    use super::*;
1943    use serde_json::json;
1944
1945    /// A docling JSON document from `body` children and the item buckets.
1946    fn doc(body: &[&str], texts: Value, groups: Value, tables: Value) -> Value {
1947        json!({
1948            "schema_name": "DoclingDocument",
1949            "name": "t",
1950            "body": {"self_ref": "#/body", "children": body.iter().map(|r| json!({"$ref": r})).collect::<Vec<_>>()},
1951            "texts": texts, "groups": groups, "tables": tables, "pictures": []
1952        })
1953    }
1954
1955    fn text(i: usize, label: &str, text: &str, extra: Value) -> Value {
1956        let mut v = json!({"self_ref": format!("#/texts/{i}"), "children": [], "label": label, "text": text, "content_layer": "body"});
1957        for (k, x) in extra.as_object().unwrap() {
1958            v[k] = x.clone();
1959        }
1960        v
1961    }
1962
1963    fn blocks(json: &Value) -> Vec<Value> {
1964        let (s, _) = from_docling_json(json, &PandocExportOptions::default()).unwrap();
1965        let v: Value = serde_json::from_str(&s).unwrap();
1966        assert_eq!(v["pandoc-api-version"], json!(PANDOC_API_VERSION));
1967        v["blocks"].as_array().unwrap().clone()
1968    }
1969
1970    /// #538: a text item's note calls become `Note`s at their offsets —
1971    /// glued to the word before, a `Space` where the text had one — and the
1972    /// note bodies they carry are not written again as blocks.
1973    #[test]
1974    fn note_calls_become_notes_at_their_call_sites() {
1975        let j = doc(
1976            &["#/texts/0", "#/texts/1"],
1977            json!([
1978                text(
1979                    0,
1980                    "text",
1981                    "debut suite end",
1982                    json!({"_notes": [[5, "first"], [15, "last"]]})
1983                ),
1984                text(
1985                    1,
1986                    "footnote",
1987                    "first",
1988                    json!({"content_layer": "furniture", "_note_body": true})
1989                ),
1990            ]),
1991            json!([]),
1992            json!([]),
1993        );
1994        let note = |t: &str| ast::note(vec![ast::para(vec![ast::str_(t)])]);
1995        assert_eq!(
1996            blocks(&j),
1997            vec![ast::para(vec![
1998                ast::str_("debut"),
1999                note("first"),
2000                ast::space(),
2001                ast::str_("suite"),
2002                ast::space(),
2003                ast::str_("end"),
2004                note("last"),
2005            ])]
2006        );
2007        // With the furniture layer on, a placed note body is still not a block.
2008        let opts = PandocExportOptions {
2009            layers: ContentLayers::ALL,
2010            ..Default::default()
2011        };
2012        let (s, _) = from_docling_json(&j, &opts).unwrap();
2013        assert!(!s.contains("docling-footnote"), "{s}");
2014    }
2015
2016    /// #538: a furniture footnote nothing calls (Python docling's JSON has no
2017    /// call sites) still reaches the AST, as a trailing `Note`.
2018    #[test]
2019    fn uncalled_furniture_footnotes_trail_as_notes() {
2020        let j = doc(
2021            &["#/texts/0", "#/texts/1"],
2022            json!([
2023                text(0, "text", "body", json!({})),
2024                text(
2025                    1,
2026                    "footnote",
2027                    "MARKFN",
2028                    json!({"content_layer": "furniture"})
2029                ),
2030            ]),
2031            json!([]),
2032            json!([]),
2033        );
2034        assert_eq!(
2035            blocks(&j),
2036            vec![
2037                ast::para(vec![ast::str_("body")]),
2038                ast::para(vec![ast::note(vec![ast::para(vec![ast::str_("MARKFN")])])]),
2039            ]
2040        );
2041    }
2042
2043    /// #537: every picture is an `Image` — embedded by default, in a `Para`
2044    /// without a caption, in a `Figure` (caption = alt text) with one; with
2045    /// no target (placeholder mode, an undecodable payload) it is classed
2046    /// `docling-placeholder` instead of vanishing as raw HTML.
2047    #[test]
2048    fn pictures_are_always_images() {
2049        let mut j = doc(
2050            &["#/pictures/0", "#/pictures/1", "#/pictures/2"],
2051            json!([text(0, "caption", "A duck", json!({})),]),
2052            json!([]),
2053            json!([]),
2054        );
2055        j["pictures"] = json!([
2056            {"self_ref": "#/pictures/0", "children": [], "label": "picture", "content_layer": "body",
2057             "image": {"uri": "data:image/png;base64,AAAA"}, "captions": []},
2058            {"self_ref": "#/pictures/1", "children": [], "label": "picture", "content_layer": "body",
2059             "image": {"uri": "data:image/png;base64,BBBB"}, "captions": [{"$ref": "#/texts/0"}]},
2060            {"self_ref": "#/pictures/2", "children": [], "label": "picture", "content_layer": "body",
2061             "captions": []},
2062        ]);
2063        let b = blocks(&j);
2064        assert_eq!(
2065            b[0],
2066            ast::para(vec![ast::image(&[], vec![], "data:image/png;base64,AAAA")])
2067        );
2068        assert_eq!(
2069            b[1],
2070            ast::figure(
2071                &[],
2072                ast::caption(vec![ast::plain(vec![
2073                    ast::str_("A"),
2074                    ast::space(),
2075                    ast::str_("duck")
2076                ])]),
2077                vec![ast::plain(vec![ast::image(
2078                    &[],
2079                    vec![ast::str_("A"), ast::space(), ast::str_("duck")],
2080                    "data:image/png;base64,BBBB"
2081                )])]
2082            )
2083        );
2084        assert_eq!(
2085            b[2],
2086            ast::para(vec![ast::image(&["docling-placeholder"], vec![], "")])
2087        );
2088        assert_eq!(b.len(), 3);
2089        // Placeholder mode keeps the node, without the pixels.
2090        let opts = PandocExportOptions {
2091            image_mode: ImageMode::Placeholder,
2092            ..Default::default()
2093        };
2094        let (s, _) = from_docling_json(&j, &opts).unwrap();
2095        assert!(!s.contains("base64") && !s.contains("RawBlock"), "{s}");
2096        assert_eq!(s.matches("docling-placeholder").count(), 3, "{s}");
2097    }
2098
2099    /// #833: a caption of formatted runs (an inline group under the caption
2100    /// item) is the table's caption, its runs decorated, and nothing of it
2101    /// strays into the body; a multi-run heading with a section under it
2102    /// is a `Header` followed by the section's blocks; a multi-run list
2103    /// item with a nested list is `[Plain, BulletList]`; a formatted,
2104    /// linked single-run caption keeps `Strong` and `Link`.
2105    #[test]
2106    fn captions_headings_and_items_keep_their_inline_groups() {
2107        let mut j = doc(
2108            &["#/texts/0", "#/tables/0", "#/texts/3", "#/groups/2"],
2109            json!([
2110                text(
2111                    0,
2112                    "caption",
2113                    "",
2114                    json!({"children": [{"$ref": "#/groups/0"}]})
2115                ),
2116                text(1, "text", "Table 1:", json!({"formatting": {"bold": true}})),
2117                text(
2118                    2,
2119                    "text",
2120                    "Multi-run caption",
2121                    json!({"hyperlink": "https://x.org/c"})
2122                ),
2123                text(
2124                    3,
2125                    "section_header",
2126                    "",
2127                    json!({"level": 1, "children": [{"$ref": "#/groups/1"}, {"$ref": "#/texts/6"}]})
2128                ),
2129                text(4, "text", "1.1", json!({"formatting": {"bold": true}})),
2130                text(5, "text", "Formatted header", json!({})),
2131                text(6, "text", "Body under header.", json!({})),
2132                text(
2133                    7,
2134                    "list_item",
2135                    "",
2136                    json!({"children": [{"$ref": "#/groups/3"}, {"$ref": "#/groups/4"}]})
2137                ),
2138                text(8, "text", "Bold run", json!({"formatting": {"bold": true}})),
2139                text(9, "text", "and tail", json!({})),
2140                text(10, "list_item", "Nested child", json!({})),
2141                text(
2142                    11,
2143                    "caption",
2144                    "Linked bold",
2145                    json!({"formatting": {"bold": true}, "hyperlink": "https://x.org/spec"})
2146                ),
2147            ]),
2148            json!([
2149                {"self_ref": "#/groups/0", "children": [{"$ref": "#/texts/1"}, {"$ref": "#/texts/2"}], "label": "inline", "name": "group"},
2150                {"self_ref": "#/groups/1", "children": [{"$ref": "#/texts/4"}, {"$ref": "#/texts/5"}], "label": "inline", "name": "group"},
2151                {"self_ref": "#/groups/2", "children": [{"$ref": "#/texts/7"}], "label": "list", "name": "list"},
2152                {"self_ref": "#/groups/3", "children": [{"$ref": "#/texts/8"}, {"$ref": "#/texts/9"}], "label": "inline", "name": "group"},
2153                {"self_ref": "#/groups/4", "children": [{"$ref": "#/texts/10"}], "label": "list", "name": "list"},
2154            ]),
2155            json!([{"self_ref": "#/tables/0", "children": [], "label": "table",
2156                    "captions": [{"$ref": "#/texts/0"}, {"$ref": "#/texts/11"}],
2157                    "data": {"num_rows": 1, "num_cols": 1, "table_cells": [
2158                        {"text": "v", "row_span": 1, "col_span": 1, "start_row_offset_idx": 0, "end_row_offset_idx": 1,
2159                         "start_col_offset_idx": 0, "end_col_offset_idx": 1, "column_header": false}]}}]),
2160        );
2161        j["texts"][0]["children"] = json!([{"$ref": "#/groups/0"}]);
2162        let b = blocks(&j);
2163        assert_eq!(b.len(), 4, "{b:?}");
2164        assert_eq!(b[0]["t"], "Table");
2165        let caption = &b[0]["c"][1][1][0];
2166        assert_eq!(caption["t"], "Plain");
2167        assert_eq!(
2168            caption["c"][0],
2169            ast::wrap(
2170                "Strong",
2171                vec![ast::str_("Table"), ast::space(), ast::str_("1:")]
2172            )
2173        );
2174        assert_eq!(caption["c"][2]["t"], "Link");
2175        let linked = caption["c"].as_array().unwrap().last().unwrap();
2176        assert_eq!(linked["t"], "Link");
2177        assert_eq!(linked["c"][1][0]["t"], "Strong");
2178        assert_eq!(b[1]["t"], "Header");
2179        assert_eq!(b[1]["c"][2][0], ast::wrap("Strong", vec![ast::str_("1.1")]));
2180        assert_eq!(
2181            b[2],
2182            ast::para(vec![
2183                ast::str_("Body"),
2184                ast::space(),
2185                ast::str_("under"),
2186                ast::space(),
2187                ast::str_("header.")
2188            ])
2189        );
2190        assert_eq!(b[3]["t"], "BulletList");
2191        let item = b[3]["c"][0].as_array().unwrap();
2192        assert_eq!(item.len(), 2);
2193        assert_eq!(item[0]["t"], "Plain");
2194        assert_eq!(item[0]["c"][0]["t"], "Strong");
2195        assert_eq!(item[1]["t"], "BulletList");
2196    }
2197
2198    /// #834: the `Image`'s alt text is the picture's description (the
2199    /// author's alt text) when it has one, the caption only without; the
2200    /// displayed size (`_display`, inches) becomes `width` / `height`.
2201    #[test]
2202    fn pictures_carry_alt_text_and_display_size() {
2203        let mut j = doc(
2204            &["#/pictures/0", "#/pictures/1"],
2205            json!([text(0, "caption", "A duck", json!({})),]),
2206            json!([]),
2207            json!([]),
2208        );
2209        j["pictures"] = json!([
2210            {"self_ref": "#/pictures/0", "children": [], "label": "picture", "content_layer": "body",
2211             "meta": {"description": {"text": "Architecture diagram", "created_by": "alt_text"}},
2212             "image": {"uri": "data:image/png;base64,AAAA", "_display": [2.5, 1.25]}, "captions": [{"$ref": "#/texts/0"}]},
2213            {"self_ref": "#/pictures/1", "children": [], "label": "picture", "content_layer": "body",
2214             "image": {"uri": "data:image/png;base64,BBBB"}, "captions": [{"$ref": "#/texts/0"}]},
2215        ]);
2216        let b = blocks(&j);
2217        let image = &b[0]["c"][2][0]["c"][0];
2218        assert_eq!(image["t"], "Image");
2219        assert_eq!(
2220            image["c"][0],
2221            json!(["", [], [["width", "2.5in"], ["height", "1.25in"]]])
2222        );
2223        assert_eq!(
2224            image["c"][1],
2225            json!([
2226                ast::str_("Architecture"),
2227                ast::space(),
2228                ast::str_("diagram")
2229            ])
2230        );
2231        let image = &b[1]["c"][2][0]["c"][0];
2232        assert_eq!(image["c"][0], json!(["", [], []]));
2233        assert_eq!(
2234            image["c"][1],
2235            json!([ast::str_("A"), ast::space(), ast::str_("duck")])
2236        );
2237        assert_eq!(inches(2.5), "2.5in");
2238        assert_eq!(inches(1.0), "1in");
2239        assert_eq!(inches(0.33333), "0.3333in");
2240    }
2241
2242    /// #835: the markers' number style, delimiter and start.
2243    #[test]
2244    fn list_attributes_follow_the_markers() {
2245        assert_eq!(
2246            list_attributes(&["a)", "b)"]),
2247            (1, "LowerAlpha", "OneParen")
2248        );
2249        assert_eq!(
2250            list_attributes(&["iii.", "iv."]),
2251            (3, "LowerRoman", "Period")
2252        );
2253        assert_eq!(
2254            list_attributes(&["(3)", "(4)"]),
2255            (3, "Decimal", "TwoParens")
2256        );
2257        assert_eq!(
2258            list_attributes(&["c)", "d)"]),
2259            (3, "LowerAlpha", "OneParen")
2260        );
2261        assert_eq!(list_attributes(&["i."]), (1, "LowerRoman", "Period"));
2262        assert_eq!(list_attributes(&["v.", "vi."]), (5, "LowerRoman", "Period"));
2263        assert_eq!(list_attributes(&["x.", "y."]), (24, "LowerAlpha", "Period"));
2264        assert_eq!(list_attributes(&["B."]), (2, "UpperAlpha", "Period"));
2265        assert_eq!(list_attributes(&["IV)"]), (4, "UpperRoman", "OneParen"));
2266        assert_eq!(list_attributes(&["aa)"]), (27, "LowerAlpha", "OneParen"));
2267        assert_eq!(list_attributes(&["12"]), (12, "Decimal", "DefaultDelim"));
2268        assert_eq!(list_attributes(&["3."]), (3, "Decimal", "Period"));
2269        assert_eq!(list_attributes(&["1.2."]), (1, "Decimal", "Period"));
2270        assert_eq!(list_attributes(&[]), (1, "Decimal", "Period"));
2271        let j = doc(
2272            &["#/groups/0"],
2273            json!([
2274                text(
2275                    0,
2276                    "list_item",
2277                    "third",
2278                    json!({"enumerated": true, "marker": "(3)"})
2279                ),
2280                text(
2281                    1,
2282                    "list_item",
2283                    "fourth",
2284                    json!({"enumerated": true, "marker": "(4)"})
2285                ),
2286            ]),
2287            json!([{"self_ref": "#/groups/0", "children": [{"$ref": "#/texts/0"}, {"$ref": "#/texts/1"}], "label": "list", "name": "list"}]),
2288            json!([]),
2289        );
2290        let b = blocks(&j);
2291        assert_eq!(
2292            b[0]["c"][0],
2293            json!([3, {"t": "Decimal"}, {"t": "TwoParens"}])
2294        );
2295    }
2296
2297    /// #836: the document's metadata is Pandoc's `meta`; a reviewer
2298    /// comment is a `comment-start` / `comment-end` span pair around the
2299    /// annotated item's inlines, with its author and date, a reply a
2300    /// further pair; the notes-layer groups never come out as blocks.
2301    #[test]
2302    fn metadata_and_comments_reach_the_ast() {
2303        let mut j = doc(
2304            &["#/texts/0", "#/groups/0"],
2305            json!([
2306                text(
2307                    0,
2308                    "text",
2309                    "The timeout is 30 s.",
2310                    json!({"comments": [{"$ref": "#/groups/0"}]})
2311                ),
2312                text(
2313                    1,
2314                    "text",
2315                    "[author: Reviewer (SR), time: 2026-01-15T12:00:00Z]: Verify it.",
2316                    json!({"content_layer": "notes"})
2317                ),
2318                text(2, "text", "Agreed", json!({"content_layer": "notes"})),
2319            ]),
2320            json!([{"self_ref": "#/groups/0", "children": [{"$ref": "#/texts/1"}, {"$ref": "#/texts/2"}], "label": "comment_section", "name": "comment-3", "content_layer": "notes"}]),
2321            json!([]),
2322        );
2323        j["metadata"] = json!({
2324            "title": "Synthetic Spec", "authors": ["A. Author", "B. Author"], "keywords": ["architecture", "protocol"],
2325            "language": "en-US", "created": "2026-01-15T10:00:00Z", "subject": "Gateways"
2326        });
2327        let (s, _) = from_docling_json(&j, &PandocExportOptions::default()).unwrap();
2328        let v: Value = serde_json::from_str(&s).unwrap();
2329        assert_eq!(
2330            v["meta"]["title"],
2331            ast::meta_inlines(vec![
2332                ast::str_("Synthetic"),
2333                ast::space(),
2334                ast::str_("Spec")
2335            ])
2336        );
2337        assert_eq!(v["meta"]["author"]["t"], "MetaList");
2338        assert_eq!(v["meta"]["author"]["c"].as_array().unwrap().len(), 2);
2339        assert_eq!(
2340            v["meta"]["keywords"]["c"][1],
2341            ast::meta_inlines(vec![ast::str_("protocol")])
2342        );
2343        assert_eq!(v["meta"]["lang"], ast::meta_string("en-US"));
2344        assert_eq!(
2345            v["meta"]["date"],
2346            ast::meta_inlines(vec![ast::str_("2026-01-15T10:00:00Z")])
2347        );
2348        assert_eq!(v["meta"]["subject"]["t"], "MetaInlines");
2349        let b = v["blocks"].as_array().unwrap();
2350        assert_eq!(b.len(), 1, "{b:?}");
2351        let inl = b[0]["c"].as_array().unwrap();
2352        assert_eq!(inl[0]["t"], "Span");
2353        assert_eq!(
2354            inl[0]["c"][0],
2355            json!([
2356                "",
2357                ["comment-start"],
2358                [
2359                    ["comment-id", "3"],
2360                    ["author", "Reviewer (SR)"],
2361                    ["date", "2026-01-15T12:00:00Z"]
2362                ]
2363            ])
2364        );
2365        assert_eq!(
2366            inl[0]["c"][1],
2367            json!([ast::str_("Verify"), ast::space(), ast::str_("it.")])
2368        );
2369        assert_eq!(
2370            inl[1]["c"][0],
2371            json!(["", ["comment-start"], [["comment-id", "3-1"]]])
2372        );
2373        assert_eq!(inl[2], ast::str_("The"));
2374        let last = inl.last().unwrap();
2375        assert_eq!(
2376            last["c"][0],
2377            json!(["", ["comment-end"], [["comment-id", "3-1"]]])
2378        );
2379        assert_eq!(
2380            inl[inl.len() - 2]["c"][0],
2381            json!(["", ["comment-end"], [["comment-id", "3"]]])
2382        );
2383        // With the notes layer on, the comment texts are still not blocks.
2384        let opts = PandocExportOptions {
2385            layers: ContentLayers::ALL,
2386            ..Default::default()
2387        };
2388        let (s, _) = from_docling_json(&j, &opts).unwrap();
2389        assert_eq!(s.matches("Verify").count(), 1, "{s}");
2390        assert_eq!(
2391            split_comment("[time: t]: x"),
2392            (None, Some("t".into()), "x".into())
2393        );
2394        assert_eq!(split_comment("bare"), (None, None, "bare".into()));
2395        assert_eq!(meta_of(&json!({})), json!({}));
2396    }
2397
2398    #[test]
2399    fn text_splits_into_words_spaces_and_line_breaks() {
2400        assert_eq!(
2401            text_inlines("  two  words\nnext\u{a0}line "),
2402            vec![
2403                ast::str_("two"),
2404                ast::space(),
2405                ast::str_("words"),
2406                ast::line_break(),
2407                ast::str_("next\u{a0}line"),
2408            ]
2409        );
2410        assert!(text_inlines("   ").is_empty());
2411    }
2412
2413    #[test]
2414    fn headings_paragraphs_and_lists() {
2415        let j = doc(
2416            &["#/texts/0", "#/texts/1", "#/texts/2", "#/groups/0"],
2417            json!([
2418                text(0, "title", "Ducks", json!({})),
2419                text(1, "section_header", "Diet", json!({"level": 1})),
2420                text(
2421                    2,
2422                    "text",
2423                    "Ducks eat plants.",
2424                    json!({"formatting": {"bold": true}, "hyperlink": "https://x.org"})
2425                ),
2426                text(
2427                    3,
2428                    "list_item",
2429                    "seeds",
2430                    json!({"enumerated": true, "marker": "3."})
2431                ),
2432                text(
2433                    4,
2434                    "list_item",
2435                    "insects",
2436                    json!({"enumerated": true, "marker": "4."})
2437                ),
2438            ]),
2439            json!([{"self_ref": "#/groups/0", "children": [{"$ref": "#/texts/3"}, {"$ref": "#/texts/4"}], "label": "list", "name": "list"}]),
2440            json!([]),
2441        );
2442        let b = blocks(&j);
2443        assert_eq!(b[0], ast::header(1, vec![ast::str_("Ducks")]));
2444        assert_eq!(b[1], ast::header(2, vec![ast::str_("Diet")]));
2445        // formatting inside, the link around it all
2446        assert_eq!(b[2]["t"], "Para");
2447        assert_eq!(b[2]["c"][0]["t"], "Link");
2448        assert_eq!(b[2]["c"][0]["c"][1][0]["t"], "Strong");
2449        assert_eq!(b[3]["t"], "OrderedList");
2450        assert_eq!(b[3]["c"][0][0], 3, "start from the first marker");
2451        assert_eq!(
2452            b[3]["c"][1],
2453            json!([
2454                [ast::plain(vec![ast::str_("seeds")])],
2455                [ast::plain(vec![ast::str_("insects")])]
2456            ])
2457        );
2458    }
2459
2460    /// #848: the two heading projections at their bounds — docling's
2461    /// title-reserving `n + 1` (missing → 2, 0 → 1, capped at 6) and the
2462    /// source's own `n` (missing / 0 → 1, 7 → 6); the title is `Header 1`
2463    /// under both.
2464    #[test]
2465    fn heading_depths_under_both_projections() {
2466        let j = doc(
2467            &[
2468                "#/texts/0",
2469                "#/texts/1",
2470                "#/texts/2",
2471                "#/texts/3",
2472                "#/texts/4",
2473                "#/texts/5",
2474                "#/texts/6",
2475                "#/texts/7",
2476            ],
2477            json!([
2478                text(0, "title", "T", json!({})),
2479                text(1, "section_header", "d1", json!({"level": 1})),
2480                text(2, "section_header", "d2", json!({"level": 2})),
2481                text(3, "section_header", "d5", json!({"level": 5})),
2482                text(4, "section_header", "d6", json!({"level": 6})),
2483                text(5, "section_header", "none", json!({})),
2484                text(6, "section_header", "zero", json!({"level": 0})),
2485                text(7, "section_header", "seven", json!({"level": 7})),
2486            ]),
2487            json!([]),
2488            json!([]),
2489        );
2490        let depths = |heading_depth: HeadingDepth| -> Vec<u64> {
2491            let options = PandocExportOptions {
2492                heading_depth,
2493                ..Default::default()
2494            };
2495            let (s, _) = from_docling_json(&j, &options).unwrap();
2496            let v: Value = serde_json::from_str(&s).unwrap();
2497            v["blocks"]
2498                .as_array()
2499                .unwrap()
2500                .iter()
2501                .map(|b| b["c"][0].as_u64().unwrap())
2502                .collect()
2503        };
2504        assert_eq!(depths(HeadingDepth::Docling), [1, 2, 3, 6, 6, 2, 1, 6]);
2505        assert_eq!(depths(HeadingDepth::Source), [1, 1, 2, 5, 6, 1, 1, 6]);
2506        assert_eq!("Source".parse::<HeadingDepth>(), Ok(HeadingDepth::Source));
2507        assert!("deep".parse::<HeadingDepth>().is_err());
2508    }
2509
2510    /// #852: a Markdown link in an item's text (the flat chain's
2511    /// hyperlink) is a `Link`; brackets that are not one stay text.
2512    #[test]
2513    fn markdown_links_in_text_become_links() {
2514        assert_eq!(
2515            text_inlines("Jump to [target](#target-section) now [x] (y)"),
2516            vec![
2517                ast::str_("Jump"),
2518                ast::space(),
2519                ast::str_("to"),
2520                ast::space(),
2521                ast::link(vec![ast::str_("target")], "#target-section"),
2522                ast::space(),
2523                ast::str_("now"),
2524                ast::space(),
2525                ast::str_("[x]"),
2526                ast::space(),
2527                ast::str_("(y)"),
2528            ]
2529        );
2530        assert_eq!(
2531            text_inlines("[a](https://x.org/a)[b](https://x.org/b)"),
2532            vec![
2533                ast::link(vec![ast::str_("a")], "https://x.org/a"),
2534                ast::link(vec![ast::str_("b")], "https://x.org/b"),
2535            ]
2536        );
2537        assert_eq!(
2538            text_inlines("see ![alt](img.png) here"),
2539            vec![
2540                ast::str_("see"),
2541                ast::space(),
2542                ast::str_("![alt](img.png)"),
2543                ast::space(),
2544                ast::str_("here"),
2545            ]
2546        );
2547    }
2548
2549    /// #851: a run's `_revision` is Pandoc's insertion / deletion span,
2550    /// and an explicit accept / reject view drops the comment spans.
2551    #[test]
2552    fn revisions_become_spans_and_views_drop_comments() {
2553        let j = doc(
2554            &["#/texts/0", "#/texts/1"],
2555            json!([
2556                text(
2557                    0,
2558                    "text",
2559                    "old",
2560                    json!({"_revision": {"kind": "deletion", "author": "Fixture", "date": "2025-01-01T00:00:00Z"}})
2561                ),
2562                text(
2563                    1,
2564                    "text",
2565                    "Commented",
2566                    json!({"comments": [{"$ref": "#/groups/0"}]})
2567                ),
2568                text(
2569                    2,
2570                    "text",
2571                    "[author: Fixture, time: 2025-01-01T00:00:00Z]: Review note",
2572                    json!({"content_layer": "notes"})
2573                ),
2574            ]),
2575            json!([{"self_ref": "#/groups/0", "children": [{"$ref": "#/texts/2"}], "label": "comment_section", "name": "comment-3", "content_layer": "notes"}]),
2576            json!([]),
2577        );
2578        let b = blocks(&j);
2579        assert_eq!(
2580            b[0],
2581            ast::para(vec![ast::span(
2582                &["deletion"],
2583                &[
2584                    ("author".into(), "Fixture".into()),
2585                    ("date".into(), "2025-01-01T00:00:00Z".into())
2586                ],
2587                vec![ast::str_("old")]
2588            )])
2589        );
2590        assert_eq!(
2591            b[1]["c"][0]["t"], "Span",
2592            "comments written without a policy"
2593        );
2594        let accept = PandocExportOptions {
2595            track_changes: Some(TrackChanges::Accept),
2596            ..Default::default()
2597        };
2598        let (s, _) = from_docling_json(&j, &accept).unwrap();
2599        let v: Value = serde_json::from_str(&s).unwrap();
2600        assert_eq!(v["blocks"][1], ast::para(vec![ast::str_("Commented")]));
2601    }
2602
2603    /// #850: a heading numbered by the backend (`_number`) writes its
2604    /// authored text alone; text that merely starts with digits is kept.
2605    #[test]
2606    fn numbered_headings_write_the_authored_text() {
2607        let j = doc(
2608            &["#/texts/0", "#/texts/1", "#/texts/2"],
2609            json!([
2610                text(
2611                    0,
2612                    "section_header",
2613                    "1.1 2026 Results",
2614                    json!({"level": 2, "_number": "1.1"})
2615                ),
2616                text(1, "section_header", "2026 Results", json!({"level": 1})),
2617                text(
2618                    2,
2619                    "section_header",
2620                    "3 Odd",
2621                    json!({"level": 1, "_number": "4"})
2622                ),
2623            ]),
2624            json!([]),
2625            json!([]),
2626        );
2627        let b = blocks(&j);
2628        assert_eq!(
2629            b[0],
2630            ast::header(
2631                3,
2632                vec![ast::str_("2026"), ast::space(), ast::str_("Results")]
2633            )
2634        );
2635        assert_eq!(
2636            b[1],
2637            ast::header(
2638                2,
2639                vec![ast::str_("2026"), ast::space(), ast::str_("Results")]
2640            )
2641        );
2642        // A `_number` the text does not open with is left alone.
2643        assert_eq!(
2644            b[2],
2645            ast::header(2, vec![ast::str_("3"), ast::space(), ast::str_("Odd")])
2646        );
2647    }
2648
2649    /// #849: a bookmarked heading carries its identifier, a bookmark in
2650    /// body text an empty anchor span, and an internal link's `#id` target
2651    /// is written as is.
2652    #[test]
2653    fn anchors_become_header_ids_and_anchor_spans() {
2654        let j = doc(
2655            &["#/texts/0", "#/texts/1", "#/texts/2"],
2656            json!([
2657                text(
2658                    0,
2659                    "section_header",
2660                    "Target section",
2661                    json!({"level": 1, "_anchors": ["target-section", "Alias"]})
2662                ),
2663                text(1, "text", "Jump", json!({"hyperlink": "#target-section"})),
2664                text(2, "text", "Body mark", json!({"_anchors": ["BodyMark"]})),
2665            ]),
2666            json!([]),
2667            json!([]),
2668        );
2669        let b = blocks(&j);
2670        assert_eq!(
2671            b[0],
2672            ast::header_with_id(
2673                2,
2674                "target-section",
2675                vec![
2676                    ast::anchor("Alias"),
2677                    ast::str_("Target"),
2678                    ast::space(),
2679                    ast::str_("section")
2680                ]
2681            )
2682        );
2683        assert_eq!(
2684            b[1],
2685            ast::para(vec![ast::link(vec![ast::str_("Jump")], "#target-section")])
2686        );
2687        assert_eq!(
2688            b[2],
2689            ast::para(vec![
2690                ast::anchor("BodyMark"),
2691                ast::str_("Body"),
2692                ast::space(),
2693                ast::str_("mark")
2694            ])
2695        );
2696        assert_eq!(auto_identifier("Target section"), "target-section");
2697        assert_eq!(auto_identifier("2026 Results"), "results");
2698        assert_eq!(auto_identifier("1.1 Été & hiver!"), "été-hiver");
2699        assert_eq!(auto_identifier("***"), "section");
2700    }
2701
2702    #[test]
2703    fn code_formulas_checkboxes_and_unmapped_labels() {
2704        let j = doc(
2705            &["#/texts/0", "#/texts/1", "#/texts/2", "#/texts/3"],
2706            json!([
2707                text(0, "code", "print(1)", json!({"code_language": "Python"})),
2708                text(1, "formula", "E = mc^2", json!({})),
2709                text(2, "checkbox_selected", "done", json!({})),
2710                text(3, "reference", "[1] Duck, D. (2020).", json!({})),
2711            ]),
2712            json!([]),
2713            json!([]),
2714        );
2715        let b = blocks(&j);
2716        assert_eq!(b[0], ast::code_block(Some("python"), "print(1)"));
2717        assert_eq!(b[1], ast::para(vec![ast::math(true, "E = mc^2")]));
2718        assert_eq!(b[2]["c"][0], ast::str_("☒"));
2719        assert_eq!(b[3]["t"], "Div");
2720        assert_eq!(b[3]["c"][0][1], json!(["docling-reference"]));
2721    }
2722
2723    #[test]
2724    fn tables_keep_spans_header_rows_and_captions() {
2725        let cell = |r0: usize, c0: usize, rs: usize, cs: usize, t: &str, header: bool| {
2726            json!({"text": t, "row_span": rs, "col_span": cs,
2727                   "start_row_offset_idx": r0, "end_row_offset_idx": r0 + rs,
2728                   "start_col_offset_idx": c0, "end_col_offset_idx": c0 + cs,
2729                   "column_header": header})
2730        };
2731        let j = doc(
2732            &["#/tables/0"],
2733            json!([text(0, "caption", "Table 1: Ducks", json!({}))]),
2734            json!([]),
2735            json!([{"self_ref": "#/tables/0", "children": [{"$ref": "#/texts/0"}], "label": "table",
2736                    "captions": [{"$ref": "#/texts/0"}],
2737                    "data": {"num_rows": 2, "num_cols": 2, "table_cells": [
2738                        cell(0, 0, 1, 2, "Species", true),
2739                        cell(1, 0, 1, 1, "Mallard", false),
2740                        cell(1, 1, 1, 1, "Anas", false)]}}]),
2741        );
2742        let b = blocks(&j);
2743        assert_eq!(b.len(), 1, "the caption is rendered by its table only");
2744        let t = &b[0];
2745        assert_eq!(t["t"], "Table");
2746        assert_eq!(
2747            t["c"][1][1][0],
2748            ast::plain(vec![
2749                ast::str_("Table"),
2750                ast::space(),
2751                ast::str_("1:"),
2752                ast::space(),
2753                ast::str_("Ducks")
2754            ])
2755        );
2756        assert_eq!(t["c"][2].as_array().unwrap().len(), 2);
2757        let head = t["c"][3][1].as_array().unwrap();
2758        assert_eq!(head.len(), 1);
2759        assert_eq!(head[0][1][0][3], 2, "colspan");
2760        assert_eq!(t["c"][4][0][3].as_array().unwrap().len(), 1, "one body row");
2761    }
2762
2763    #[test]
2764    fn api_version_is_checked() {
2765        for ok in ["1.23", "1.23.1", "1.23.1.1"] {
2766            assert!(check_api_version(ok).is_ok(), "{ok}");
2767        }
2768        for bad in ["1.22", "2.0", "1", "1.23.2", "1.23.1.2", "x.y", ""] {
2769            let e = check_api_version(bad).unwrap_err();
2770            assert!(e.to_string().contains("only 1.23"), "{bad}: {e}");
2771        }
2772        let err = to_pandoc(
2773            &DoclingDocument::new("t"),
2774            &PandocExportOptions {
2775                api_version: Some("1.22".into()),
2776                ..Default::default()
2777            },
2778        )
2779        .unwrap_err();
2780        assert_eq!(
2781            err,
2782            PandocError::UnsupportedApiVersion {
2783                requested: "1.22".into()
2784            }
2785        );
2786    }
2787
2788    #[test]
2789    fn empty_document_is_a_valid_pandoc_document() {
2790        let (s, artifacts) =
2791            to_pandoc(&DoclingDocument::new("e"), &PandocExportOptions::default()).unwrap();
2792        assert!(artifacts.is_empty());
2793        assert_eq!(
2794            s,
2795            r#"{"pandoc-api-version":[1,23,1,1],"meta":{},"blocks":[]}"#
2796        );
2797        assert_eq!(s, DoclingDocument::new("e").export_to_pandoc_json());
2798    }
2799}