Module textparse 

Pure-Rust PDF text extraction (replacing pdfium’s glyph layer).

pdfium reports rendered glyph boxes, which diverge from docling’s docling-parse C++ parser at exactly the points that drive conformance: generated spaces get a zero-width box, combining diacritics get a real-width box, and ligature/fraction glyphs land at different x. This module instead reconstructs each glyph’s box from the font’s own advance widths and the PDF text/graphics matrices — the same information docling-parse uses — so a space is as wide as the font says and a combining mark has zero advance.

The output is the same [Glyph] stream pdfium produces (native PDF coordinates, y-up), fed straight into the existing docling-parse line sanitizer ([crate::dp_lines]). Only the digital text layer is handled here; pages without one still fall back to OCR upstream.

Structs

PageParserCells

One page’s text cells from the pure-Rust parser: prose line cells, per-word cells, and code line cells — all from a single glyph parse. Replaces the pdfium text path (roadmap item 6) when the parser drop is enabled.

Functions

content_diagnosis

Localize where a page’s text is lost, for the text_layer diagnostic. Extraction can come up empty at three different points — no content stream reached the parser, the stream did not decode into operators, or it ran but produced no glyphs (fonts/encodings) — and from the outside all three look the same. Report them per page.

debug_glyphs

Debug: raw glyph stream (ch, ll, lr, lb, lt) (native coords) for page index, before the sanitizer. For comparing char cells to docling-parse.

pdf_all_cells

Full parser text layer: prose + word + code cells per page, glyphs parsed once. prose/words come from the docling-parse contraction ([crate::dp_lines]); code splits only at the parser’s own space glyphs (monospace keeps its source spacing). Used by the pipeline to retire pdfium’s text path.

pdf_text_pages

Whole pages for the text-layer-only conversion (crate::convert_text_layer): the parser’s prose/word/code cells plus page geometry, assembled into [PdfPage]s with no rendered image and no link annotations. Everything here is pure Rust (lopdf), so it compiles without the ml feature — including on wasm32. A page the parser can’t read (no text layer) comes back with empty cells; there is no pdfium fallback on this path.

pdf_textlines

Public entry: per-page (width, height, line cells) for a PDF, via the Rust text parser + the docling-parse line sanitizer. Used by the pipeline and the textparse_dump example.

pdf_words

Debug/diagnostic entry: per-page (width, height, word cells) for a PDF, via the Rust parser glyphs run through the docling-parse word grouping. Used to compare parser word cells against docling-parse’s word_cells oracle (roadmap item 6).

text_layer_is_vestigial

Is this “text layer” a vestige rather than the document’s text?

xref_repair_status

Why the cross-reference repair did or did not fire, for the text_layer diagnostic. A PDF that will not load is indistinguishable from a scan in production (both convert to nothing), so the reason has to be askable.