Skip to main content

pdf_untangle/
lib.rs

1//! # pdf-untangle
2//!
3//! Untangle garbled PDF text: layout-aware extraction that reconstructs reading
4//! order, strips page furniture (headers/footers/page numbers), repairs
5//! hyphenation and page-break flow, and — with a *profile* — splits a document
6//! into structured, citable provisions (rules, statutes, sections).
7//!
8//! A PDF stores text as thousands of individually positioned glyphs with no
9//! notion of paragraphs, columns, headings, or reading order. Naive extraction
10//! yields the familiar mishmash. This crate rebuilds the logical document in
11//! three layers:
12//!
13//! 1. **extract** — PDF → positioned text runs per page (pluggable backends)
14//! 2. **layout** — runs → ordered lines/blocks (columns, furniture, flow)
15//! 3. **structure** — blocks → provisions, driven by a declarative TOML
16//!    [`Profile`] (heading patterns, citation template, date formats)
17//!
18//! The same crate ships a CLI (`cargo install pdf-untangle`).
19//!
20//! ## Status
21//!
22//! Layer 1 (extraction) is implemented with two feature-gated backends —
23//! `backend-pdfium` (Google PDFium, highest fidelity, native library loaded
24//! at runtime) and `backend-pure` (pure Rust, zero native dependencies; the
25//! crate default) — see [`extract::runs`] and the [`extract`](mod@extract)
26//! module docs.
27//! Layer 2 (layout reconstruction) is implemented: [`layout::reconstruct`]
28//! turns per-page runs into reading-ordered lines and blocks with columns
29//! handled, page furniture stripped (with accounting), hyphenation repaired,
30//! and indentation quantized — this is what `pdf-untangle text` prints.
31//! Layer 3 (profile-driven structure) is implemented: [`structure::detect`]
32//! applies a [`Profile`] to the reconstructed layout — this is what the
33//! high-level [`extract()`](crate::extract()) entry point and the
34//! `pdf-untangle extract` / `pdf-untangle check` subcommands run.
35//!
36//! ## Example
37//!
38//! ```no_run
39//! # fn main() -> anyhow::Result<()> {
40//! let profile = pdf_untangle::Profile::builtin("cnmi-rules")?;
41//! let doc = pdf_untangle::extract(std::path::Path::new("R05.pdf"), &profile)?;
42//! for p in &doc.provisions {
43//!     println!("{} — {} (effective {})", p.citation, p.heading.as_deref().unwrap_or(""), p.effective_date);
44//! }
45//! assert!(doc.line_coverage >= 0.98, "unconsumed: {:?}", doc.unconsumed_lines);
46//! # Ok(()) }
47//! ```
48
49pub mod extract;
50pub mod layout;
51pub mod output;
52pub mod profile;
53pub mod structure;
54
55pub use output::{Document, Provision};
56pub use profile::Profile;
57
58use std::path::Path;
59
60use anyhow::{Context, Result};
61
62/// Extract structured provisions from a text-based PDF file using `profile`.
63///
64/// This is the single high-level entry point: Layer 1 (extraction) → Layer 2
65/// (layout reconstruction) → Layer 3 (profile-driven structure detection).
66/// The file's name (final path component) becomes each provision's
67/// `source_file`. See [`extract_bytes`] to work from an in-memory PDF.
68pub fn extract(pdf: &Path, profile: &Profile) -> Result<Document> {
69    let bytes = std::fs::read(pdf).with_context(|| format!("reading {}", pdf.display()))?;
70    let source_file = pdf
71        .file_name()
72        .map(|n| n.to_string_lossy().into_owned())
73        .unwrap_or_else(|| pdf.display().to_string());
74    extract_bytes(&bytes, profile, &source_file)
75}
76
77/// [`extract`], but from PDF bytes already in memory. `source_file` is
78/// recorded verbatim on every provision (use the original file name).
79pub fn extract_bytes(pdf_bytes: &[u8], profile: &Profile, source_file: &str) -> Result<Document> {
80    let pages = extract::runs(pdf_bytes)?;
81    let layout = layout::reconstruct(&pages);
82    structure::detect(&layout, profile, source_file)
83}