pdq

Rust-native PDF split and merge MVP.

Runtime constraints:

• does not invoke the qpdf binary;
• does not use a subprocess wrapper;
• does not link libqpdf through FFI.

The first implementation uses lopdf as a pure-Rust PDF object model and writer. It focuses on valid split/merge outputs for ordinary PDFs. Advanced qpdf behavior such as linearization, forms, outlines, and full compatibility with unusual historical PDFs is intentionally out of scope for the current MVP. Damaged cross-reference data is the one repair pdq performs — see Repair.

Encrypted PDFs (RC4, AES-128, AES-256) are supported as inputs: files are decrypted while loading, and outputs are always written unencrypted (like qpdf --decrypt). Files encrypted with only an owner password — the common case — open without any flags because the empty user password is tried first. Files that require a real password take --password on split, split-pages, merge, and page-count. render goes through hayro's own parser, which likewise opens owner-password-only files but has no --password option, so PDFs with a real user password cannot be rendered.

Commands

cargo test
cargo run --bin pdq -- split input.pdf --out 1-3 out-1.pdf --out 4-z out-2.pdf
cargo run --bin pdq -- split-pages --output 'page-%d.pdf' input.pdf
cargo run --bin pdq -- split-pages --output 'chunk-%d.pdf' --pages-per-file 3 input.pdf
cargo run --bin pdq -- merge --output merged.pdf a.pdf b.pdf
cargo run --bin pdq -- page-count input.pdf
cargo run --bin pdq -- split-pages --password secret --output 'page-%d.pdf' encrypted.pdf

split-pages --pages-per-file N groups consecutive pages into files of at most N pages each (%d is the 1-based chunk index; the last chunk may contain fewer pages). The default of 1 writes one page per file.

page-count prints the number of pages to stdout. By default it trusts the root /Pages /Count — the same semantics as qpdf --show-npages — and automatically falls back to a validated page-tree walk when /Count is missing, malformed, negative, or implausibly large. Pass --strict to force the validated walk, which counts the exact leaf pages split/split-pages would resolve and is immune to lying metadata.

--password decrypts inputs that need a user or owner password; the outputs of split, split-pages, and merge are written decrypted either way.

Tests may use qpdf as a development validator when it is available on PATH. The runtime implementation must remain qpdf-free.

Repair

Files with damaged cross-reference data — truncated or garbage xref tables, destroyed trailers, tables whose offsets point at the wrong objects — are repaired automatically, the way qpdf, Poppler, and pdf.js recover them: the raw file is scanned for N G obj headers and the cross-reference table is rebuilt from what is actually there (best effort). Repair is strictly a last resort and never runs on well-formed files: it only starts after the normal parse fails, or after a fetch proves the xref lies about an offset, so healthy files pay nothing for it. A repaired read emits one warning line on stderr, and outputs built from a repaired source are always full rewrites with a fresh, valid xref — never byte copies of the damage.

Two classes stay hard errors by design: encrypted files with damaged xref data (repair cannot decrypt; the error suggests a dedicated repair tool), and files where no catalog can be recovered at all. In both cases the error names the damaged cross-reference data rather than a generic parse failure.

Render

pdq render rasterizes pages to PNG through hayro, a pure-Rust PDF renderer, so the qpdf-free and FFI-free constraints still hold. Pages render in parallel and %d in the output pattern receives the original, zero-padded page number. The command lives behind the render cargo feature, which is enabled by default; build with --no-default-features for a smaller split/merge-only binary. Rendering requires Rust 1.92 or newer.

Real-Document Testing

tests/real_world.rs builds raw-byte replicas of the two court-document families from the benchmark corpus (deep balanced page tree with an image filter zoo; flat page tree with one shared resources dictionary) and asserts split/merge behavior on them, including resource-pruning regression guards.

tests/corpus.rs runs pdq against a directory of actual PDFs with qpdf as ground truth, classifying each file (pass / note / skip / warn / fail):

scripts/fetch_corpus.sh --fixtures --qpdf --pdfjs   # reproducible anywhere
scripts/fetch_corpus.sh --local ~/Downloads          # plus your own PDFs
cargo test --release --test corpus -- --ignored --nocapture

No PDFs are versioned: --qpdf/--pdfjs fetch the public test corpora from their upstream repositories, and --fixtures regenerates the anonymized benchmark replicas (12,732 and 2,642 pages) from the seeded generator in scripts/make_fixtures.py — private documents stay strictly local.

The corpus lives in corpus/ (gitignored; local files are symlinked). Use PDQ_CORPUS_DIR to point elsewhere, PDQ_CORPUS_MAX_FILES to cap a run, and PDQ_CORPUS_STRICT=1 to also fail on scope gaps where qpdf handles a file that pdq refuses.

Benchmarks

Measured 2026-07-04 on two real-world court PDFs: 200 MB / 12,732 pages and 26 MB / 2,642 pages. The files stay outside the repo; scripts/make_fixtures.py synthesizes PII-free replicas with the same structural pathology (object counts, page-tree shape, shared-resources pattern, filter zoo) that reproduce these timings within noise.

Wall time is hyperfine --warmup 1 --runs 5 mean (page count: warmup 2, 10 runs), 120 s timeout. Every completed output was validated by page count and qpdf --warning-exit-0 --check; split scenarios validated first, middle, and last files. qpdf used --remove-unreferenced-resources=no for copy-like paths where applicable.

The 12,732-page rewrite is a statistical tie with MuPDF (overlapping σ); every other scenario is a pdq win. To reproduce the timing matrix:

PDQ_BIG_PDF=/path/to/12732-pages.pdf \
PDQ_SMALL_PDF=/path/to/2642-pages.pdf \
scripts/benchmark.sh

The chart is generated by scripts/gen_benchmark_svg.py (data at the top of the script) into assets/benchmark.svg.