rapid-structure-rs

rapid-structure-rs is a reusable Rust document-structure engine for PaddleOCR-family ONNX models. One page pipeline covers three capabilities:

• text — detection, text-region cropping, recognition and CTC decoding (PP-OCRv6 family);
• table — table region location plus SLANet-plus structure decoding to HTML / Markdown;
• formula — PP-FormulaNet_plus-M recognition with a Pix2Text MFD detector for routing.

It owns image normalization, polygon output, model resolution, ONNX Runtime sessions, the three output channels (JSON / HTML / Markdown), an evaluation harness, and a local web demo (rapidstruct serve). It does not depend on Tauri, SQLite, a clipboard implementation, or a specific application.

Renamed: this crate was rapid-ocr-rs before v1.0.0 — the name only described the text half of it. See CHANGELOG.md for the rename entry.

Platform support

The only supported platform is Windows x64 with the MSVC ABI (x86_64-pc-windows-msvc). Other targets fail at compile time with a single explicit compile_error! from src/platform_gate.rs instead of a scatter of "missing Windows API" diagnostics; the gate itself is verifiable with pwsh -NoProfile -File tools/check_platform_gate.ps1.

Explicit non-goals: Windows x86 (i686), Windows ARM64, the Windows GNU ABI, and also Wine, WSL, Linux and macOS. This is a deliberate narrowing, not an unfinished port. 32-bit Windows x86 is a real limitation rather than a cfg change: the precompiled ONNX Runtime, the DirectML/CUDA execution-provider DLLs and the ~566 MB formula recognition model are all x64 artifacts, and the crate itself is only built and verified on the MSVC ABI (import library names, #[link(name = "psapi")], the ort prebuilt runtime). Windows ARM64 has the same problem in a different shape — the whole native stack would have to be re-collected and re-measured. A future port gets its own platform plan rather than incremental cfg additions here.

Platform differences are expected to live only in platform implementation modules (currently src/runtime/memory.rs); business code must not branch on the OS.

License

The rapid-structure-rs source code is licensed under the Apache License, Version 2.0. See LICENSE and NOTICE. Third-party dependencies, ONNX Runtime binaries, OCR model weights, and model dictionaries have separate license and redistribution terms; see THIRD_PARTY_NOTES.md before redistributing a complete application or model bundle.

Installation and external assets

Add the crate from crates.io:

cargo add rapid-structure-rs

The crate publishes Rust code and small contract fixtures only. It does not bundle ONNX Runtime DLLs, execution-provider DLLs, OCR models, formula models, table models, or dictionaries. The application must download those assets, verify their declared SHA-256 values, and place the runtime DLLs where Windows can load them. The model manifests and the rapidstruct model-management commands provide the authoritative URLs and checksums.

The optimized PP-FormulaNet model is mirrored at ModelScope and Hugging Face. The default manifest uses a pinned ModelScope revision; both copies must match the SHA-256 recorded in assets/default_models.yaml before use.

Current baseline

• PP-OCRv6 detector + recognizer; direction classifier disabled unless a compatible classifier artifact is explicitly configured.
• DirectML is requested by the default feature set on Windows x64. If DirectML cannot load or reject the model, the ordinary OCR and table sessions rebuild with CPU and expose that fallback in provider resolution. Formula sessions also report provider resolution and reject an unavailable accelerator when strict provider behavior is required. Select ProviderPreference::Cpu explicitly for a CPU-only deployment; CUDA remains an opt-in feature and all providers should be benchmarked on the target machine. CANN is not a Windows target and has been removed.
• The stable application-neutral API is OcrEngine::recognize(OcrRequest). OwnedPixelBuffer accepts BGRA/RGBA/RGB/Gray pixels, padding stride, bottom-up buffers, and an optional ROI.
• ImageInput supports encoded bytes, file paths, synchronous URL loading with connect/request timeouts, decoded RecImage, and strided pixel buffers. Applications that must avoid network access should use Encoded, Pixels, Image, or File inputs.
• OcrOutput contains text, line polygons, scores, provider resolution, and stage timings. Word- and character-level boxes are intentionally optional (recognition.words, three modes off / words / chars, default off); the CLI takes --words / --chars and serve takes POST /api/ocr?words=..., all three mapped onto the same WordOutputMode vocabulary. The boxes come from CTC post-processing, not from a character-detection model: every character is aligned on the full per-step CTC probability surface by a Viterbi forced alignment over a 2L+1 state graph (src/ocr/rec/align.rs), its frame range is tiled into column boundaries, and mapped back onto the page through the detection quad's homography. A line whose frame budget cannot hold its text falls back to the older average-character-width estimate and says so (WordInfo::align_status), so a downgrade is never silent. Either way the boxes must not be presented as pixel-accurate character boundaries: the cross-axis extent (line height) is still inherited from the whole line, because CTC never predicts it. How far off they are is measured rather than asserted: docs/13-charbox-benchmark.md holds the pre-alignment ctc baseline and the post-alignment run side by side (pixel-ink ground truth, per-character IoU, signed boundary error, selection precision/recall and the cost of each words= mode) together with the criteria an alignment change has to clear.
• Turning those boxes into "drag across the image and copy the text" is a host presentation-layer responsibility, not a library one: this crate never depends on a browser, DOM or clipboard. The bundled web demo shows one complete way to do it — ?words=chars feeds a transparent DOM text layer above the image, every span affine-mapped from its own word polygon (tilted lines are not replaced by axis-aligned boxes), while the canvas keeps drawing the visual boxes. Formula and table content deliberately stays out of that layer, since formula.latex and table.markdown are not what the user's eyes see.
• Each copy exit has one fixed meaning, and the checks that pin it are listed in docs/05 §9.9: dragging on the image copies recognized characters and merges a cross-region selection in regions[] order (document order, which on a vertical or multi-column page is not visual order), “Copy full text” adds each table's markdown and each formula's LaTeX while skipping the text lines a table already consumed, double-clicking a formula row copies its LaTeX, and a rendered table is copied cell by cell. A visual-Unicode rendering of a formula is not offered.

Minimal usage

use rapid_structure_rs::{
    ClassifierPlan, ClassifierPolicy, DetectionPolicy, EngineConfig, FormulaPolicy, ImageInput,
    OcrEngine, OcrRequest, OutputPolicy, PreprocessPolicy, RapidOcrEngine,
    RecognitionPolicy, StagePlan, WordOutputMode,
};

let mut config = EngineConfig::default();
config.global.use_cls = false;
config.det.model_path = Some("PP-OCRv6_det_medium.onnx".into());
config.det.allow_download = false;
config.rec.model.model_path = Some("PP-OCRv6_rec_medium.onnx".into());
config.rec.model.rec_keys_path = Some("ppocrv6_dict.txt".into());
config.rec.model.allow_download = false;
let mut engine = RapidOcrEngine::new(config)?;
let output = engine.recognize(OcrRequest {
    input: ImageInput::File("screen.png".into()),
    roi: None,
    scale_hint: None,
    stages: StagePlan {
        detect: true,
        classify: ClassifierPlan { policy: ClassifierPolicy::Off, apply_rotation: false },
        recognize: true,
    },
    preprocess: PreprocessPolicy::default(),
    detection: DetectionPolicy::default(),
    recognition: RecognitionPolicy { words: WordOutputMode::Off },
    output: OutputPolicy::default(),
    // Formula routing is off by default, so the text path is byte-identical to a
    // build without the formula feature.
    formula: FormulaPolicy::default(),
})?;
println!("{} regions", output.regions.len());
# Ok::<(), rapid_structure_rs::RapidOcrError>(())

PreprocessPolicy::default() leaves max_side as None, so the engine's global.max_side_len controls resizing. Set preprocess.max_side only when a request needs to override that limit. Encoded bytes, files, and URLs are dimension-probed and checked against max_decode_pixels before pixel decode. File and URL inputs are additionally bounded by max_encoded_bytes (default 128 MiB, PreprocessPolicy), with Content-Length checks and a streaming read cap for HTTP responses.

All new integrations should use the generic OcrEngine trait and OcrRequest. The former scattered run/OcrResult API is internal and is not part of the public crate contract.

Formula recognition

rapid-structure-rs exposes two formula capabilities: an independent PP-FormulaNet_plus-M ONNX recognizer, and optional page-level formula routing that reuses the ordinary OCR pipeline without touching its CTC contract.

Independent formula API

use rapid_structure_rs::{FormulaRecognizer, RuntimeConfig};

let mut recognizer = FormulaRecognizer::from_model(
    "pp_formulanet_plus_m.onnx".as_ref(),
    &RuntimeConfig::default(),
)?;
let result = recognizer.recognize(&image::open("formula.png")?)?;
println!("{}", result.latex);
# Ok::<(), Box<dyn std::error::Error>>(())

• FormulaRecognizer::recognize_batch preserves input order and defaults to a maximum batch size of 16.
• FormulaRecognition contains latex, raw token_ids, eos_index, truncated, model_id, elapsed_ms, and batch_size. elapsed_ms is the wall time of the call that produced the result: every result of one recognize_batch shares the same value (the whole batch), so divide by batch_size for a per-image estimate. It is never a per-sample measurement that grows with position in the batch.
• recognize_encoded, recognize_file, and recognize_url delegate to the shared input loader (input::image_loader), so encoded-byte, decoded-pixel, streaming-read, header-probe, timeout, and error semantics are identical to ordinary OCR.
• The default sequence limit is 4096. The model's in-graph Loop pads the whole batch to 2561 columns when any sample fails to emit EOS, so a limit at or below that value would reject an entire batch and lose the samples that did recognize correctly. Set a larger limit only if a different model needs it.
• Provider strictness: a formula session reports the requested and selected provider. With fail_if_provider_unavailable = true, an unavailable accelerator returns RapidOcrError::UnsupportedProvider instead of falling back. Request ProviderPreference::Cpu explicitly if CPU is intended.

Page-level formula routing

OcrRequest::formula (FormulaPolicy) turns a whole page into text regions plus typed formula regions:

use rapid_structure_rs::{
    FormulaPolicy, ImageInput, OcrEngine, OcrRegion, OcrRequest, RapidOcrEngine, RegionKind,
};

# fn example(engine: &mut RapidOcrEngine) -> Result<(), rapid_structure_rs::RapidOcrError> {
let request = OcrRequest {
    // ... stages / preprocess / detection / recognition / output as usual ...
    # input: ImageInput::File("page.png".into()),
    # roi: None,
    # scale_hint: None,
    # stages: Default::default(),
    # preprocess: Default::default(),
    # detection: Default::default(),
    # recognition: Default::default(),
    # output: Default::default(),
    formula: FormulaPolicy {
        enabled: true,
        model_path: Some("pp_formulanet_plus_m.onnx".into()),
        detector_path: Some("pix2text-mfd-1.5.onnx".into()),
        ..FormulaPolicy::default()
    },
};
let output = engine.recognize(request)?;
for region in output.regions.iter().filter(|r| r.kind == RegionKind::Formula) {
    println!("{:?}", region.formula.as_ref().map(|f| &f.latex));
}
# Ok(())
# }

How routing works:

1. Formula regions come from the optional page detector (pix2text-mfd-1.5.onnx, a YOLO11 detect model) and from FormulaPolicy::input_regions, which lets a caller declare regions without a detector.
2. Candidates are filtered by min_area_ratio and deduplicated by iou_threshold / containment. The result is capped by max_regions and is independent of candidate order.
3. Formula pixels are painted white before the ordinary text pipeline runs, so CTC never executes on them. "Formula regions skip CTC" is an execution guarantee, not post-hoc filtering. Whitening is per region, not a per-text-box overlap rule: there is no text-detection-coverage threshold, and FormulaPolicy deliberately exposes no such knob (an earlier draft documented a text_overlap_skip_ratio that never affected behavior; it was removed rather than left as a fake API).
4. Each formula region is cropped from the untouched original image, recognized by FormulaRecognizer, and appended as a RegionKind::Formula region that keeps its polygon, detector score, and model id. Formula regions never carry a CTC recognition outcome (they have no meaningful per-character confidence), and OcrOutput::validate() enforces that invariant.
5. roi and tiled preprocessing are rejected with a structured error while formula routing is enabled, because their coordinate mapping would diverge from original-image formula coordinates.

Output formats:

• JSON adds kind: "text" | "formula", latex/eos_index/truncated on formula items, and a top-level formulas array in reading order.
• Markdown renders formula regions as $$...$$ display math in reading order. Inner $$ is escaped, blank lines are dropped, and a truncated result gets an invisible <!-- formula truncated: no EOS token --> marker.
• HTML renders formula polygons separately, lists the LaTeX with a copy button, and reports data-truncated / data-eos per formula.
• Raw token ids are only emitted when FormulaPolicy::include_token_ids is set.

Known limitations (explicit, and pinned by tests):

• Detector precision is not perfect. On the repository's 12 smoke pages the detector reports false positives on non-formula pages (code listings, dense prose) at the default confidence_threshold = 0.25. The route now runs one original-image OCR pass as evidence and rejects candidates whose covered text is clearly code; ambiguous candidates still remove the whole covered region. Raise confidence_threshold/min_area_ratio to trade recall for precision.
• Whitening changes the pixels the text detector sees, so text regions outside the formula region can be re-segmented. Measured on 08数字公式与符号.png: whitening one row merges/splits a distant row (42.7 ms becomes 42.7 + ms). Content is not lost, but region boundaries and line grouping can change. Callers that require byte-stable text segmentation should not enable formula routing.
• A missed formula (below threshold, or a detector that returned nothing) stays in the text channel, so ordinary CTC will emit its best text result. The arbitration path is conservative and does not invent a formula region.
• Formula crops use the axis-aligned bounding box of the region quadrilateral.

Model assets:

formula recognizer:
https://www.modelscope.cn/seeyuer/pp-formulanet-plus-m-optimized/resolve/0bb8154e3583d24613eaa74448de43ecf4ae73ec/pp_formulanet_plus_m.onnx
SHA-256 6e509ec4b7cda604dc9bf80a876ae6494693deea4c7f4157a4dc0eba0ebda745

page formula detector:
models/formula/detection/pix2text-mfd-1.5.onnx (Pix2Text-MFD-1.5, YOLO11m detect)

The models and tokenizer metadata are not bundled. Download them separately, verify SHA-256, and review upstream licenses before redistribution (THIRD_PARTY_NOTES.md records the conflicting license metadata on the detector).

RapidDoc post-processing equivalence

FormulaRecognizer post-processing is remove_chinese_text_wrapping -> fix_latex -> ftfy.fix_text, matching RapidDoc's order. The deterministic ftfy steps are ported exactly (tables generated from the real ftfy by tools/build_ftfy_tables.py, verified byte-for-byte against Python): remove_terminal_escapes, fix_c1_controls, fix_latin_ligatures, fix_character_width, uncurl_quotes, fix_line_breaks, NFC normalization, and remove_control_characters.

The heuristic mojibake repairs (fix_encoding, restore_byte_a0, replace_lossy_sequences, decode_inconsistent_utf8) and unescape_html are not implemented; fix_surrogates cannot apply to a Rust String. This is not a silent gap: tests/fixtures/formula-postprocess/cases.json enumerates the divergence cases and asserts them, and the same fixture records that across all 357,022 labels in datasets/formula the deterministic subset explains every observed ftfy change (1 of 357,022 lines, a curly-quote uncurl). postprocess_latex must therefore not be described as fully equivalent to RapidDoc's post-processing.

Model integrity

The single model inventory is ModelSet (src/model_set.rs): one entry per file with its role, expected SHA-256 and size. validate_model_files returns the state of every file (missing / present / corrupt) in one pass, and ModelManifest::validate_files is a thin wrapper over it that fails on the first file that is not present. A set is complete only when every file is present and every file carries a hash, so an unhashed entry can never report a complete set. File names must be bare relative names: absolute paths, path separators and .. are rejected by one shared rule.

There is exactly one authoritative source per model directory (src/model_source.rs): if <model-dir>/manifest.json exists it is the only source and assets/default_models.yaml is not consulted at all; otherwise the default table is the only source. There is no merge and no optional override. A manifest must declare schema_version: 1 and a files array; the legacy four-field shape (detector/recognizer/dictionary/classifier) and any other schema_version produce a locating error. If the selected source lacks a role the requested pipeline needs, the error lists the missing roles instead of degrading silently.

Keep model weights outside the crate source tree and record their upstream URL, revision, checksum, and redistribution terms in the consuming application's manifest (see assets/manifest.example.json).

What the digest cache does — and does not — guarantee

Every per-file digest question (/api/models, the state table returned by validate_model_files, and the two formula load paths) is answered through one process-wide identity-keyed cache (src/model_verify.rs); the admission check on both queues is existence-only, so it reports null digests instead of paying for a re-hash. A file's identity is its path, size, mtime and the SHA-256 of its first and last 64 KiB; the cached value is the full SHA-256 that was actually computed for that identity. A hit costs one stat plus one 128 KiB read, independent of file size; a full re-hash happens on first sight, and again whenever the size, the content windows, or (for reporting) the mtime change. For files of 128 KiB or less the two windows overlap and cover the whole content.

The promise is deliberately narrow and exact:

content is verified at first use and re-verified whenever the file's identity changes; the service does not claim that the on-disk content is trusted at all times.

What a hit reports, exactly: present together with a digest means that digest is the result of the last full verification of this path. It is not a statement that the bytes on disk were just re-read. The cache is a performance heuristic, not an integrity guarantee, and present is not evidence that the current disk content was verified.

The residual blind spot is a same-size, same-mtime edit that also keeps the first and last 64 KiB byte-identical (edit the middle only). The partial digest is a heuristic that narrows the window, not a security boundary — an attacker who can write the file can also preserve its head and tail. Three deterministic entry points re-hash anyway: --reverify-models at startup (cold-verifies every file the run will load and refuses to start when one is missing or corrupt), POST /api/models/reverify while running (clears the cache, cold-verifies, and rebuilds the engine session so a replaced file cannot keep being served from memory), and every load that carries a trusted digest — such a load reads the whole file, and when that full digest disagrees with the cached one the stale entry is dropped before the verdict comes back, including on the request whose comparison fails. A middle-only edit therefore survives at most until the next guarded load, not indefinitely.

That is a narrowed window, not a real-time guarantee: a session already built from a now-falsified entry keeps running until an explicit reload, and a load without a trusted digest computes no digest at all, so it can neither confirm nor falsify what the cache holds.

The wording boundary above is machine-enforced, not a review habit: tools/check_model_state_claims.ps1 fails when a sentence uses an out-of-scope phrase such as “integrity guarantee” or “trusted at all times” without its own negation, and fails when one of the scope statements above disappears from its file. -SelfTest re-proves the rule still bites, because a guard that silently stops reporting is worse than no guard at all.

One run model plan

rapidstruct serve resolves one list of "files this run will load" at startup and everything reads it: --reverify-models, POST /api/models/reverify, the pipelines block of /api/models, admission on both queues, and the two loading paths (pin_engine_paths and FormulaPolicy).

• Text pipeline: detector + recognizer + dictionary, plus classifier when use_cls is on.
• Formula pipeline: only when formula routing is enabled (a detector is configured through --formula-detector, or declared by the model set as the formula_detector role) — formula_recognizer and formula_detector.
• Formula routing disabled ⇒ the formula models are not in the plan at all: neither entry point reads the 566 MB recognizer, and a corrupt one neither blocks startup nor ordinary OCR.

The two semantics are compatible and both hold:

• --reverify-models is an explicit opt-in meaning "verify this run's whole plan at startup and refuse to start if any file in it is unusable". With formula enabled a corrupt formula model (detector or recognizer) fails startup, and the error names the file and which pipeline it belongs to.
• Runtime admission stays scoped per pipeline: a corrupt formula model yields a 409 with detail.scope = "formula" on the formula queue while ordinary OCR keeps working. POST /api/models/reverify reports per pipeline — the text engine outcome (pipelines.text.outcome) and the formula result plus its routing state (pipelines.formula.*) — so "text ready, formula corrupt" is explicit instead of a bare ready.

What "verified" means without a declared digest

An external --formula-detector that no model set declares has no trusted digest. "Verified" then means exactly three things: the file exists, it is readable, and it looks like an ONNX ModelProto (its protobuf prologue starts with the ir_version field: tag 0x08 followed by a varint in 1..=64; an empty file is rejected). This is a heuristic, not an integrity proof: the service reports sha256: null for such a file and never claims its content was checked. The cold verification still computes and reports the actual digest it read. When a model set does declare a digest, that declared value is always enforced by hash.

Flow logging: following one upload end to end

rapidstruct serve writes no flow log by default. Turn it on with --log-level flow or the equivalent environment variable:

rapidstruct serve --model-dir <DIR> --log-level flow
RAPID_STRUCTURE_SERVE_LOG=flow rapidstruct serve --model-dir <DIR>     # same switch, CLI wins

Every HTTP request then gets one line (request id, method, path, status, response bytes, duration) and every job gets lifecycle lines (id, queue, the admission decision and its reason, queue position, wait time, run time, terminal state, result size; on failure the status code plus code and detail). Both kinds share the same request id, so one upload can be followed from the request line to the terminal line:

serve-flow: req=7 POST /api/ocr -> 202 bytes=334 in 6ms
serve-flow: req=7 job=job-0000000000000003 queue=text admission=accepted decision=run queued position=0
serve-flow: req=7 job=job-0000000000000003 queue=text running wait_ms=3
serve-flow: req=7 job=job-0000000000000003 queue=text terminal state=succeeded result_bytes=35201 backend_ms=881

A refused upload has no job id — the request id is its only correlation key, which is exactly what is needed when "the upload did nothing" has to be explained:

serve-flow: req=9 queue=formula admission=rejected status=409 code=models_corrupt detail={...}

An unknown level (--log-level chatty) refuses to start and names the switch, rather than silently behaving as off. The startup banner always prints the effective level and the current value of RAPID_STRUCTURE_SERVE_LOG (<unset> when unset).

The page has a matching switch: ?verbose=1 (or ?verbose=0 to turn it off, or Ctrl+Alt+L; the choice is remembered in localStorage). The diagnostics panel then lists the most recent requests with their method, URL, HTTP status, duration and error code, including the response body of failures. With verbose off, that section still appears whenever the last request failed — so "why did nothing render?" is answerable without opening devtools.

The /result contract is pinned from both sides

The inline page parses /api/jobs/{id}/result against a frozen field set and refuses to render anything that does not match it (regions[], kind, polygon.points as exactly four [x, y] pairs, recognition.text, recognition.score, formula.latex, plain_text, timings, timing_ledger). A renamed or missing field is reported with its exact path and never silently tolerated. Four independent checks keep the two sides from drifting apart:

CLI and benchmark

Visual OCR report

The CLI renders one offline HTML report per image in a directory:

cargo run --bin rapidstruct -- report \
  --input-dir ./datasets/text \
  --output-dir ./reports/ocr \
  --config ./models/configs/test-config-small.yaml

For every supported image in the input directory, the command copies the source image into the output directory and writes a sibling .html report that renders the image with every OCR polygon in original image coordinates. The report has no server or external JavaScript dependency and can be opened directly in a browser. The current CLI does not expose a separate HTML output flag for a single run; use --json for one-off machine-readable output or report for the directory-level visual pass.

cargo run --bin rapidstruct -- run --img-path screen.png --config config.yaml --json
cargo run --bin bench_warm_e2e -- --config config.yaml --images-dir ./screens --rounds 3

For accuracy regression, provide a JSON manifest with image, text, and optional quadrilateral boxes fields, then run:

cargo run --bin rapidstruct -- evaluate \
  --manifest ./datasets/text/golden-manifest.json \
  --config ./models/configs/test-config-small.yaml \
  --output ./reports/evaluation.json

The evaluator reports per-image and aggregate Unicode CER, exact-text match, detection precision and recall, and matched quadrilateral polygon IoU. The repository fixture ./datasets/text/golden-manifest.json is generated from the visible text in prototypes/ocr-location-test.html; it is a real text regression baseline for all 12 images. Its boxes arrays are intentionally empty until every text polygon is independently annotated, so detection precision/recall/IoU are emitted as null rather than being inferred from predictions.

The benchmark reports model initialization, detector/recognizer timings, and warm wall-clock P50/P90. For reproducible CPU comparisons, pin the image resize limit and all ONNX Runtime sessions to the same thread profile:

cargo run --bin bench_warm_e2e -- \
  --config config.yaml --images-dir ./screens \
  --warmup-rounds 1 --rounds 3 \
  --max-side-len 1280 --intra-threads 16

--max-side-len overrides the global image bound and --intra-threads overrides detector/classifier/recognizer CPU sessions. The benchmark is a measurement tool, not an accuracy guarantee; compare accuracy separately on the same image set. GPU providers can be selected explicitly when the corresponding Cargo feature and runtime libraries are installed:

On the repository's 12-image PP-OCRv6 small set, the warm timing comparison used the same three-round, 16-thread condition. CER values are from the corresponding 12-image evaluation runs:

The 1280 setting is therefore exposed as a recommended application profile, not a changed library default: it was about 34% faster than 2000 on this machine while the fixture CER did not regress. Re-run the evaluation on the target corpus before changing an application's quality/latency trade-off.

cargo run --features cuda-provider,download-binaries --bin bench_warm_e2e -- \
  --config config.yaml --provider cuda --device-id 0 \
  --warmup-rounds 1 --rounds 3

DirectML is available on Windows with directml-provider. Do not assume GPU is faster: OCR detection produces variable text regions and recognition may be split into many small dynamic batches. Benchmark CPU, CUDA, and DirectML on the target image set before selecting an application default. CUDA and DirectML runs in strict provider mode fail at startup when the requested provider is unavailable instead of silently falling back to CPU.

On the repository's 12-image benchmark (PP-OCRv6 small, 2000px bound), DirectML measured about 490 ms/image over three warm rounds. A comparable CPU 16-thread run measured about 1,016 ms/image (the earlier single-round CPU sample was about 690 ms/image). The same manifest's mean CER was identical (0.44765) for CPU and DirectML. These are machine-specific measurements, not library-wide guarantees; rerun the benchmark on the target GPU before changing an application default.

For an application or CLI that wants the ONNX Runtime binaries copied during build, use the convenience feature:

cargo run --features cli-default --bin rapidstruct -- run --img-path screen.png

cli-default expands to download-binaries, copy-dylibs, and directml-provider. The library's default feature set requests DirectML but does not copy runtime binaries; applications that need a CPU-only build can use --no-default-features --features ort-runtime and set ProviderPreference::Cpu.

Local web UI in one command

The released package does not ship the models, and rapidstruct serve takes a model directory plus the model-specific switches. serve.ps1 (repository root) wraps both: it reads a small JSON profile, inspects the run's model plan before serving, prints the download URL for every missing file (or, with consent, downloads it and verifies the declared SHA-256), and only then starts the server.

pwsh -File serve.ps1                  # fast profile: models/text/small + the table model
pwsh -File serve.ps1 -DryRun          # print what it would run
pwsh -File serve.ps1 -ModelsOnly      # check/download models, do not serve
pwsh -File serve.ps1 -Download        # pre-consent to auto-download missing files
pwsh -File serve.ps1 -ServeConfig models/configs/serve-local-full.json

Precedence is command line > profile > built-in defaults. The profiles are models/configs/serve-local.json (fast: ~1 s/page on a 3200x2000 page) and models/configs/serve-local-full.json (medium OCR + formula + table: 18-27 s/page on the same page). Per-file download URLs come from the server's own /api/models (sets[].files[].source_url), so the launcher never re-implements model resolution; --table-model is the one startup-time path check, so it is repaired before the server starts. It never fabricates a URL: a file whose set declares no source_url is reported as "no declared download URL". Details: docs/05-local-web-demo-implementation.md §3.1.

API replacement acceptance

Exact numeric equality with the previous implementation is not an acceptance requirement: the unified API changes region ordering, stage reporting, provider resolution, and some post-processing details. Replacement acceptance is a functional regression check with explicit tolerances on the repository fixture (benchmarks/image-bench/api-comparison-small-medium.json):

• mean CER ≤ 0.03 on the 12-image fixture;
• mean polygon IoU ≥ 0.85 where box annotations exist;
• absolute detected-region count delta ≤ 5% versus the old API;
• no per-image hard failure.

Exact-image text match is reported for information only and is not required. The current small/medium comparison is inside these thresholds.

Formula benchmark and evaluation

formula_eval is the phase-9 evaluation tool for all three datasets. It writes a stable sampling manifest (a stable hash of the dataset-relative path plus the ground truth — not "first N", and not the image bytes), per-sample records, failure classification, exact/normalized match, CER, EOS/truncated counts, throughput with P50/P95, peak working set, and provider/thread settings.

Sampling is a stable hash of dataset-relative path + ground truth: the same dataset selects the same samples on any machine and under any absolute root, and SampleStrategy::Hash never degenerates into "first N". Image content is deliberately not part of the sampling key — it is recorded separately as content_sha256, so a data change is reported as a content change instead of silently re-selecting a different subset.

Manifests carry three digests:

$Model = "<pp_formulanet_plus_m.onnx>"
$TestSet = "./datasets/formula"   # collection root

# PaddleX example set, 501 val images, with a reproducible manifest.
cargo run --release --bin formula_eval -- --model $Model --dataset-root $TestSet `
  --dataset latexocr --split validate --batch-size 8 `
  --manifest-output target/formula-eval/manifest-val-501.json `
  --output target/formula-eval/val-501.json

# Rust/Python link comparison over exactly the same samples.
python tools/formula_reference.py --model $Model --dataset-root $TestSet `
  --dataset latexocr --manifest target/formula-eval/manifest-val-501.json `
  --output target/formula-eval/python-val-501.json
cargo run --release --bin formula_eval -- --model $Model --dataset-root $TestSet `
  --dataset latexocr --split validate `
  --expect-manifest target/formula-eval/manifest-val-501.json `
  --python-reference target/formula-eval/python-val-501.json `
  --output target/formula-eval/val-501-compared.json

# im2latex smoke and full test set (10,355 references, 71 empty labels).
cargo run --release --bin formula_eval -- --model $Model --dataset-root $TestSet `
  --dataset im2latex --split test --limit 100 --output target/formula-eval/im2latex-100.json
cargo run --release --bin formula_eval -- --model $Model --dataset-root $TestSet `
  --dataset im2latex --split test --output target/formula-eval/im2latex-full.json

# UniMER: one sub-run per subset; HWE is reported separately, never averaged in.
foreach ($subset in 'spe','cpe','sce','hwe') {
  cargo run --release --bin formula_eval -- --model $Model --dataset-root $TestSet `
    --dataset unimer --subset $subset --output "target/formula-eval/unimer-$subset.json"
}

# The whole sequence above, including the CPU benchmark, in one run.
pwsh -NoProfile -File tools/run_formula_evaluation.ps1

formula_eval --expect-manifest fails if the current selection does not hash to the recorded manifest, so a re-run either reproduces the same sample set or reports the difference instead of silently evaluating a different subset. It also verifies the image-content digest when the recorded manifest has one. Add --manifest-only to do this check without loading the model (see "Verification tiers").

Long datasets can be split across processes and merged back:

# 5 shards, each evaluating every 5th manifest entry.
foreach ($i in 0..4) {
  Start-Process cargo -ArgumentList @('run','--release','--bin','formula_eval','--',
    '--model',$Model,'--dataset-root',$TestSet,'--dataset','unimer','--subset','cpe',
    '--shard',"$i/5",'--expect-manifest',"target/formula-eval/manifest-unimer-cpe.json",
    '--output',"target/formula-eval/unimer-cpe.shard$i.json")
}
# Merge reorders records by manifest and recomputes metrics with the same summarizer.
cargo run --release --bin formula_eval -- --merge-shards target/formula-eval/unimer-cpe.shard*.json `
  --output target/formula-eval/unimer-cpe.json

The merge refuses to combine reports whose model.sha256, provider, batch_size, dataset/split/subset, manifest, image content digest, or Python reference digest disagree, and it aggregates a Python comparison across all shards rather than keeping the first shard's partial counts.

Benchmark:

cargo run --release --bin formula_bench -- --model $Model `
  --image <formula1.png> --image <formula2.png> `
  --rounds 5 --warmup 1 --batch-sizes 1,2,4,8 --provider cpu `
  --output target/formula-eval/bench-cpu.json

formula_bench reports per-stage min/max/mean/P50/P95/stddev over the measured rounds, per-image and per-batch end-to-end latency, whether batch size changed the token sequence, the resolved provider and whether a CPU fallback occurred, and the process peak working set (windows:GetProcessMemoryInfo.PeakWorkingSetSize — the only memory口径 in the crate; a failed sample is reported with its Win32 error instead of being omitted). Formula throughput is always reported separately from ordinary OCR; --ocr-baseline <bench.json> only adds the ordinary OCR benchmark side by side and never merges the two into one number.

deterministic_tokens compares each batch row against the same image decoded alone. The comparison uses the tokens up to and including EOS, because the in-graph Loop pads every row of a batch to the same width: comparing whole rows would report a difference that is only padding. Measured batch=1/2/4 over real val images are identical by that definition.

Tests and external assets

cargo test --all-targets passes in a clean clone without any model or dataset: the contract fixtures under tests/fixtures/** are committed, and every test that needs the real 594 MB model or the public datasets skips with a message when the assets are absent. There is no development-machine absolute path fallback anywhere in the crate.

# Force external-asset tests to fail instead of skipping (CI with staged assets).
$env:RAPID_STRUCTURE_REQUIRE_EXTERNAL_ASSETS = "1"
$env:RAPID_STRUCTURE_MODEL_ROOT = "<workspace>/models"
$env:RAPID_STRUCTURE_FORMULA_TEST_ROOT = "<workspace>/datasets/formula"
cargo test --all-targets

Quality gates

cargo test --all-targets
cargo fmt --all -- --check
cargo clippy --all-targets -- -D warnings
cargo check --features directml-provider,cuda-provider

Verification tiers: what to run, and when

Full-dataset evaluation is not a routine regression step. A complete run over val + im2latex + all four UniMER subsets is ~40,000 images and takes hours (UniMER-CPE alone is ~3 h with sharding, because its mean label is 658 characters and the in-graph autoregressive Loop dominates). Run the cheapest tier that can actually falsify your change.

Commands for each tier are in "Formula benchmark and evaluation" above. The committed baseline tests/baseline/formula-evaluation-2026-10-03.json may be reused instead of re-running a full dataset only while all of the following are unchanged: the model file, image preprocessing, the tokenizer, post-processing, batch inference and EOS handling, and the metric implementation. If any of them changes, the stored numbers no longer describe the current code and the affected full run must be repeated.

Two properties make the cheap tiers meaningful:

• --manifest-only resolves the dataset and writes the manifest (including image content hashes) without loading the model, so a data-integrity check costs seconds instead of hours: formula_eval --manifest-only --dataset-root $TestSet --dataset latexocr --split validate --expect-manifest <manifest.json>
• --expect-manifest verifies the sample-selection digest (manifest_sha256) and the image-content digest (content_sha256), so replacing an image file without changing its path or label is detected. sample_set_sha256 distinguishes "a different set of samples" from "the same samples in a different order". Manifests recorded before content_sha256 existed still pin the sample set, and --expect-manifest says so explicitly instead of pretending to verify content.

Long datasets can be sharded across processes when a full run is genuinely required (--shard INDEX/COUNT + --merge-shards); the manifest still covers the whole set, the merge reuses the same summarizer, and it refuses to merge reports from different models, providers, batch sizes, or Python references.

Provider feature verification

Provider features are opt-in and can be compile-checked or tested on a machine without a GPU:

cargo check --features directml-provider
cargo check --features cuda-provider
cargo check --features directml-provider,cuda-provider
cargo test --features directml-provider
cargo test --features cuda-provider

Windows x64 acceptance

The stage-by-stage evidence for the Windows x64 (MSVC) narrowing lives in docs/04-windows-phase-reports.md; the plan is docs/03-windows-only-optimization-tasks.md (file name kept from the first round). Summary of the final state:

• 12-image OCR gates are bit-identical to the pre-refactor baseline (mean CER 0.44765135645866394, region average 34.8333), release binaries are 3.7% smaller, and the im2latex-100 formula smoke reproduces exactly.
• ONNX Runtime inference is about 85.6% of page time and the named Rust side about 13.7%, with a stated residual of -6.75 ms/page (0.69%): the timing ledger is a diagnostic instrument whose named components are measured over different (sequential) ranges than total_ms — the outer OcrTimings::preprocess_ms window ends before inner.run() is entered, so no wall-clock interval is counted twice — and it therefore does not form a strict partition of total_ms. It is not the acceptance basis for any performance claim. The stage-6 conclusion (the bottleneck is ONNX Runtime, not Rust hot paths) rests on the inference share being an order of magnitude larger than every Rust component, which a residual of that size cannot overturn. See docs/04-windows-phase-reports.md.
• No speedup is claimed anywhere: the thread matrix, the -C target-cpu=x86-64-v3 A/B (-2.84% median over 5 interleaved pairs, one pair +19.35%) and the formula batch curve all land inside this machine's run-to-run noise, which reaches 29-39% between identical binaries. Any future performance claim must come from an interleaved A/B on the target machine.
• Acceleration claims require measurement: tools/check_provider_claims.ps1 fails any report that claims a non-CPU provider without fallback while its measured p50 is within 10% of CPU. On this machine DirectML passes (1.99x) and CUDA fails (1.00x, cuDNN missing), which is the documented finding rather than a tool bug.

Implementation reference

The pipeline behavior is cross-checked against RapidOCR and the local paddle-ocr-rs reference. Their licenses and model licenses remain separate from this crate's Apache-2.0 license; see THIRD_PARTY_NOTES.md.