# Differential testing tolerances vs DjVuLibre
This document records the per-codec tolerance thresholds used by the
`diff_djvulibre` harness (`examples/diff_djvulibre.rs`) and the CI
workflow `.github/workflows/diff.yml`. The versioned corpus manifest and
machine-enforced coverage contract live in `conformance/corpus.json`; the
public history is published at
https://matyushkin.github.io/djvu-rs/dev/conformance/. Tracked under #192 and
#682. See `docs/conformance.md` for the complete local reproduction and
artifact contract.
## Harness
```
cargo run --release --features cli --example diff_djvulibre -- \
[--width N] [--tolerance T] [--max-pages M] <file.djvu> [...]
```
For each page, both `djvu_rs` and `ddjvu` are asked to render at the
same target size; the resulting RGBA / RGB pixmaps are compared per
pixel. A pixel is "mismatched" if `max(|Δr|, |Δg|, |Δb|) > tolerance`.
## Why tolerance > 0 is acceptable
A small per-channel difference is **not** a decoder bug:
* **IW44**: lossy wavelet, both implementations carry intermediate
rounding errors that diverge by ±1 LSB.
* **Resampling**: when target render size < native page size, both
pipelines apply a downsampling filter. They are not identical
(different kernels, different colour-space conversions). At small
widths this dominates the diff.
* **YCbCr ↔ RGB**: the BT.601 / BT.709 matrices and rounding bias
differ subtly between implementations.
Differences > a few LSB or > 5% of pixels are **decoder bugs** and
should be filed as separate issues.
## Per-codec ceilings (`PAGE_CEILING_PCT`, `MEAN_DELTA_CEILING`)
The CI gate enforces global ceilings; per-codec breakdown is
informational for triage.
| **JB2 only (bilevel)** | 0 | < 0.5% | < 0.5 |
| **IW44 only (photo)** | 4 | < 5% | < 1.5 |
| **Mixed JB2 + IW44 (scan)** | 4 | < 5% | < 1.5 |
| **Native-resolution render** | 4 | < 0.5% | < 0.2 |
| **Downsampled render < 600px** | 8 | < 14% | < 3.0 |
Empirical results, `--width 600 --tolerance 4`:
| `boy.djvu` | JB2+IW44 | 0.000% | 0 | 0.00 |
| `chicken.djvu` | JB2+IW44 | 0.000% | 0 | 0.00 |
| `colorbook.djvu` | IW44 | < 1% | 22 | 0.20 |
| `watchmaker.djvu` † | IW44 | 0.03% | 22 | 0.05 |
† `watchmaker.djvu` not in fixtures; see `references/djvujs/library/assets/`.
## CI ceilings (currently enforced)
`.github/workflows/diff.yml` runs weekly + on dispatch and fails if any
page in the *bit-perfect-baseline* corpus exceeds:
* `PAGE_CEILING_PCT = 0.8`
* `MEAN_DELTA_CEILING = 0.2`
The CI corpus is the subset of `tests/fixtures/*.djvu` that is
bit-perfect or comfortably within the strict native-resolution ceilings today
(see the empirical table above). Any future regression above those ceilings
fails the gate.
The gate is fail-closed: every document/page declared in the manifest must
produce exactly one result. A missing fixture, missing `ddjvu`, parse/render
failure, malformed JSONL record, duplicate record, or silently skipped page
fails report generation. Each published `summary.json` records the Git commit,
pinned DjVuLibre identity, input SHA-256 digests, corpus digest, render policy,
and schema version so historical results remain attributable. The same gate
compares normalized text, annotation map-area counts, and bookmark trees with
`djvused`; a missing or divergent semantic plane also fails closed.
## Known divergences (excluded from CI gate, tracked separately)
Before #831 the pinned Ubuntu DjVuLibre 3.5.28 baseline reported
`colorbook.djvu` page 0 at `0.7488%` mismatch with tolerance 4, so the global
ceiling is `0.8%`; the mean-delta ceiling remains the tighter `0.2` guard. The
conformance artifact records the exact DjVuLibre package identity so this
baseline cannot silently drift between tool builds.
Resolved under **#831**: a render at page size now reproduces DjVuLibre's
`get_bgpixmap` / `stencil` rules exactly instead of approximating them:
* A reduced BG44 (`red = compute_red(page, plane)`, 2..=12) is enlarged with
`GPixmapScaler` coordinates (`prepare_coord`, 1/16-pixel steps, first
coordinate slightly negative), rows counted **from the bottom**, a
vertical pass rounded to 8 bits, then a horizontal pass.
* FG44 takes the whole cell `(x / red, y_from_bottom / red)` with no
interpolation.
The #279 centre alignment was close, but every page whose height is not a
multiple of `red` was shifted by one or more rows. With the fix, every page of
all 21 fixtures (360 pages) is bit-exact against local `ddjvu` at tolerance 0,
and the conformance corpus now covers every page of `colorbook` (62),
`history` (3), `czech` (85) and `carte` (1). The last two joined under #833,
after the reader learnt the DIRM shared-annotation type (flag 3, `A` in
`djvused ls`) and the `(metadata …)` block in it, and the gate learnt that
`djvused ls` shows all thumbnails as one `T <thumbnails>` row. Measured before the fix at
tolerance 4: colorbook 53/62 pages over the gate, czech 52/85, history 2/3
(page 0 at 36.7%), carte 1/1 (1.17%). Regions of a page-size render use the
same rules; down-scaled and zoomed renders keep the centre-aligned mapping. The digest test
`native_reduced_bg_fg_matches_ddjvu_digest` in `tests/document_and_render.rs`
pins the result.
Resolved under **#279** (superseded for page-size renders by #831):
* `colorbook.djvu` page 0 — after centre-aligned BG sampling, integer FG/BG
colour-cell pitch, and nearest-cell FG44 lookup, the native diff is within the
strict gate. Reproducer:
`cargo run --release --features cli --example diff_djvulibre -- --width 99999 --tolerance 4 --max-pages 1 tests/fixtures/colorbook.djvu`.
The earlier local reference build measured `mismatch_pct = 0.2673%`, while
the pinned Ubuntu DjVuLibre 3.5.28 package used by CI measures `0.7488%`.
Both remain far below the pre-fix `3.4482%` / mean Δ `0.659`; the published
dashboard keeps tool identities and results separate rather than flattening
them into one baseline.
Stage checks showed the JB2 mask matches ddjvu exactly and raw BG44 at
754×1223 matches ddjvu exactly; the fixed drift was native-page
compositing/upscaling, especially FG44 pixels. Rejected hypotheses:
endpoint-only plane mapping improved only to 2.07%; old nearest-neighbour
FG44 mapping without centre/integer-cell alignment regressed to 10.93%;
ad-hoc per-layer subpixel offsets improved to ~1.60% but were not kept because
they are not a clean-room format rule.
* `navm_fgbz.djvu` pages 3 + 4 — after the #279 sampling change they measure
`0.0010%` / `0.0271%` mismatch at width 2550 with mean Δ `0.001` / `0.003`,
within the strict native gate.
## Filing divergences
If the CI gate fails or local development surfaces a page with
`mismatch_pct > 5%` at native resolution:
1. Save the failing JSON line from `diff_results.jsonl`.
2. Reproduce locally with the same tolerance + width.
3. Attempt to localise the codec: re-run with the same args but a
single `tests/fixtures/<file>.djvu` to isolate.
4. File a new issue under the `bug` label, link to #192, attach:
- Source `.djvu`
- djvu-rs render PNG
- ddjvu render PPM
- Diff visualisation (`magick compare`)