oxideav-h264

Pure-Rust H.264 / AVC (ITU-T Rec. H.264 | ISO/IEC 14496-10) codec for the oxideav framework.

Status

Active spec-driven implementation built solely from the ITU-T Rec. H.264 | ISO/IEC 14496-10 specification (2024-08 edition) as the single authoritative source. The decoder reconstructs I / P / B slices (4:2:0, 4:2:2 and 4:4:4 chroma) through full intra / inter prediction, transform, deblocking and DPB output ordering on both the CAVLC and CABAC entropy paths; the encoder emits CAVLC and CABAC streams across Baseline / Main / High profiles. MBAFF (macroblock-adaptive frame/field) frame pictures decode byte-exact with frame-coded pairs (round 410: pair-interleaved deblock addressing, spec-exact §8.7.2.1 bS, Table 6-4 inter MV neighbour derivation; mbaff-interlaced BitExact gate) and with FIELD-coded pairs (round 413: §8.4.2.1 per-field reference selection with zero-copy field-view MC, §7.4.4 skipped-pair flag inference, the §9.3.3.1.x FIELD CABAC residual contexts (Tables 9-22/9-23) with §8.5.6/§8.5.7 field inverse scans, exact Table 6-4 block-level ctx probes, and the §8.7 per-MB MBAFF deblock walker incl. the dual field-mode top edge and the NOTE 3 field |Δmv| threshold; mbaff-field-pairs BitExact gate). Round 416 closed the CAVLC MBAFF tail (mbaff-field-cavlc BitExact gate: the residual walker's §9.2.1.1 nC probes now route through the exact §6.4.12.2 Table 6-4 process). §8.5.15 lossless transform-bypass (qpprime_y_zero_transform_bypass_flag=1, QP′Y == 0) decodes losslessly — identity §8.5.10..§8.5.13 legs + V/H intra residual DPCM (lossless-transform-bypass BitExact gate). PAFF (field_pic_flag == 1) decodes byte-exact end-to-end (round 416): field pictures reconstruct as half-height pictures (§7.4.2.1.1 eq. 7-26) with the §8.5.6 FIELD coefficient scan, reference lists initialise through the §8.2.4.2.2/.2.4 + §8.2.4.2.5 field process (per-field entries, alternating parity from the current field's own parity) with the §8.2.4.3 field RPLM, opposite-parity references apply the §8.4.1.4 Table 8-10 chroma-MV adjustment, per-field §8.7 deblocking uses the field-picture bS rules, and the §C.4.4 driver pairs complementary fields into full-height output frames (frame POC = Min of the two field POCs) — five PAFF fixtures (I fields, P field pairs, cross-parity reference, frame-as-field-reference, mixed frame/field switching) gate BitExact, generated by the crate's own clean-room PAFF field ENCODER (encoder::field: interlaced SPS, field slice headers, field-scan CAVLC emission, Table 8-10 chroma predictors, per-field deblock) and black-box-validated byte-identical against a stock reference binary. Round 436 closed the standing interlaced follow-ups: PAFF B field pairs decode AND encode byte-exact — non-reference B/B pairs between P/P anchors (coding order 0, 2, 1, 4, 3, …) with both the §8.4.1.2.2 spatial and the §8.4.1.2.3 temporal direct derivation on fields (colPic = the same-parity anchor field, all eq. 8-201/8-202 distances on per-FIELD §8.2.1 order counts) — which first required fixing the §8.2.5.3 sliding window to count a complementary reference field pair as ONE frame slot (per-entry counting had halved the effective DPB for field streams). §8.2.5.4 MMCO field-marking landed with per-parity DPB markings: field MMCO 1/2/3/4/6 mark individual FIELDS (pair-exception eviction rules included), the §8.2.4.2.5 interleave skips fields not marked with the matching reference class, and two encoder axes gate it end-to-end (long-term field-pair promotion via MMCO 4 + two field MMCO 3 ops with RPLM long_term_pic_num splicing, and a field MMCO 1 unmark exercising the "missing field is ignored" list fallback). The encoder field 8×8 transform scan (§8.5.7 Table 8-14 field row, pre-composed for the §7.4.5.3.3 CAVLC split + writer-remap pipeline) enables transform_8x8 in field pictures across I/P/B. All 13 new PAFF gates decode byte-exactly in our decoder and a black-box reference binary. Round 440 completed the interlaced co-located derivation and the PAFF encoder tail. Decoder: the §8.4.1.2.1 process now implements every Table 8-6/8-7/8-8 frame/field cross row (colPic as a decoded frame / decoded field / field of a decoded frame / complementary field pair; mbAddrCol1..7 with vertMvScale and the eq. 8-182 AFRM tie-break; MapColToList0 by picture identity with the field↔frame index doubling; per-FIELD eq. 8-201/8-202 distances), frame-slice reference lists collapse complementary field pairs into §8.2.4.2.1/.2.3 frame-level UNITS (re-interleaved for MC; non-paired fields excluded), and MBAFF intra prediction gathers its reference SAMPLES through the §6.4.12 Table 6-4 process (mixed field/frame pairs previously read raster-adjacent rows). All four staged JVT conformance bitstreams gate byte-exact end-to-end as hard asserts: camp_mot_picaff0_full (PAFF CABAC IBBP temporal direct, 30/30), CAPA1_TOSHIBA_B (90/90), CVPA1_TOSHIBA_B (90/90) and — round 443 — CAPAMA3_Sand_F (PAFF + MBAFF, 50/50): the §8.4.1.3 MVP neighbour probes now use the §6.4.11.7 exact sample locations, whose yN PARITY drives the Table 6-4 field-MB selection inside a field-coded neighbouring pair (the B/C/D probes at yP − 1 of a partition previously read the wrong field macroblock's motion data in mixed field/frame AFRM pictures). Encoder: implicit-weight B fields (PaffConfig::b_implicit_weight — stride-3 anchors put the two non-reference B pairs at unequal per-field POC distances so the §8.4.2.3.3 weights genuinely leave 32/32: 43/21 and 22/42 at logWD = 5, applied to luma AND chroma) and B REFERENCE fields (PaffConfig::b_reference_fields — a stride-4 layout stores a reference B pair through the sliding window and the non-reference B pairs use the coded B FIELDS as RefPicList0[0] / RefPicList1[0] — a B field as the §8.4.1.2.1 colPic — with an §8.2.4.3.1 RPLM re-splicing the previous anchor on later P fields), both self-roundtrip byte-exact and black-box-validated under spatial AND temporal direct. Round 448 closed the ChromaArrayType-0 surface — the last deferred leg of the High 4:4:4 decode row. Monochrome (4:0:0, chroma_format_idc = 0) decodes AND encodes end-to-end (High-family SPS, no §7.3.5.1 intra_chroma_pred_mode, luma-only §7.3.5.3 residual, Table 9-4(b) CBP, luma-only §8.7 deblocking, single-plane output), and separate_colour_plane_flag = 1 streams decode via the literal §8.1 three-invocation model: every coded slice routes to one of three internal monochrome sub-decoders selected by its §7.4.3 colour_plane_id, each running the complete per-plane pipeline (POC, marking, DPB, §C.4 bumping), with the three output streams re-assembled into three-plane pictures (0 → S_L, 1 → S_Cb, 2 → S_Cr) and the §8.5.9 luma-path scaling lists selected by iYCbCr = colour_plane_id. The encoder carries matching 4:0:0 and per-plane SCP legs (CAVLC + CABAC; I_16x16 + I_NxN + CABAC Intra_8x8 IDRs, P with P_Skip/P_L0_16x16, the full B mode set under spatial AND temporal direct, the §8.6.4 8x8-transform trials) plus encoder::scp::encode_scp_sequence for whole three-plane streams — 44 new gates: 4:0:0 streams byte-exact in our decoder AND a black-box reference decoder; SCP streams (which the stock reference binary rejects: "separate color planes are not supported") gated by self-roundtrip plus the §8.1 compositional identity against the black-box-validated standalone monochrome decodes. Round 451 opened the Extended-profile (88) surface: SP / SI switching pictures (§8.6) and slice data partitioning (§7.3.2.9). SP slices reconstruct through the literal transform-domain process — eq. 8-415 forward core transform of the prediction, the §8.6.1 QPY-dequant + QSY-requant chain (eqs. 8-416/8-420, chroma 8-424..8-431) or the §8.6.2 switching chain (8-432..8-441), §8.5.12 at qP = QSY/QSC (sMbFlag == 1) and Clip1(rij) output with no second prediction add — and the Table 7-12 SI macroblock type (Intra_4x4-coded, raw 0 with the offset-by-one I-type remap) decodes through the same switching path; §8.7.2.1's SP/SI bullets give every MB of such slices the intra boundary strengths. A clean-room SP/SI ENCODER (encoder::sp) emits primary SP pictures, switching SP pictures and SI pictures over an all-I_PCM IDR anchor, and the defining property is gated byte-exactly: a switching SP picture predicted from a DIFFERENT stream's decode state, and an SI picture predicted from nothing at all, both reproduce the primary SP reconstruction BYTE-FOR-BYTE through deblocking; primary SP chains are byte-exact against the encoder mirror at QP != QS and close-agreement-gated against a black-box reference decoder (which rounds negative §8.6 coefficients off the literal equations and lacks the switching semantics entirely). Data-partitioned slices (NAL types 2/3/4) decode with the category-2 elements read from partition A and every residual() (and I_PCM payload) routed to partition B (intra collective types) or C (inter) per §7.4.2.9; dual-form gates emit the SAME coded content as one NAL and as an A+B+C triple and require identical reconstruction — which flushed out a real pre-existing defect (the CAVLC I_PCM parse never committed its §9.2.1.1 nC-grid slot, desyncing any CAVLC slice mixing I_PCM with coded macroblocks). The round also gated 12-/14-bit decode (I_PCM sample-exact round-trips and Intra_16x16 DC-residual pictures byte-exact against the black-box reference decoder at both depths), arbitrary slice order (every slice-order rotation of multi-slice pictures decodes byte-identically — which exposed and fixed a §6.4.8 availability defect where a not-yet-decoded neighbour counted as same-slice) and redundant-coded-picture discard (§7.4.2.2). The Annex F/G/H/I scalable / multiview / 3D extensions remain the outstanding surface (see the coverage matrix below).

No external decoder source is consulted while writing this implementation — only the spec PDF. Conformance is verified by behavioural diff: our decoder is run against synthetic and public-domain real-world streams and the YUV output is compared against an independent reference decoder run separately. The staged fixture corpus (tests/docs_corpus.rs) decodes each fixture byte-for-byte against its expected.yuv. As of round 416 the corpus holds 36 fixtures and all 36 targets gate BitExact over their full frame counts; round 440 added four staged upstream JVT conformance bitstreams (docs/video/h264/conformance/) and as of round 443 all four gate byte-exact as hard asserts (camp_mot_picaff0_full 30/30, CAPA1_TOSHIBA_B 90/90, CVPA1_TOSHIBA_B 90/90, CAPAMA3_Sand_F 50/50) — every profile/tool combination staged (Baseline/Main/ High/High10/High422/High444; 8- and 10-bit; 4:2:0/4:2:2/4:4:4; CAVLC + CABAC; I/P/B with temporal + spatial direct, explicit + implicit weighted prediction, multi-ref, multi-slice, non-flat scaling matrices, 8×8 transform, MBAFF interlaced frame- AND field-coded pairs (CABAC and CAVLC), PAFF field pictures (I/P, cross-parity references, picture-adaptive frame/field switching), lossless transform-bypass, Annex-B + avcC framing) decodes byte-exactly.

The spec PDF itself can't be redistributed (ITU-T copyright) so it lives in the private OxideAV/docs repo at video/h264/T-REC-H.264-202408-I.pdf. Clone that repo next to your oxideav checkout if you're an internal contributor; otherwise the document is freely downloadable from https://www.itu.int/rec/T-REC-H.264 (pick the 2024-08 edition).

Specification coverage

Encoder coverage

The encoder emits both CAVLC and CABAC elementary streams; every coded stream is cross-checked bit-exact against an independent reference decoder run separately. Round 420 added rate control (CBR + capped VBR) and a registry encoder factory — the codec registry now carries h264_encoder::make_encoder next to the decoder, with a typed option schema (rc=auto/cqp/cbr/vbr, qp, bitrate, max_bitrate, buffer_size, gop, cabac, bframes). Round 443 extended rate control to B pictures (see the table row).

Round 475 — registry layouts, idr_pic_id, avcC. The registry encoder accepts every layout the crate's High-family encoder codes: Yuv420P / Yuv422P / Yuv444P / Gray8 at 8 bit and the *P10Le / *P12Le / Gray10Le / Gray12Le variants (declared on the registry descriptor so the Layer-2 gateway ladder sees them). 8-bit 4:2:0 input (grey coded as 4:2:0 with constant chroma) keeps the Baseline / Main session — IDR / P / B, rate control, streams byte-identical to earlier releases (integration_registry_bit_identity pins them). Every other layout rides encoder::deep::DeepSession under the §A.2.4 – §A.2.7 minimum profile (High 100 / High 10 110 / High 4:2:2 122 / High 4:4:4 Predictive 244): IDR + P pictures (Intra_16x16, P_Skip / P_L0_16x16 / Intra_16x16 fallback), CAVLC or CABAC, fixed QP (rc=cqp; no B pictures and no rate control on this path), transform_8x8, deblock, and lossless=true (transform bypass at QP′Y = 0, profile 244, reconstruction equal to the source). profile=auto|baseline|main|high|high10|high422|high444 picks the profile explicitly (one that does not admit the layout is refused; an explicit High-family profile routes 8-bit 4:2:0 input through the High-family encoder). Every IDR access unit carries a §7.4.3 idr_pic_id that advances per IDR (0, 1, 2, … mod 65536), so an all-intra gop=1 sequence never reads as redundant slices of one picture. format=annexb (default: start codes, in-band SPS / PPS, empty extradata) or format=avcc (4-byte length-prefixed samples without parameter sets / filler; extradata = the ISO/IEC 14496-15 §5.3.2.1 AVCDecoderConfigurationRecord from the avcc module — builder + parser + sample framing). Gates: every layout × CAVLC / CABAC decodes in our decoder within the lossy bound and byte-identically in a black-box reference decoder; lossless 4:2:2 / 4:4:4 / 10- / 12-bit / grey exact in both; avcC extradata + samples decode identically to the Annex B stream. Intra_4x4 at > 8 bit (intra_4x4, default on; DeepSession::with_intra_4x4): §8.3.1 block-sequential prediction at 8 / 10 / 12 / 14 bit with the §8.3.1.1 predicted-mode coding, the §8.5.15 V / H DPCM under transform bypass and the §8.3.4.5 4:4:4 chroma planes, CAVLC + CABAC, competing with Intra_16x16 per macroblock on Σ (SAD + mode bits · λ) — bit-exact in our decoder and byte-exact in the reference decoder across 8 layouts × both entropy coders, lossy and lossless; lossless 12-bit 4:4:4 shrinks 49572 → 33308 B, lossy 10-bit 4:2:0 3588 → 3446 B at equal PSNR (QP 26). Measured (64x48 synthetic, 3 pictures, CAVLC, bytes / min luma PSNR): 4:2:2 8-bit 3389 B / 45.2 dB at QP 20, 2009 B / 38.4 dB at QP 28, 1048 B / 30.8 dB at QP 36; 4:2:0 10-bit 2429 / 45.6, 1479 / 38.2, 858 / 30.9; 4:4:4 12-bit 4970 / 45.7, 3009 / 38.1, 1492 / 30.9. Not yet on this path: 4:0:0 (chroma_format_idc = 0) coding for grey input, Intra_8x8 and Intra_NxN inside P pictures at > 8 bit, B pictures and rate control for the High-family layouts.

Goals

• Spec-faithful: every clause of §7, §8, §9 implemented from the spec text. Comments cite the clause number being implemented.
• Pure Rust: no unsafe outside what the std-lib already requires; no C dependencies.
• No external decoder code: nothing copied from any reference implementation. Conformance is verified by output diff against an independent reference decoder, not by code reading.
• Pluggable into the existing oxideav Decoder / Encoder trait surface (decoder AND encoder factories are registered with the codec registry; both stay directly callable per the dual-API convention).

Benchmarks

Criterion benches live under benches/:

• inter_pred_bench — §8.4.2 luma + chroma fractional interpolation (scalar / chunked / default 6-tap kernels, 4x4 / 8x8 / 16x16 blocks).
• transform_bench — §8.5 inverse 4x4 and 8x8 transforms at QP 26.
• decode — end-to-end H264CodecDecoder over five in-process corpora: 64x64 + 128x96 Baseline IDR, IDR+4×P Baseline chain, IDR+P+B Main GOP, and a 64x64 High 4:2:2 IDR. All streams are synthesised from oxideav-h264's own encoder so the bench needs no on-disk fixtures.
• encode — sibling to decode, driving the public encoder entry points across seven in-process variants: Baseline CAVLC IDR (64x64 + 128x96), Baseline IDR + 4×P chain (real per-MB integer ME), Main IDR + P + B GOP (§8.4.1.2.2 spatial-direct + bipred), CABAC IDR + IDR + P (encode_idr_cabac / encode_p_cabac with the trellis-quant refinement on), and High 4:2:2 IDR. Reports both per-frame (Throughput::Elements, ≈ frames-encoded per second) and per-source-byte (Throughput::Bytes, MiB/s of raw planar input absorbed). The encoded-stream byte count is a compression-efficiency metric and stays in the integration tests rather than the bench.

Run with cargo bench --bench decode -- --quick or cargo bench --bench encode -- --quick for fast smokes, or omit --quick for the full Criterion report.

Profiles + features in scope

The long-term target is a complete implementation of ITU-T Rec. H.264 | ISO/IEC 14496-10 — every profile and every coding tool in the spec. We're starting from the subset in widespread use and widening out from there:

Intermediate profiles (High 10 Intra, CAVLC 4:4:4 Intra, etc.) land opportunistically as the pieces they need go in. Nothing in the spec is considered out-of-scope forever — the table is a phasing plan, not a scope fence.

License

MIT — see LICENSE.