container_probe/lib.rs
1//! Robust media **container-format detection** over a caller-owned byte slice.
2//!
3//! `container-probe` identifies whether a byte prefix (a file's leading bytes,
4//! a network read) is an MPEG-2 Transport Stream, an ISOBMFF file, a
5//! Matroska/WebM file, MXF, MPEG—PS, FLV, WAV, Ogg, ASF, or a raw ADTS AAC /
6//! MP3 / Annex B elementary stream. One-shot over a slice: the crate holds no
7//! buffer, owns no IO, and keeps no state.
8//!
9//! # Detected formats
10//!
11//! | Format | How it is detected | Best confidence tier |
12//! |---|---|---|
13//! | MPEG-2 TS | sync lattice over 188/192/204/208-byte strides | `LATTICE_STRONG` |
14//! | ISOBMFF | box-chain walk (ISO/IEC 14496-12 §4.2) | `STRUCTURAL` |
15//! | Matroska/WebM | EBML magic + `DocType` | `CERTAIN` |
16//! | MXF | partition-pack key + BER length | `CERTAIN` |
17//! | MPEG-PS | pack start code + marker bits | `STRUCTURAL` |
18//! | FLV | `"FLV"` signature + header fields | `STRONG` |
19//! | WAV | `"RIFF".."WAVE"` | `STRONG` |
20//! | Ogg | `"OggS"` | `STRONG` |
21//! | ASF | 16-byte header GUID | `STRONG` |
22//! | ADTS AAC | frame-length chain | `LATTICE_STRONG` |
23//! | MP3 | frame-length chain | `LATTICE_STRONG` |
24//! | Annex B (H.264) | start-code NAL chain | `LATTICE_STRONG` |
25//!
26//! # Known gaps
27//!
28//! - **Annex B detection is H.264 only.** HEVC (H.265) NAL units use a 2-byte
29//! header with `nal_unit_type` at bits `[6:1]`, which this prober does not
30//! parse — it validates the H.264 1-byte header, so an HEVC stream fails the
31//! range check at the first NAL. HEVC is deliberately not implemented: this
32//! workspace does not implement a format without a real fixture to test it
33//! against, and no HEVC Annex B fixture exists in the repository.
34//!
35//! # The scored confidence model
36//!
37//! Every registered prober runs over the same bytes and returns a scored
38//! candidate (an `Evidence`): a `Confidence` tier and a `Detail`. All probers
39//! **always** run, so the answer does not depend on declaration order. The
40//! highest score wins:
41//!
42//! | Tier | Value | Meaning |
43//! |---|---|---|
44//! | `CERTAIN` | 240 | Magic **plus** a structural confirmation |
45//! | `STRONG` | 192 | Unambiguous magic at a defined offset |
46//! | `STRUCTURAL` | 160 | A validated structure chain |
47//! | `LATTICE_STRONG` | 128 | `>= 8` lattice/frame confirmations |
48//! | `LATTICE_WEAK` | 96 | 3-7 lattice/frame confirmations |
49//! | `HEURISTIC` | 64 | A signature with real false-positive risk |
50//!
51//! If the top two candidates are within `TIE_THRESHOLD` (16), the result is
52//! [`Probe::Ambiguous`] with every candidate listed — never an arbitrary pick.
53//! A container matched at `LATTICE_STRONG` or above zeroes every
54//! elementary-stream candidate (ADTS/MP3/Annex B), on the principle that ES
55//! frames inside a container payload are expected data, not evidence the file
56//! is a raw stream.
57//!
58//! # `Insufficient` vs `Unknown`
59//!
60//! These are deliberately distinct, and a caller must not conflate them:
61//!
62//! - [`Probe::Insufficient { need_at_least }`](Probe) means **read more bytes** —
63//! nothing conclusive matched yet, but a longer buffer could change that. Use
64//! `need_at_least` to decide how much more to buffer.
65//! - [`Probe::Unknown`] means **stop** — nothing matched and more bytes will not
66//! help.
67//!
68//! ## The loop that terminates
69//!
70//! `need_at_least` is guaranteed to exceed the number of bytes the probe
71//! actually **examined**, which is `min(len, budget)` — not `len`. So a caller
72//! that only grows the buffer, while leaving [`probe`]'s fixed
73//! `DEFAULT_BUDGET` in place, can stall once the budget caps the read. Feed
74//! `need_at_least` back as the budget too:
75//!
76//! ```no_run
77//! # use container_probe::{probe_with_budget, Probe, DEFAULT_BUDGET};
78//! # fn read_at_least(_n: usize) -> Vec<u8> { Vec::new() }
79//! let mut buf = read_at_least(DEFAULT_BUDGET);
80//! let verdict = loop {
81//! match probe_with_budget(&buf, buf.len()) {
82//! Probe::Insufficient { need_at_least, .. } if need_at_least > buf.len() => {
83//! let grown = read_at_least(need_at_least);
84//! // EOF with nothing conclusive: stop. The probe cannot answer
85//! // from this file, and no further read will change that.
86//! if grown.len() <= buf.len() {
87//! break Probe::Unknown;
88//! }
89//! buf = grown;
90//! }
91//! other => break other,
92//! }
93//! };
94//! # let _ = verdict;
95//! ```
96//!
97//! Passing `buf.len()` as the budget is what makes each turn examine ground the
98//! last one did not. The `grown.len() <= buf.len()` arm is the caller's own
99//! termination guarantee at EOF, and is not optional: the crate cannot know
100//! whether more bytes exist.
101//!
102//! # `no_std` + `alloc`
103//!
104//! The crate is `#![no_std]` and links only `alloc`. Its single runtime
105//! dependency is `broadcast-common`. The only allocation is the candidate `Vec`
106//! of a genuine `Ambiguous` result. Build it without the default features for a
107//! pure-`alloc` target.
108//!
109//! # Example
110//!
111//! ```
112//! use container_probe::{probe, Probe};
113//!
114//! // An empty slice is `Insufficient`: reading more bytes could make it any
115//! // registered format, so a caller must not stop — it must read more. This is
116//! // the honest contract for a buffer that has not yet ruled anything out.
117//! let p = probe(&[]);
118//! match p {
119//! Probe::Insufficient { need_at_least, .. } => assert!(need_at_least >= 1),
120//! _ => unreachable!("an empty slice cannot be concluded from"),
121//! }
122//! ```
123//!
124//! # Non-goals
125//!
126//! - **No demuxing** — the probe identifies a format; parsing its content is a
127//! demuxer's job.
128//! - **No codec identification** — "this is TS" is the answer, not "this TS
129//! carries H.264".
130//! - **No file IO** — `no_std`; the caller supplies bytes.
131//! - **No format conversion or repair** — identification only.
132//! - **No incremental/streaming API** — the probe is one-shot over a slice; a
133//! streaming caller reads more and re-probes, guided by `Insufficient`'s
134//! `need_at_least`.
135//!
136//! `no_std` + `alloc`; runtime dependency is `broadcast-common` only.
137
138#![no_std]
139#![forbid(unsafe_code)]
140#![cfg_attr(docsrs, feature(doc_cfg))]
141// `container-probe` README is the canonical quick-start surface; `readme =
142// "README.md"` in Cargo.toml makes docs.rs/crates.io render it, and this line
143// also feeds its `rust` code blocks to rustdoc as doctests. The `doctest` cfg is
144// set by rustdoc only when collecting doctests, so the README is compiled as
145// doctests (never part of the build), which is what keeps the headline example
146// from silently rotting (it once matched all four `Probe` variants with no
147// wildcard arm, an E0004 for any downstream consumer, and nothing caught it).
148#![cfg_attr(doctest, doc = include_str!("../README.md"))]
149
150extern crate alloc;
151
152// Unit tests in the prober modules need `std` for fixture file IO; the test
153// build links it even though the crate is `#![no_std]`.
154#[cfg(test)]
155extern crate std;
156
157mod adts;
158mod annexb;
159mod asf;
160mod ebml;
161mod flv;
162mod isobmff;
163mod mp3;
164mod mpegps;
165mod mxf;
166mod ogg;
167mod riff;
168mod ts;
169
170use alloc::vec::Vec;
171
172/// MXF Partition Pack kind — the `PartitionKind` byte, byte 14 of the
173/// Partition Pack Key UL (SMPTE ST 377-1 §7.2-7.4).
174///
175/// Typed rather than a raw `u8` because the spec names these values: leaving
176/// the byte exposed made every consumer re-implement the same three-way lookup,
177/// which is what the decode-completeness rule exists to prevent. The crate
178/// already solves the identical problem with `IsobmffLayout` and `DocType`.
179#[non_exhaustive]
180#[derive(Debug, Clone, Copy, PartialEq, Eq)]
181pub enum PartitionKind {
182 /// Header Partition (`0x02`) — ST 377-1 §7.2.
183 Header,
184 /// Body Partition (`0x03`) — ST 377-1 §7.3.
185 Body,
186 /// Footer Partition (`0x04`) — ST 377-1 §7.4.
187 Footer,
188 /// A value outside the range this crate's prober accepts. Carried rather
189 /// than discarded so `Display` stays lossless; read it with
190 /// [`PartitionKind::as_u8`], which is why the field itself need not be
191 /// matchable downstream.
192 #[non_exhaustive]
193 Other(u8),
194}
195
196impl PartitionKind {
197 /// The spec token for this value.
198 pub fn name(&self) -> &'static str {
199 match self {
200 PartitionKind::Header => "Header",
201 PartitionKind::Body => "Body",
202 PartitionKind::Footer => "Footer",
203 PartitionKind::Other(_) => "reserved",
204 }
205 }
206
207 /// The wire byte this kind was decoded from.
208 pub fn as_u8(&self) -> u8 {
209 match self {
210 PartitionKind::Header => 0x02,
211 PartitionKind::Body => 0x03,
212 PartitionKind::Footer => 0x04,
213 PartitionKind::Other(b) => *b,
214 }
215 }
216}
217
218broadcast_common::impl_spec_display!(PartitionKind, Other);
219
220/// The identified container/stream format.
221///
222/// Each variant maps to exactly one prober module. `#[non_exhaustive]` so
223/// adding a format is a minor bump.
224#[non_exhaustive]
225#[derive(Debug, Clone, Copy, PartialEq, Eq)]
226pub enum Format {
227 /// MPEG-2 Transport Stream — ISO/IEC 13818-1, 188-byte packets.
228 MpegTs,
229 /// ISO Base Media File Format (`.mp4`/`.mov`/CMAF) — ISO/IEC 14496-12.
230 Isobmff,
231 /// MPEG Program Stream — ISO/IEC 13818-1 §2.5.
232 MpegPs,
233 /// Matroska — EBML container.
234 Matroska,
235 /// WebM — an EBML container with `DocType "webm"`.
236 WebM,
237 /// FLV — Adobe Flash Video, File Format Specification v10.1.
238 Flv,
239 /// Material Exchange Format — SMPTE ST 377-1.
240 Mxf,
241 /// RIFF/WAVE — Microsoft WAV.
242 Wav,
243 /// Ogg — Xiph Ogg container.
244 Ogg,
245 /// ASF — Advanced Systems Format.
246 Asf,
247 /// Raw ADTS AAC elementary stream — ISO/IEC 13818-7.
248 AdtsAac,
249 /// Raw MPEG-1/2 Layer III (MP3) elementary stream — ISO/IEC 11172-3.
250 Mp3,
251 /// Annex B NAL-unit byte stream — ITU-T H.264 only (HEVC is not detected;
252 /// see the crate-root "Known gaps").
253 AnnexB,
254}
255
256impl Format {
257 /// The human-readable name of the format.
258 pub fn name(&self) -> &'static str {
259 match self {
260 Format::MpegTs => "MpegTs",
261 Format::Isobmff => "Isobmff",
262 Format::MpegPs => "MpegPs",
263 Format::Matroska => "Matroska",
264 Format::WebM => "WebM",
265 Format::Flv => "Flv",
266 Format::Mxf => "Mxf",
267 Format::Wav => "Wav",
268 Format::Ogg => "Ogg",
269 Format::Asf => "Asf",
270 Format::AdtsAac => "AdtsAac",
271 Format::Mp3 => "Mp3",
272 Format::AnnexB => "AnnexB",
273 }
274 }
275
276 /// `true` when this format is an elementary (packetised payload) stream
277 /// that a strong container match should suppress — ADTS AAC, MP3, and
278 /// Annex B. These carry no container framing of their own and routinely
279 /// appear *inside* a container's payload, so a container verdict at
280 /// `LATTICE_STRONG` or above must outvote them (see
281 /// `suppress_elementary_streams`).
282 pub fn is_elementary_stream(&self) -> bool {
283 matches!(self, Format::AdtsAac | Format::Mp3 | Format::AnnexB)
284 }
285}
286
287broadcast_common::impl_spec_display!(Format);
288
289/// Evidence strength behind a match, in named tiers (see the `TIER_*`
290/// constants and the crate-root confidence model).
291///
292/// The score is set once by the prober that produced the candidate; the
293/// harness only compares them. A higher value is stronger evidence. The value
294/// is opaque; read it with [`Confidence::as_u8`].
295#[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord)]
296pub struct Confidence(u8);
297
298impl Confidence {
299 /// The raw score.
300 #[must_use]
301 pub const fn as_u8(self) -> u8 {
302 self.0
303 }
304
305 /// The human-readable name of this confidence tier (the `TIER_*` table).
306 pub fn name(&self) -> &'static str {
307 match self.0 {
308 TIER_CERTAIN => "CERTAIN",
309 TIER_STRONG => "STRONG",
310 TIER_STRUCTURAL => "STRUCTURAL",
311 TIER_LATTICE_STRONG => "LATTICE_STRONG",
312 TIER_LATTICE_WEAK => "LATTICE_WEAK",
313 TIER_HEURISTIC => "HEURISTIC",
314 _ => "unknown",
315 }
316 }
317}
318
319broadcast_common::impl_spec_display!(Confidence);
320
321/// How an ISOBMFF file carries its sample metadata — the discriminator a
322/// consumer needs to choose a demuxer.
323///
324/// ISO/IEC 14496-12 allows both shapes and they are demuxed differently: a
325/// fragmented file's samples are described by `moof` movie fragments (§8.8,
326/// the CMAF/fMP4 shape), a progressive file's by `moov` sample tables (§8.7).
327/// The prober walks the top-level boxes anyway, so it reports what it saw
328/// rather than making every consumer re-walk the chain to find out — the same
329/// reason [`Detail::Ts`] carries the stride and phase it measured.
330#[non_exhaustive]
331#[derive(Debug, Clone, Copy, PartialEq, Eq)]
332pub enum IsobmffLayout {
333 /// A `moof` box was seen: fragmented (CMAF / fMP4 / DASH segment).
334 Fragmented,
335 /// A `moov` box was seen and no `moof`: progressive, with sample tables.
336 Progressive,
337 /// Neither was seen within the probed region — e.g. a buffer holding only
338 /// `ftyp` and the start of a large `mdat`. The file is ISOBMFF, but which
339 /// shape is undetermined from these bytes; a consumer should read further
340 /// rather than assume.
341 Unknown,
342}
343
344impl IsobmffLayout {
345 /// Stable label.
346 pub fn name(&self) -> &'static str {
347 match self {
348 IsobmffLayout::Fragmented => "fragmented",
349 IsobmffLayout::Progressive => "progressive",
350 IsobmffLayout::Unknown => "unknown",
351 }
352 }
353}
354
355broadcast_common::impl_spec_display!(IsobmffLayout);
356
357/// What a prober learned on the way to its conclusion — the difference between
358/// "it is TS" and "it is TS, 192-byte stride, first sync at offset 4".
359#[non_exhaustive]
360#[derive(Debug, Clone, Copy, PartialEq, Eq)]
361pub enum Detail {
362 /// MPEG-2 TS lattice: packet `stride` (188/192/204/208) and the byte
363 /// offset of the first sync (`phase`), so a demuxer need not re-derive them.
364 #[non_exhaustive]
365 Ts {
366 /// Byte distance between consecutive sync bytes (a packet length).
367 stride: u16,
368 /// Byte offset of the first sync byte in the probed region.
369 phase: u16,
370 },
371 /// ISOBMFF: the `ftyp` major brand (if seen) and how many top-level boxes
372 /// chained cleanly.
373 #[non_exhaustive]
374 Isobmff {
375 /// The 4-character code from the `ftyp` major brand, if one was read.
376 major_brand: Option<[u8; 4]>,
377 /// Number of top-level boxes that chained cleanly into the buffer.
378 boxes_walked: u8,
379 /// Which structural layout the top-level walk observed — the
380 /// discriminator between a fragmented and a progressive file.
381 layout: IsobmffLayout,
382 },
383 /// Matroska/WebM: the EBML header's `DocType` string.
384 #[non_exhaustive]
385 Ebml {
386 /// The EBML DocType.
387 doc_type: DocType,
388 },
389 /// FLV: the header's `TypeFlags` (which tag types are present) and the
390 /// `DataOffset` header size.
391 #[non_exhaustive]
392 Flv {
393 /// `TypeFlags` bit 0 — audio tags present.
394 has_audio: bool,
395 /// `TypeFlags` bit 2 — video tags present.
396 has_video: bool,
397 /// The `DataOffset` field (bytes 5..9), the header size.
398 data_offset: u32,
399 },
400 /// MXF: the decoded Partition Pack kind (byte 14 of the UL).
401 #[non_exhaustive]
402 Mxf {
403 /// Which Partition Pack this is — see [`PartitionKind`].
404 partition_kind: PartitionKind,
405 },
406 /// MPEG-PS: whether the pack header's SCR/mux-rate marker bits validated.
407 #[non_exhaustive]
408 MpegPs {
409 /// `true` when the SCR `'01'` prefix, the four SCR marker bits and the
410 /// two `program_mux_rate` marker bits all validated (`STRUCTURAL`);
411 /// `false` when only the pack start code matched (`HEURISTIC`).
412 structurally_valid: bool,
413 },
414 /// No format-specific detail to report.
415 None,
416}
417
418impl Detail {
419 /// The human-readable name of the detail variant.
420 pub fn name(&self) -> &'static str {
421 match self {
422 Detail::Ts { .. } => "Ts",
423 Detail::Isobmff { .. } => "Isobmff",
424 Detail::Ebml { .. } => "Ebml",
425 Detail::Flv { .. } => "Flv",
426 Detail::Mxf { .. } => "Mxf",
427 Detail::MpegPs { .. } => "MpegPs",
428 Detail::None => "None",
429 }
430 }
431
432 /// The `ftyp`/`styp` major brand as a string, when the identifying file
433 /// carried one (an ISOBMFF result). The brand is a registered 4-character
434 /// code, so this is the ergonomic view of the raw `[u8; 4]` in
435 /// [`Detail::Isobmff`].
436 ///
437 /// Returns `None` when the detail is not `Isobmff`, when no brand was
438 /// observed, or when the 4 bytes are not valid UTF-8.
439 #[must_use]
440 pub fn major_brand_str(&self) -> Option<&str> {
441 if let Detail::Isobmff { major_brand, .. } = self {
442 // `major_brand` is a reference into `self` (match ergonomics on the
443 // connected `&self`), so the returned `&str` borrows from `self`.
444 let bytes: &[u8; 4] = major_brand.as_ref()?;
445 core::str::from_utf8(&bytes[..]).ok()
446 } else {
447 None
448 }
449 }
450}
451
452/// A lossless `Display` for [`Detail`]: every data-bearing variant renders its
453/// fields, so `Detail::Ts { stride, phase }` is not collapsed to just `"Ts"`.
454/// `Display` delegates to [`Detail::name`] (the #204 convention) for the label,
455/// then appends the field data the variant actually carries.
456impl core::fmt::Display for Detail {
457 fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
458 // The label comes from `name()`, never from a literal in these arms.
459 // It used to be spelled out in every arm, which made the doc above
460 // ("delegates to `Detail::name`") false and let the two drift silently:
461 // `api_labels.rs` pins only `Detail::None`, so a renamed variant would
462 // have reported one label from `name()` and another from `Display`.
463 f.write_str(self.name())?;
464 match self {
465 Detail::Ts { stride, phase } => {
466 write!(f, " {{ stride: {stride}, phase: {phase} }}")
467 }
468 Detail::Isobmff {
469 major_brand,
470 boxes_walked,
471 layout,
472 } => {
473 let brand = major_brand
474 .map(|b| alloc::string::String::from_utf8_lossy(&b[..]).into_owned())
475 .unwrap_or_else(|| "<none>".into());
476 write!(
477 f,
478 " {{ major_brand: {brand:?}, boxes_walked: {boxes_walked}, layout: {layout} }}"
479 )
480 }
481 Detail::Ebml { doc_type } => write!(f, " {{ doc_type: {doc_type} }}"),
482 Detail::Flv {
483 has_audio,
484 has_video,
485 data_offset,
486 } => write!(
487 f,
488 " {{ has_audio: {has_audio}, has_video: {has_video}, data_offset: {data_offset} }}"
489 ),
490 Detail::Mxf { partition_kind } => {
491 write!(f, " {{ partition_kind: {partition_kind} }}")
492 }
493 Detail::MpegPs { structurally_valid } => {
494 write!(f, " {{ structural: {structurally_valid} }}")
495 }
496 // No fields to append.
497 Detail::None => Ok(()),
498 }
499 }
500}
501
502/// The EBML `DocType` string (`"webm"` or `"matroska"`) decoded from an EBML
503/// header's `EBMLDocType` element.
504#[non_exhaustive]
505#[derive(Debug, Clone, Copy, PartialEq, Eq)]
506pub enum DocType {
507 /// `DocType == "webm"` (WebM container).
508 Webm,
509 /// `DocType == "matroska"` (Matroska container).
510 Matroska,
511 /// Any other, unrecognised `DocType` string.
512 Other,
513}
514
515impl DocType {
516 /// The human-readable name of the DocType.
517 pub fn name(&self) -> &'static str {
518 match self {
519 DocType::Webm => "Webm",
520 DocType::Matroska => "Matroska",
521 DocType::Other => "Other",
522 }
523 }
524}
525
526broadcast_common::impl_spec_display!(DocType);
527
528/// One scored candidate: a format, the evidence strength behind it, and what
529/// the prober learned on the way.
530///
531/// `#[non_exhaustive]` so adding a field is not a breaking change.
532#[non_exhaustive]
533#[derive(Debug, Clone, Copy, PartialEq, Eq)]
534pub struct Candidate {
535 /// The candidate container/stream format.
536 pub format: Format,
537 /// The evidence strength behind this candidate.
538 pub confidence: Confidence,
539 /// Prober-specific detail.
540 pub detail: Detail,
541}
542
543/// What the probe concluded.
544#[non_exhaustive]
545#[derive(Debug, Clone, PartialEq, Eq)]
546pub enum Probe {
547 /// A single best match.
548 #[non_exhaustive]
549 Identified {
550 /// The winning format.
551 format: Format,
552 /// Its evidence strength.
553 confidence: Confidence,
554 /// Prober detail.
555 detail: Detail,
556 },
557 /// Two or more candidates within `TIE_THRESHOLD`, ordered by descending
558 /// confidence. A caller that wants a decision takes the first; a caller
559 /// that wants correctness refuses.
560 #[non_exhaustive]
561 Ambiguous {
562 /// The tied candidates, best first.
563 candidates: Vec<Candidate>,
564 },
565 /// Nothing matched, but a longer buffer could change that.
566 #[non_exhaustive]
567 Insufficient {
568 /// The minimum buffer length that could plausibly resolve a match.
569 need_at_least: usize,
570 },
571 /// Nothing matched and more bytes will not help.
572 Unknown,
573}
574
575impl Probe {
576 /// The human-readable name of the probe outcome variant.
577 pub fn name(&self) -> &'static str {
578 match self {
579 Probe::Identified { .. } => "Identified",
580 Probe::Ambiguous { .. } => "Ambiguous",
581 Probe::Insufficient { .. } => "Insufficient",
582 Probe::Unknown => "Unknown",
583 }
584 }
585}
586
587broadcast_common::impl_spec_display!(Probe);
588
589impl Probe {
590 /// The `Confidence` of the winning candidate, if any — a convenience for
591 /// the common `Identified` / `Ambiguous` first-wins read.
592 ///
593 /// Returns `None` for `Insufficient`/`Unknown`.
594 #[must_use]
595 pub fn best_confidence(&self) -> Option<Confidence> {
596 match self {
597 Probe::Identified { confidence, .. } => Some(*confidence),
598 Probe::Ambiguous { candidates } => candidates.first().map(|c| c.confidence),
599 Probe::Insufficient { .. } | Probe::Unknown => None,
600 }
601 }
602}
603
604// ---------------------------------------------------------------------------
605// Confidence model — named tiers, no bare numbers (design §"Confidence model").
606// ---------------------------------------------------------------------------
607
608/// Unambiguous magic at a defined offset **plus** a structural check confirming
609/// it (e.g. EBML magic + valid DocType, MXF partition key + well-formed BER).
610const TIER_CERTAIN: u8 = 240;
611/// Unambiguous magic at a defined offset, no further validation available
612/// (FLV signature, `RIFF`…`WAVE`, `OggS`, ASF header GUID).
613const TIER_STRONG: u8 = 192;
614/// A validated structure chain, not merely a signature (an ISOBMFF box chain
615/// whose sizes fit the buffer; a MPEG-PS pack header with valid marker bits).
616const TIER_STRUCTURAL: u8 = 160;
617/// A repeating sync lattice with many confirmations (`>= 8` TS syncs at a
618/// consistent stride).
619const TIER_LATTICE_STRONG: u8 = 128;
620/// A repeating lattice with few confirmations (3-7 TS syncs).
621const TIER_LATTICE_WEAK: u8 = 96;
622/// A signature with meaningful false-positive probability (bare MPEG-PS pack
623/// start code, bare AnnexB start code).
624const TIER_HEURISTIC: u8 = 64;
625/// Two candidates whose scores are within this gap are reported as
626/// `Probe::Ambiguous` rather than silently choosing one.
627const TIE_THRESHOLD: u8 = 16;
628/// Default byte budget for [`probe`] — comfortably above the worst case (a
629/// 208-byte-stride TS lattice needing 8 confirmations plus a full phase
630/// search). Design §Performance.
631///
632/// Public because a caller draining an [`Probe::Insufficient`] loop needs it:
633/// `probe` never examines more than this many bytes however long the buffer
634/// is, so growing the buffer alone can stall. See the crate-root "The loop
635/// that terminates".
636pub const DEFAULT_BUDGET: usize = 64 * 1024;
637
638/// The named tiers, exposed for tests. A prober assigns one of these (e.g.
639/// [`Confidence::LATTICE_STRONG`]).
640impl Confidence {
641 /// Unambiguous magic **plus** a structural confirmation (240).
642 pub const CERTAIN: Confidence = Confidence(TIER_CERTAIN);
643 /// Unambiguous magic at a defined offset only (192).
644 pub const STRONG: Confidence = Confidence(TIER_STRONG);
645 /// A validated structure chain (160).
646 pub const STRUCTURAL: Confidence = Confidence(TIER_STRUCTURAL);
647 /// `>= 8` lattice confirmations at a consistent stride (128).
648 pub const LATTICE_STRONG: Confidence = Confidence(TIER_LATTICE_STRONG);
649 /// 3-7 lattice confirmations (96).
650 pub const LATTICE_WEAK: Confidence = Confidence(TIER_LATTICE_WEAK);
651 /// A signature with meaningful false-positive probability (64).
652 pub const HEURISTIC: Confidence = Confidence(TIER_HEURISTIC);
653}
654
655/// A scored match from one prober. Internal; `Candidate` is the public form.
656#[derive(Debug, Clone, Copy, PartialEq, Eq)]
657struct Evidence {
658 confidence: Confidence,
659 detail: Detail,
660}
661
662/// What one prober concluded about its format.
663#[derive(Debug, Clone, Copy, PartialEq, Eq)]
664enum Outcome {
665 /// The format matched with this evidence.
666 Match(Evidence),
667 /// The buffer may resolve with at least this many bytes.
668 Insufficient(usize),
669 /// No match, and more bytes will not change that.
670 None,
671}
672
673/// The single shared decision every prober must make: is "no match" a *proof of
674/// non-membership* — or merely the walk *running off the end of the supplied
675/// region* mid-structure?
676///
677/// This is the whole `Insufficient` vs `Unknown` contract in one place. The
678/// [`Probe::Unknown`] branch means "stop, more bytes will not help", which is
679/// only ever true when the prober examined data it *fully saw* and that data is
680/// invalid for the format. If the prober would have kept walking had the buffer
681/// been longer, it has proven nothing and must answer [`Outcome::Insufficient`]
682/// with `need` — a lower bound on bytes that could change the verdict.
683///
684/// `ran_out` signals which of the two happened: `true` when the prober stopped
685/// because the region ended mid-structure (→ [`Outcome::Insufficient`], "read
686/// more"), `false` when it examined the data and ruled the format out
687/// (→ [`Outcome::None`], "stop"). Re-deciding this judgement separately in
688/// each of the twelve probers is what let the contract break repeatedly at
689/// different lengths on different formats; it lives here so the only question a
690/// prober answers is the factual one — "did I run out of bytes?" — never the
691/// policy one.
692pub(crate) fn ran_out_or_ruled_out(ran_out: bool, need: usize) -> Outcome {
693 if ran_out {
694 Outcome::Insufficient(need)
695 } else {
696 Outcome::None
697 }
698}
699
700/// Force a prober's `need_at_least` to exceed the bytes actually examined.
701///
702/// `Insufficient` promises the caller that reading more can change the answer.
703/// A `need` at or below `limit` breaks that promise: the caller re-probes,
704/// examines the same ground, gets the same answer, and never advances.
705///
706/// Enforced here, once, rather than trusted from twelve probers — the crate has
707/// shipped this defect twice, from two different probers, with a green suite
708/// both times.
709///
710/// A prober's own structural need is kept when it is larger; that is the more
711/// useful hint (it names the byte the structure actually reaches, so one read
712/// suffices instead of one-byte-at-a-time crawling).
713///
714/// This is a **backstop**. Every prober is currently expected to report a
715/// structural need, so in normal operation this returns `need` unchanged. It is
716/// separated out rather than inlined so it can be tested directly: a guard with
717/// no reachable failing input is a guard nobody can trust.
718fn normalise_need(need: usize, limit: usize) -> usize {
719 core::cmp::max(need, geometric_floor(limit))
720}
721
722/// The smallest answer a prober with **no structural need** may give.
723///
724/// Half again as much as was examined, plus one. Strictly greater than `limit`,
725/// so the caller always advances, and *geometric*, so a caller that cannot name
726/// a structure still converges in O(log n) reads rather than O(n).
727///
728/// The arithmetic alternative is what shipped and had to be undone: every
729/// prober computed `max(structural_floor, have + unit)`, and once `have`
730/// overtook the floor the second term won, growing by one unit per turn —
731/// `+4` for Annex B, `+188` for TS. The documented caller loop then reached
732/// 36 bytes of a 256 KiB file after twelve reads. Terminating and useless are
733/// different things, and only a turn-count bound tells them apart.
734///
735/// Over-asking is safe for a lower bound because the caller clamps to EOF and
736/// re-probes what it actually got: a file shorter than this figure is probed
737/// whole on the next turn, so nothing that was decidable becomes undecidable.
738fn geometric_floor(limit: usize) -> usize {
739 limit.saturating_add(limit / 2).saturating_add(1)
740}
741
742/// A registered prober. Each prober is a pure function over a slice read no
743/// further than `limit` bytes.
744type Prober = fn(&[u8], limit: usize) -> Outcome;
745
746/// The registry of probers, in declaration order. **Adding a prober for a new
747/// format is a one-line change here**: append `(Format::X, x::probe)` and the
748/// harness picks it up. Order does not bias the result — all probers always
749/// run and the highest score wins.
750///
751/// One entry per format. The EBML prober is registered under `Format::Matroska`
752/// and reports `Format::WebM` (or stays Matroska) via its `Detail::Ebml`
753/// `DocType`, resolved by [`candidate_format`].
754const PROBERS: &[(Format, Prober)] = &[
755 (Format::MpegTs, ts::probe),
756 (Format::Isobmff, isobmff::probe),
757 (Format::Matroska, ebml::probe),
758 (Format::Mxf, mxf::probe),
759 (Format::MpegPs, mpegps::probe),
760 (Format::Flv, flv::probe),
761 (Format::Wav, riff::probe),
762 (Format::Ogg, ogg::probe),
763 (Format::Asf, asf::probe),
764 (Format::AdtsAac, adts::probe),
765 (Format::Mp3, mp3::probe),
766 (Format::AnnexB, annexb::probe),
767];
768
769/// Resolve the final candidate `Format`, given a registry format and the
770/// prober's `Detail`. Only the EBML prober ever disagrees: `Detail::Ebml` with
771/// `DocType::Webm` is a WebM, anything else is a Matroska (the registry label).
772fn candidate_format(format: Format, detail: Detail) -> Format {
773 match detail {
774 Detail::Ebml { doc_type } => match doc_type {
775 DocType::Webm => Format::WebM,
776 DocType::Matroska | DocType::Other => Format::Matroska,
777 },
778 _ => format,
779 }
780}
781
782/// Cross-prober **suppression**: when any container matches at
783/// `LATTICE_STRONG` (128) or above, every elementary-stream candidate is
784/// dropped.
785///
786/// ADTS frames, MP3 frames, and Annex B NAL units routinely appear *inside* a
787/// real container's payload — they are the expected data, not evidence the file
788/// is raw elementary audio or video. A high-entropy container can also align
789/// enough syncwords to score weakly, so an elementary-stream candidate is never
790/// allowed to outvote or tie a genuine container. The container's high score is
791/// exactly the proof that it is (also) a container; ruling out the ES reading
792/// is the one-directional, named trade this function makes (design §"Confidence
793/// model", "Cross-prober suppression"). This must never be an implicit ordering
794/// a later edit can silently reverse — it is the explicit final transformation
795/// before scoring.
796fn suppress_elementary_streams(candidates: &mut Vec<Candidate>) {
797 let container_wins = candidates
798 .iter()
799 .any(|c| !c.format.is_elementary_stream() && c.confidence.as_u8() >= TIER_LATTICE_STRONG);
800 if container_wins {
801 candidates.retain(|c| !c.format.is_elementary_stream());
802 }
803}
804
805/// Probe with the default budget (`DEFAULT_BUDGET`).
806pub fn probe(data: &[u8]) -> Probe {
807 probe_with_budget(data, DEFAULT_BUDGET)
808}
809
810/// Probe reading at most `budget` bytes of `data`.
811///
812/// Every registered prober runs over the same bytes and scores its candidate;
813/// the highest score wins. If the top two are within `TIE_THRESHOLD` the
814/// result is `Ambiguous` (ordered by score); otherwise `Identified`. With no
815/// candidates, `Insufficient` if any prober could conclude from more bytes
816/// (reporting the smallest such `need_at_least`), else `Unknown`.
817pub fn probe_with_budget(data: &[u8], budget: usize) -> Probe {
818 // Read no further than this; `budget` may exceed the buffer.
819 let limit = core::cmp::min(data.len(), budget);
820
821 let mut candidates: Vec<Candidate> = Vec::new();
822 let mut need_more: Option<usize> = None;
823
824 for (format, prober) in PROBERS {
825 match prober(data, limit) {
826 Outcome::Match(ev) => candidates.push(Candidate {
827 format: candidate_format(*format, ev.detail),
828 confidence: ev.confidence,
829 detail: ev.detail,
830 }),
831 Outcome::Insufficient(need) => {
832 need_more = Some(match need_more {
833 Some(prev) => core::cmp::min(prev, need),
834 None => need,
835 });
836 }
837 Outcome::None => {}
838 }
839 }
840
841 // Cross-prober suppression: a container matched strongly -> zero the
842 // elementary-stream candidates before scoring (see
843 // `suppress_elementary_streams`).
844 suppress_elementary_streams(&mut candidates);
845
846 match candidates.len() {
847 0 => match need_more {
848 // `need_at_least` MUST exceed the bytes actually examined, or the
849 // contract is a lie and the documented caller loop never advances.
850 //
851 // This is enforced here, once, rather than trusted from twelve
852 // probers, because the crate has now shipped this defect twice from
853 // two different probers. `ebml` reported `region.len() + 1`, which
854 // looks like strict progress but is not: `limit` is capped at
855 // `budget`, so `region.len()` saturates at `DEFAULT_BUDGET` and the
856 // answer froze at 65537. Supplying more than that gave
857 // `need_at_least <= supplied` — a fixed point a caller obeying the
858 // contract spins on forever. `mp3`'s ID3 skip reached the same
859 // fixed point from an honest, structure-derived need that simply
860 // exceeded the budget.
861 //
862 // Normalising to a geometric floor guarantees the number always asks for
863 // ground not yet examined. A prober's own structural need is kept
864 // when it is larger, since that is the more useful hint.
865 Some(need) => Probe::Insufficient {
866 need_at_least: normalise_need(need, limit),
867 },
868 None => Probe::Unknown,
869 },
870 _ => {
871 // Descending confidence, then a stable format-name tie-break.
872 candidates.sort_by(|a, b| {
873 b.confidence
874 .cmp(&a.confidence)
875 .then_with(|| a.format.name().cmp(b.format.name()))
876 });
877 probe_to_identify(&candidates)
878 }
879 }
880}
881
882fn probe_to_identify(candidates: &[Candidate]) -> Probe {
883 let top = &candidates[0];
884 if let Some(second) = candidates.get(1) {
885 // Scores are within `TIE_THRESHOLD` -> genuinely ambiguous. List only
886 // the candidates actually within `TIE_THRESHOLD` of the winner (the
887 // tied set), not every lower-scored also-ran: `Probe::Ambiguous`
888 // documents "two or more candidates within `TIE_THRESHOLD`", and a
889 // candidate well below the winner is not part of that tie.
890 if top
891 .confidence
892 .as_u8()
893 .saturating_sub(second.confidence.as_u8())
894 <= TIE_THRESHOLD
895 {
896 let tied: Vec<Candidate> = candidates
897 .iter()
898 .copied()
899 .take_while(|c| {
900 top.confidence.as_u8().saturating_sub(c.confidence.as_u8()) <= TIE_THRESHOLD
901 })
902 .collect();
903 return Probe::Ambiguous { candidates: tied };
904 }
905 }
906 Probe::Identified {
907 format: top.format,
908 confidence: top.confidence,
909 detail: top.detail,
910 }
911}
912
913#[cfg(test)]
914mod sampled_prober_needs {
915 //! A **sampled** check that a prober's `need_at_least` does not move with
916 //! the buffer, over the seeds and lengths below.
917 //!
918 //! # What this does and does not prove
919 //!
920 //! An earlier revision of this module was called
921 //! `no_length_relative_needs` and its commit said it "closed the class
922 //! across all 12 probers". An audit measured it: reverting five probers to
923 //! length-relative needs one at a time, it caught **two** — `annexb` and
924 //! `isobmff`. It missed `ts`, `adts` and `mp3`, which were the very sites
925 //! that commit set out to protect. The seeds and lengths simply do not
926 //! reach the branch where those needs are computed.
927 //!
928 //! It is kept because two is more than none, and renamed because a test
929 //! whose name asserts a closure it does not deliver is worse than no test:
930 //! it is what persuaded the author the class was closed.
931 //!
932 //! # Where the real guarantee lives
933 //!
934 //! Convergence does **not** depend on any prober being well behaved.
935 //! [`normalise_need`] raises every answer to a geometric floor, so the
936 //! documented caller loop converges in O(log n) reads no matter what a
937 //! prober reports — measured end to end: for a buffer of n bytes the public
938 //! API returns exactly `geometric_floor(n)` whenever a prober under-asks.
939 //! The guarantee is proved by `tests/insufficient_terminates.rs` (the loop
940 //! itself, bounded turns, over adversarial seeds) and by
941 //! `mod need_normalisation` (the floor's own algebra).
942 //!
943 //! A prober's need is therefore a **hint**: a better one saves a round
944 //! trip, a worse one cannot cause a crawl. For a chain walker the hint
945 //! legitimately grows as the walk advances — `isobmff` reports the end of
946 //! the box header it stopped in, which moves with how many boxes it got
947 //! through. That is progress, not a defect, and no invariant here forbids
948 //! it.
949
950 use super::*;
951 use alloc::vec;
952 use alloc::vec::Vec;
953
954 /// Seeds chosen to drive different probers into their "read more" path: a
955 /// leading magic or sync that commits the prober to looking further,
956 /// followed by nothing that resolves it.
957 fn seeds() -> Vec<(&'static str, Vec<u8>)> {
958 vec![
959 ("ts-sync", vec![0x47]),
960 (
961 "ebml-magic",
962 vec![
963 0x1A, 0x45, 0xDF, 0xA3, 0x01, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFE,
964 ],
965 ),
966 (
967 "isobmff-largesize",
968 vec![0x00, 0x00, 0x00, 0x01, b'f', b't', b'y', b'p'],
969 ),
970 ("adts-sync", vec![0xFF, 0xF1, 0x4C, 0x80, 0x22, 0x3F, 0xFC]),
971 // Mixed frame lengths (7, then 8191): a uniform-frame seed cannot
972 // expose an ADTS need computed from the buffer.
973 (
974 "adts-mixed",
975 vec![
976 0xFF, 0xF1, 0x00, 0x00, 0x00, 0xE0, 0x00, 0xFF, 0xF1, 0x00, 0x03, 0xFF, 0xE0,
977 0x00,
978 ],
979 ),
980 ("mp3-sync", vec![0xFF, 0xFB, 0x90, 0x64]),
981 // MPEG-1 Layer III, 44100 Hz, frame 1 at 32 kbps (104 bytes) then
982 // frame 2 at 320 kbps (1044). Same reason as `adts-mixed`: a
983 // uniform-bitrate seed cannot expose a need computed from the
984 // buffer, because the bound that hides it uses the first frame's
985 // length while truncation is set by the last.
986 ("mp3-mixed-bitrate", {
987 let mut v = vec![0xFF, 0xFB, 0x10, 0x00];
988 v.resize(104, 0x00);
989 v.extend_from_slice(&[0xFF, 0xFB, 0xE0, 0x00]);
990 v
991 }),
992 (
993 "id3",
994 vec![b'I', b'D', b'3', 0x04, 0x00, 0x00, 0x00, 0x00, 0x10, 0x00],
995 ),
996 ("annexb-startcode", vec![0x00, 0x00, 0x00, 0x01, 0x67]),
997 ("riff", vec![b'R', b'I', b'F', b'F']),
998 ("ogg", vec![b'O', b'g', b'g']),
999 (
1000 "mxf-key",
1001 vec![0x06, 0x0E, 0x2B, 0x34, 0x02, 0x05, 0x01, 0x01],
1002 ),
1003 ("ps-startcode", vec![0x00, 0x00, 0x01, 0xBA]),
1004 ("flv", vec![b'F', b'L', b'V']),
1005 (
1006 "asf-guid",
1007 vec![0x30, 0x26, 0xB2, 0x75, 0x8E, 0x66, 0xCF, 0x11],
1008 ),
1009 ]
1010 }
1011
1012 #[test]
1013 fn sampled_prober_needs_do_not_move_with_the_buffer() {
1014 // Short lengths are load-bearing, not padding. Most probers answer
1015 // `Insufficient(CONST)` only while the region is SHORTER than the
1016 // constant they need (a 16-byte MXF key, a 12-byte RIFF header), so a
1017 // length set that starts at 4 never reaches them at two lengths and the
1018 // guard goes blind on 7 of 12 probers. The per-prober coverage
1019 // assertion below is what surfaced that.
1020 let lengths = [
1021 1usize, 2, 3, 5, 6, 7, 10, 12, 14, 15, 20, 64, 256, 1024,
1022 // 1400 sits in the only window where a TS need computed as
1023 // `have + TS_PACKET_SIZE` overtakes the 1504-byte structural floor
1024 // while the prober still answers Insufficient. Without a length in
1025 // (1316, 1504) that revert is invisible -- an audit measured this
1026 // guard catching 2 of 5 probers, and this gap was one of the three.
1027 1400, 4096,
1028 ];
1029 let mut failures: Vec<alloc::string::String> = Vec::new();
1030 let mut exercised = 0usize;
1031 let mut covered: Vec<&Format> = Vec::new();
1032
1033 for (format, prober) in PROBERS {
1034 for (seed_name, seed) in seeds() {
1035 let mut seen: Vec<(usize, usize)> = Vec::new();
1036 for len in lengths {
1037 if len < seed.len() {
1038 continue;
1039 }
1040 let mut buf = seed.clone();
1041 buf.resize(len, 0x00);
1042 if let Outcome::Insufficient(need) = prober(&buf, buf.len()) {
1043 seen.push((len, need));
1044 }
1045 }
1046 if seen.len() < 2 {
1047 continue;
1048 }
1049 exercised += 1;
1050 covered.push(format);
1051 let first = seen[0].1;
1052 if seen.iter().any(|(_, n)| *n != first) {
1053 failures.push(alloc::format!(
1054 " {:?} on seed {seed_name}: {seen:?} -- the need moves with the \
1055 buffer length, so a caller advances one unit per read",
1056 format
1057 ));
1058 }
1059 }
1060 }
1061
1062 // Per-prober coverage, NOT an aggregate count. The first version of
1063 // this test asserted `exercised >= 6` across all pairs and was vacuous
1064 // for exactly the probers it was written for: reverting `ts` and `adts`
1065 // to length-relative needs left it green, because those two never
1066 // reached `Insufficient` at two lengths and other pairs made the total
1067 // look healthy. An aggregate threshold hides a per-prober hole -- which
1068 // is the same mistake as fixing one prober and claiming the class.
1069 let missed: Vec<&Format> = PROBERS
1070 .iter()
1071 .map(|(f, _)| f)
1072 .filter(|f| !covered.contains(f))
1073 .collect();
1074 assert!(
1075 missed.is_empty(),
1076 "no seed drives {missed:?} to Insufficient at two or more lengths, so this \
1077 guard cannot see a length-relative need in {} of {} probers. Add a seed \
1078 that reaches them rather than lowering the bar.",
1079 missed.len(),
1080 PROBERS.len()
1081 );
1082 let _ = exercised;
1083 assert!(
1084 failures.is_empty(),
1085 "a prober's need moved with the buffer at a SAMPLED length (this check is \
1086 sampled, not exhaustive -- see the module docs):\n{}",
1087 failures.join("\n")
1088 );
1089 }
1090}
1091
1092#[cfg(test)]
1093mod need_normalisation {
1094 use super::*;
1095
1096 /// A prober under-reporting its need must never reach the caller as a
1097 /// promise that cannot be kept.
1098 ///
1099 /// The two shipped instances of this defect: `ebml` reported
1100 /// `region.len() + 1`, which saturates at the budget and froze at 65537;
1101 /// `mp3`'s ID3 skip reported an honest structural need that simply exceeded
1102 /// the budget. Both left `need_at_least <= supplied` for a large enough
1103 /// buffer — a fixed point.
1104 ///
1105 /// MUTATION VERIFIED: replacing the body with `need` fails the first case
1106 /// below with `left: 10, right: 65537`.
1107 #[test]
1108 fn an_under_reported_need_is_raised_past_the_examined_bytes() {
1109 // A prober that under-reports badly at a large limit is raised to the
1110 // geometric floor (limit + limit/2 + 1), not merely to limit + 1: an
1111 // arithmetic bump converges in O(n) reads, which terminates but crawls.
1112 assert_eq!(normalise_need(10, 65536), 98_305);
1113 // Exactly at the limit is still a fixed point, so it must be raised.
1114 assert_eq!(normalise_need(4096, 4096), 6145);
1115 // Just past the limit still crawls, so it is raised too.
1116 assert_eq!(normalise_need(4097, 4096), 6145);
1117 // A larger structural need is the better hint and must be preserved,
1118 // never clamped down to the floor.
1119 assert_eq!(normalise_need(268_435_465, 65536), 268_435_465);
1120 // Degenerate: zero need at zero limit must still ask for something.
1121 assert_eq!(normalise_need(0, 0), 1);
1122 }
1123
1124 /// The invariant, stated over a range rather than at points, so a future
1125 /// formula that happens to satisfy the five cases above cannot pass.
1126 #[test]
1127 fn the_result_always_exceeds_the_limit() {
1128 for limit in [0usize, 1, 4, 188, 4095, 4096, 65535, 65536, 131_072] {
1129 for need in [0usize, 1, 4, 100, 65536, 65537, 1_000_000] {
1130 let got = normalise_need(need, limit);
1131 assert!(
1132 got > limit,
1133 "normalise_need({need}, {limit}) = {got}, which does not exceed the \
1134 {limit} bytes examined -- a caller would re-probe forever"
1135 );
1136 // Geometric, not merely greater: an answer of `limit + 1` still
1137 // converges in O(n) reads. Growth must be multiplicative so a
1138 // caller that cannot name a structure still finishes in O(log n).
1139 assert!(
1140 got >= limit + limit / 2,
1141 "normalise_need({need}, {limit}) = {got} grows arithmetically; \
1142 the floor must be at least 1.5x the bytes examined or the \
1143 documented caller loop crawls"
1144 );
1145 assert!(
1146 got >= need,
1147 "normalise_need({need}, {limit}) = {got} discarded the prober's \
1148 larger structural need {need}"
1149 );
1150 }
1151 }
1152 }
1153}
1154
1155#[cfg(test)]
1156mod tier_spacing {
1157 use super::*;
1158
1159 /// Every tier, highest first. Adding a tier without adding it here is
1160 /// caught by the exhaustiveness assertion below.
1161 const TIERS: [(&str, u8); 6] = [
1162 ("CERTAIN", TIER_CERTAIN),
1163 ("STRONG", TIER_STRONG),
1164 ("STRUCTURAL", TIER_STRUCTURAL),
1165 ("LATTICE_STRONG", TIER_LATTICE_STRONG),
1166 ("LATTICE_WEAK", TIER_LATTICE_WEAK),
1167 ("HEURISTIC", TIER_HEURISTIC),
1168 ];
1169
1170 /// The scoring model reports `Ambiguous` when the top two candidates are
1171 /// within `TIE_THRESHOLD`. For that to mean "the evidence is genuinely
1172 /// equal" rather than "the tiers happen to sit close together", adjacent
1173 /// tiers must be separated by strictly more than `TIE_THRESHOLD` — so a
1174 /// candidate on a *lower* tier can never tie with one above it, and only a
1175 /// same-tier collision is ever reported as ambiguous.
1176 ///
1177 /// This is the invariant behind moving `TIER_LATTICE_STRONG` 144 -> 128:
1178 /// at 144 the gap to `TIER_STRUCTURAL` (160) was exactly 16, so a
1179 /// `STRUCTURAL` container and a `LATTICE_STRONG` elementary stream were
1180 /// reported as tied despite the model ranking one strictly above the other.
1181 /// Without this guard the next tier added lands back in that trap silently.
1182 #[test]
1183 fn adjacent_tiers_are_further_apart_than_the_tie_threshold() {
1184 for pair in TIERS.windows(2) {
1185 let (hi_name, hi) = pair[0];
1186 let (lo_name, lo) = pair[1];
1187 assert!(
1188 hi > lo,
1189 "TIERS must be listed strictly descending: {hi_name} ({hi}) is not above {lo_name} ({lo})"
1190 );
1191 let gap = hi - lo;
1192 assert!(
1193 gap > TIE_THRESHOLD,
1194 "{hi_name} ({hi}) and {lo_name} ({lo}) are {gap} apart, which is not more than \
1195 TIE_THRESHOLD ({TIE_THRESHOLD}): two candidates on these different tiers would \
1196 be reported as Ambiguous even though the model ranks one strictly above the other"
1197 );
1198 }
1199 }
1200
1201 /// `TIERS` above must list every tier the crate defines. `Confidence`'s
1202 /// public constants are the enumeration of record, so pin the two together:
1203 /// a new `pub const` on `Confidence` whose value is missing from `TIERS`
1204 /// would otherwise skip the spacing check entirely.
1205 #[test]
1206 fn tiers_covers_every_public_confidence_constant() {
1207 let public = [
1208 ("CERTAIN", Confidence::CERTAIN),
1209 ("STRONG", Confidence::STRONG),
1210 ("STRUCTURAL", Confidence::STRUCTURAL),
1211 ("LATTICE_STRONG", Confidence::LATTICE_STRONG),
1212 ("LATTICE_WEAK", Confidence::LATTICE_WEAK),
1213 ("HEURISTIC", Confidence::HEURISTIC),
1214 ];
1215 assert_eq!(
1216 public.len(),
1217 TIERS.len(),
1218 "TIERS and Confidence's public tier constants have drifted apart"
1219 );
1220 for (name, c) in public {
1221 let found = TIERS.iter().find(|(n, _)| *n == name);
1222 let (_, v) = found.unwrap_or_else(|| panic!("tier {name} is missing from TIERS"));
1223 assert_eq!(
1224 *v,
1225 c.as_u8(),
1226 "tier {name}: TIERS says {v}, Confidence::{name} says {}",
1227 c.as_u8()
1228 );
1229 }
1230 }
1231}
1232
1233#[cfg(test)]
1234mod dispatch {
1235 use super::*;
1236 use alloc::vec;
1237
1238 /// Every `Format` variant must be *reachable*: either its own prober is a
1239 /// `PROBERS` row, or `candidate_format` can produce it from a registered
1240 /// one. Deleting a `PROBERS` row for a format that only that row reaches
1241 /// silently disables detection of a real format; this test turns that into
1242 /// a hard failure instead.
1243 ///
1244 /// The only cross-row transformation is the EBML one: the EBML prober is
1245 /// registered under `Format::Matroska` and `candidate_format` promotes a
1246 /// `DocType::Webm` to `Format::WebM`. Every other registered format maps to
1247 /// itself, so the reachable set is *exactly* `PROBERS`'s formats ∪ `WebM`.
1248 #[test]
1249 fn every_format_variant_is_reachable() {
1250 // How a `Format` value can be produced. The match below is EXHAUSTIVE
1251 // and has no wildcard arm, so adding a variant to `Format` is a
1252 // compile error here until its reachability is declared.
1253 //
1254 // That is the whole point. The previous version listed the variants in
1255 // a hand-written array and called itself "drift-shielded"; it was not.
1256 // An audit added a `Format::HevcAnnexB` variant with no prober and the
1257 // entire suite stayed green — 53 passed, 0 failed, across all 14 test
1258 // binaries. A format unreachable from any input shipped silently. A
1259 // list you maintain by hand cannot guard a list you maintain by hand.
1260 //
1261 // `#[non_exhaustive]` does not force a wildcard *inside* the defining
1262 // crate, which is exactly why this works here and would not work from
1263 // an integration test.
1264 enum Reach {
1265 /// Registered in `PROBERS` under its own name.
1266 Prober,
1267 /// Not in `PROBERS`; produced by `candidate_format` from another.
1268 Derived,
1269 }
1270 fn reachability(f: Format) -> Reach {
1271 match f {
1272 Format::MpegTs => Reach::Prober,
1273 Format::Isobmff => Reach::Prober,
1274 Format::MpegPs => Reach::Prober,
1275 Format::Matroska => Reach::Prober,
1276 Format::Flv => Reach::Prober,
1277 Format::Mxf => Reach::Prober,
1278 Format::Wav => Reach::Prober,
1279 Format::Ogg => Reach::Prober,
1280 Format::Asf => Reach::Prober,
1281 Format::AdtsAac => Reach::Prober,
1282 Format::Mp3 => Reach::Prober,
1283 Format::AnnexB => Reach::Prober,
1284 // EBML registers as `Matroska` and resolves to `WebM` by
1285 // `DocType`, so it is never its own `PROBERS` row.
1286 Format::WebM => Reach::Derived,
1287 }
1288 }
1289
1290 /// Every `Format` value, for iteration. Hand-written — but it cannot
1291 /// silently drift: the exhaustive `reachability` match above is a
1292 /// compile error until a new variant is declared, and the length
1293 /// assertion below fails if this list and `PROBERS` disagree.
1294 const ALL_FORMATS: [Format; 13] = [
1295 Format::MpegTs,
1296 Format::Isobmff,
1297 Format::MpegPs,
1298 Format::Matroska,
1299 Format::WebM,
1300 Format::Flv,
1301 Format::Mxf,
1302 Format::Wav,
1303 Format::Ogg,
1304 Format::Asf,
1305 Format::AdtsAac,
1306 Format::Mp3,
1307 Format::AnnexB,
1308 ];
1309 /// Formats reachable only via `candidate_format`: `WebM`.
1310 const DERIVED: usize = 1;
1311
1312 let registered: Vec<Format> = PROBERS.iter().map(|(f, _)| *f).collect();
1313
1314 // Pins the arithmetic: a prober added without extending ALL_FORMATS
1315 // (or vice versa) fails here rather than going unchecked.
1316 assert_eq!(
1317 ALL_FORMATS.len(),
1318 registered.len() + DERIVED,
1319 "ALL_FORMATS ({}) must equal PROBERS ({}) plus the {DERIVED} derived \
1320 format(s); one of the three has drifted",
1321 ALL_FORMATS.len(),
1322 registered.len()
1323 );
1324
1325 // The single `Derived` case must actually be derivable, not merely
1326 // declared so.
1327 assert_eq!(
1328 candidate_format(
1329 Format::Matroska,
1330 Detail::Ebml {
1331 doc_type: DocType::Webm,
1332 },
1333 ),
1334 Format::WebM,
1335 "WebM is declared Derived but candidate_format does not produce it"
1336 );
1337
1338 for variant in ALL_FORMATS {
1339 match reachability(variant) {
1340 Reach::Prober => assert!(
1341 registered.contains(&variant),
1342 "Format::{} is declared as having its own prober but is not a \
1343 PROBERS row",
1344 variant.name()
1345 ),
1346 Reach::Derived => assert!(
1347 !registered.contains(&variant),
1348 "Format::{} is declared Derived but also has a PROBERS row — \
1349 one of the two is wrong",
1350 variant.name()
1351 ),
1352 }
1353 }
1354
1355 // Every registered prober's format must appear in ALL_FORMATS, so the
1356 // enumeration cannot silently shrink either.
1357 for f in ®istered {
1358 assert!(
1359 ALL_FORMATS.contains(f),
1360 "Format::{} has a PROBERS row but is missing from ALL_FORMATS",
1361 f.name()
1362 );
1363 }
1364 }
1365
1366 /// `Probe::Ambiguous` must carry only the candidates actually within
1367 /// `TIE_THRESHOLD` of the winner, not every lower-scored also-ran (the
1368 /// variant documents "two or more candidates within `TIE_THRESHOLD`").
1369 #[test]
1370 fn ambiguous_lists_only_genuinely_tied_candidates() {
1371 let mk = |format: Format, score: u8| Candidate {
1372 format,
1373 confidence: Confidence(score),
1374 detail: Detail::None,
1375 };
1376 // Scores 240, 232, 192: 232 is within 16 of 240 (tied), 192 is not.
1377 let candidates = vec![
1378 mk(Format::MpegTs, 240),
1379 mk(Format::Isobmff, 232),
1380 mk(Format::MpegPs, 192),
1381 ];
1382 let p = probe_to_identify(&candidates);
1383 match p {
1384 Probe::Ambiguous { candidates: tied } => {
1385 assert_eq!(tied.len(), 2, "only the top two are within TIE_THRESHOLD");
1386 assert_eq!(tied[0].format, Format::MpegTs);
1387 assert_eq!(tied[1].format, Format::Isobmff);
1388 }
1389 other => panic!("expected Ambiguous, got {other:?}"),
1390 }
1391 }
1392
1393 /// A top candidate that beats its runner-up by more than `TIE_THRESHOLD` is
1394 /// `Identified`, not `Ambiguous`.
1395 #[test]
1396 fn decisive_winner_is_not_ambiguous() {
1397 let mk = |format: Format, score: u8| Candidate {
1398 format,
1399 confidence: Confidence(score),
1400 detail: Detail::None,
1401 };
1402 let candidates = vec![
1403 mk(Format::MpegTs, 240),
1404 mk(Format::Isobmff, 200),
1405 mk(Format::MpegPs, 64),
1406 ];
1407 match probe_to_identify(&candidates) {
1408 Probe::Identified { format, .. } => assert_eq!(format, Format::MpegTs),
1409 other => panic!("expected Identified, got {other:?}"),
1410 }
1411 }
1412}