Crate mediaframe

Expand description

mediaframe

A common media-stream descriptor vocabulary for media processing pipelines — codec, pixel-format, colour, frame, audio, subtitle, container, track-disposition, EXIF capture, and BCP-47 language metadata.

LoC

§Overview

A common media-stream descriptor vocabulary for media processing pipelines. The codec module covers video + audio + subtitle codec identifiers. Frame-side modules — pixel-format / colour / frame / source — cover the video pipeline. Stream-side modules cover audio (channel layout, sample / container format, bit-rate mode, EBU R128 loudness, fingerprint, tags + cover art), subtitle (format + track origin), top-level multimedia container format, FFmpeg track dispositions, EXIF capture metadata (device + geolocation), and BCP-47 language. Pure data types: no SIMD, no decoder, no codec implementation, no math — just the shared spine that a color- conversion library, a decoder backend, and a frame consumer can all speak to without agreeing on anything heavier.

§What it provides

codec — VideoCodec, AudioCodec, SubtitleCodec, DataCodec stream-descriptor enums covering every codec FFmpeg n9.0 knows under each media type, plus an Other(Utf8Bytes) lossless escape for codecs added upstream before the table is regenerated. Generated by cargo xtask gen-codec from the vendored FFmpeg table — see xtask. Alongside them, AttachmentCodec covers the three codec ids FFmpeg’s Matroska/WebM demuxer actually assigns to an AVMEDIA_TYPE_ATTACHMENT stream (ttf / otf / bin_data, all three also DataCodec variants) — codec_desc.c itself has no AVMEDIA_TYPE_ATTACHMENT media type to vendor a table from, so this one roster is hand-curated from libavformat/matroskadec.c instead; see the doc comment on AttachmentCodec and on ATTACHMENT_CODECS in xtask/src/main.rs. Requires alloc (gated behind any(feature = "std", feature = "alloc") for the Other(Utf8Bytes) arm).
color — ITU-T H.273 colour-metadata enums (color::Matrix, color::Primaries, color::Transfer, color::DynamicRange, color::ChromaLocation) bundled into color::Info, with FFmpeg-exact code points and the crate-wide Other(Utf8Bytes) escape arm on each (alloc tier). Plus DcpTargetGamut (DCI-XYZ target-gamut selection), Rotation, HDR static side- data (ContentLightLevel, ChromaCoord, MasteringDisplay, HdrStaticMetadata per SMPTE ST 2086 / FFmpeg HDR10), and DolbyVisionConfig (FFmpeg AVDOVIDecoderConfigurationRecord). Colour-enum numbering is CI-checked against the pinned FFmpeg header by cargo xtask check.
pixel_format — single PixelFormat enum covering every pixel format in FFmpeg n9.0’s AVPixelFormat (254 variants excluding GPU-resident HW formats) plus cinema-RAW additions. Coverage is verified by cargo xtask check against vendored pixfmt.h slugs — see xtask. PixelFormat::canonical() resolves a deprecated / aliased format to its (canonical_format, Option<DynamicRange>) — e.g. Yuvj420p → (Yuv420p, Full), Xv30 → V410 (both endians), Gray8a → Ya8.
frame — structural primitives (Dimensions, Rect, Plane<B>), exact-ratio building blocks (Rational, FrameRate, SampleAspectRatio as a Rational newtype), stream- descriptor metadata (FieldOrder, StereoMode — both with the Other(Utf8Bytes) escape), BayerPattern for color-filter-array description, the runtime-tagged VideoFrame<P, B>, and the orthogonal TimestampedFrame<F> wrapper bundling mediatime::Timestamp PTS + duration around any inner frame shape. Plus per-format typed *Frame<'a, BE> zero-copy borrow views + *FrameError validation (feature-gated).
source — per-format marker ZSTs (Yuv420p, Nv12, Rgb24, …), *Row<'a> borrows, *Sink subtraits, and *_to walker fns that iterate Frame → Row → PixelSink. The walker macro generates the marker / Row / Sink / walker quartet uniformly. Marker construction is Foo::new() (private () field locks shape evolution to additive changes).
container — top-level multimedia container::Format enum (Mov, Mp4, Mkv, Webm, Avi, MpegTs, …) with an Other(Utf8Bytes) lossless escape. Distinct from audio-only containers, which live on audio::ContainerFormat. Requires the alloc feature (std includes it).
audio — audio-stream descriptor vocabulary: ChannelLayout (FFmpeg AV_CHANNEL_LAYOUT_* bitmask), ContainerFormat (audio-only containers: mp3 / flac / wav / …), SampleFormat, BitRateMode (CBR / VBR / ABR), Loudness (EBU R128 integrated / true-peak / LRA), Fingerprint, embedded-metadata Tags, and CoverArt payload. Requires the alloc feature.
subtitle — subtitle::Format (Srt / WebVtt / Ass / image-based HdmvPgs / …) + TrackOrigin (Embedded / Sidecar / External). Pure stream-descriptor vocabulary — no per-cue content. Requires the alloc feature.
lang — BCP 47 language identity: three validated subtag types (Language, ScriptSubtag, Region) and the lossless whole tag that composes them (LanguageId — language, script, region, and everything past the region held verbatim). Every fold is a column of a registry vendored under xtask/vendor/ and generated into lang::registry, so an mkv’s ger and an mp4’s deu are both de, iw is he, BU is MM, and en-Latn composes as en while zh-Hans composes as itself. A structurally valid subtag the registry has never heard of is admitted rather than refused. Requires the alloc feature.
disposition — TrackDisposition bitflags mirroring FFmpeg’s AV_DISPOSITION_* (DEFAULT, FORCED, HEARING_IMPAIRED, …). Bit values are append-only, never renumbered; unknown bits round-trip losslessly through to_u32 / from_u32 via bitflags::from_bits_retain semantics. No alloc requirement — pure Copy bitflags.
capture — EXIF / capture-metadata vocabulary: Device (capture device name / model / serial) and GeoLocation (latitude / longitude / altitude with ISO-6709 parse + format). Requires the alloc feature.
buffa — optional buffa wire serialization (hand-written Message / DefaultInstance impls, no codegen) for the colour / frame / HDR vocabulary so downstream proto schemas can extern-map .mediaframe.v1 → ::mediaframe. Off by default — enable with --features buffa.
serde — optional serde::{Serialize, Deserialize} for the whole descriptor vocabulary. Wire shape by type:
- Open codec / format enums (codec::*, container::Format, subtitle::Format, audio::{ChannelLayout, ContainerFormat}) — the as_str() slug, unknown slugs ride Other.
- The FFmpeg-coded name vocabularies (colour, pixel-format, frame coded enums, audio::SampleFormat) — the same as_str() slug. They carry the same Other(Utf8Bytes) escape at the alloc tier, so an unenumerated name round-trips; at the no-alloc tier they are closed and an unrecognised slug is a serde error.
- disposition::TrackDisposition — the to_u32() integer. It is a bit set, not a name vocabulary, so a number is its faithful spelling.
- Strictly-closed coded enums (subtitle::TrackOrigin, audio::BitRateMode) — the to_u32() integer, but unknown codes are rejected as serde errors rather than collapsing to the default.
- The lang household (Language, ScriptSubtag, Region, LanguageId) — its canonical text, read back through the type’s own door, so a document holding GER deserializes to the same value one holding de does. Validated structs (GeoLocation / Fingerprint / CoverArt) deserialize through their checking constructors.
Orthogonal to the capability tiers (no-alloc Copy types included). Off by default — enable with --features serde.
arbitrary — optional arbitrary::Arbitrary for the descriptor vocabulary (fuzzing / property tests), hand-written through each type’s public constructors so try_new-validated types come out valid and open Other(_) arms are exercised. Generated values are wire-canonical (every value survives a serde round-trip). Std-only — the arbitrary crate itself is std-based — so this serves host-side fuzzing, not an embedded target. Off by default — enable with --features arbitrary.
quickcheck — optional native quickcheck::Arbitrary for the same surface, via the quickcheck-richderive derive (each type carries #[quickcheck(arbitrary = "…")] pointing at a fn(&mut Gen) -> Self helper). Independent of arbitrary. Std-only. Off by default — enable with --features quickcheck.
PixelSink + SourceFormat sealed traits re-exported at the crate root.

§Installation

[dependencies]
# Lean — codec + color + pixel_format + frame primitives.
# Adds `mediatime` + `derive_more` + `smol-bytes` (every text seat:
# the `Other` escape arms and the `lang` household's subtags).
mediaframe = "0.11"

Opt into typed *Frame<'a> borrow views + the per-format source::* walker quartet per family:

mediaframe = { version = "0.11", features = ["yuv-planar", "rgb"] }

Or take everything via the umbrella:

mediaframe = { version = "0.11", features = ["frame"] }

§Per-family feature flags

Enable only the families your pipeline actually consumes — each flag pulls in just the matching *Frame validators, *Row borrow types, marker ZSTs, walker fn, and Sink subtraits. The frame umbrella enables all of them at once.

Feature	Formats
`yuv-planar`	`Yuv420p` / `422p` / `444p` / `440p` / `411p` / `410p` + 9-16 bit (+ `444p` MSB)
`yuv-semi-planar`	`Nv12` / `16` / `21` / `24` / `42` / `Nv20`, `P010` / `210` / `410` family
`yuva`	YUVA planar 8-bit + 9-16 bit
`yuv-packed`	`Yuyv422`, `Uyvy422`, `Yvyu422`, `Uyyvyy411`
`yuv-444-packed`	`V410`, `Xv30`, `Xv36`, `Xv48`, `Ayuv64`, `Ayuv`, `Uyva`, `Vyu444`, `Vuya`, `Vuyx`, `V30X`
`y2xx`	`Y210` / `Y212` / `Y216`
`v210`	`V210`
`rgb`	`Rgb24` / `Bgr24` / `Rgba` / `Bgra` + 10-bit + 16-bit + 32-bit (`Rgb96` / `Rgba128`)
`rgb-float`	`Rgbf32` / `Rgbf16` + `Rgbaf16`/`f32`
`rgb-legacy`	`Rgb444` / `555` / `565` / `Rgb4` / `Rgb4Byte` / `Rgb8` + Bgr counterparts
`gbr`	`Gbrp` / `Gbrap` + 9-16 bit (+ MSB) + 32-bit + float
`gray`	`Gray8` / 9-16 bit / f16 / f32 / 32-bit, `Ya8` / `Ya16` / `Yaf16` / `Yaf32`
`bayer`	Bayer 8 / 10 / 12 / 14 / 16-bit × 4 patterns (LE + BE)
`xyz`	`Xyz12` (DCI-XYZ)
`mono`	`Monoblack` / `Monowhite` / `Pal8`
`frame`	umbrella — enables every sub-feature above

Deps pulled in by family features:

thiserror — every per-family feature (for *FrameError).
half — rgb-float, gbr, gray (for half::f16).
derive_more try_unwrap / unwrap — yuv-444-packed, y2xx.

§`no_std`

# Pure no_std — just enums, marker ZSTs, structural primitives.
mediaframe = { version = "0.11", default-features = false }

# no_std + alloc — adds Vec-using helpers and tests.
mediaframe = { version = "0.11", default-features = false, features = ["alloc"] }

The color, frame, and pixel_format modules work without alloc — but at that tier their vocabularies are closed: the Other(Utf8Bytes) escape needs a heap, so it is gated on any(feature = "std", feature = "alloc") and an unrecognised slug is rejected rather than carried. An error beats a wrong value. The codec, container, audio, subtitle, capture and lang modules are gated whole on the same pair. Per-family frame / source features work under no_std at every tier.

§xtask

cargo xtask sync fetches the pinned FFmpeg release tag (currently n9.0) and refreshes the vendored tables under xtask/vendor/:

ffmpeg-pixfmts.txt — every AV_PIX_FMT_<NAME> slug from libavutil/pixfmt.h.
ffmpeg-color.txt — every colour-enum code point from libavutil/pixfmt.h (matrix / primaries / transfer / range / chroma location).
ffmpeg-codecs.txt — every codec identifier under media types video / audio / subtitle / data from libavcodec/codec_desc.c.

cargo xtask gen-codec regenerates src/codec/mod.rs and its sibling src/codec/tests.rs from ffmpeg-codecs.txt plus the hand-curated ATTACHMENT_CODECS constant — one VideoCodec / AudioCodec / SubtitleCodec / DataCodec enum variant per FFmpeg codec, one AttachmentCodec variant per ATTACHMENT_CODECS entry, plus the suite that round-trips every one of them.

cargo xtask check diffs the vendored tables against the in-tree enums and fails on any missing variant or numbering drift: PixelFormat::as_str() vs ffmpeg-pixfmts.txt, colour-enum code points vs ffmpeg-color.txt, the four vendored codec enums’ variants vs ffmpeg-codecs.txt, and AttachmentCodec’s variants vs ATTACHMENT_CODECS. CI runs this so every enum stays exhaustive against the pinned FFmpeg version (or, for AttachmentCodec, against its own hand-curated source).

Vendoring only the slug / code-point lists (not the LGPL FFmpeg headers verbatim) sidesteps the license question.

§License

mediaframe is under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE, LICENSE-MIT for details.

§Feature tiers

The crate builds at three tiers, and the tier decides how open its vocabularies are.

Tier	Features	Vocabularies
no-alloc	(none)	closed — an unrecognised slug is rejected
alloc	`alloc`	open — an unrecognised slug rides `Other(Utf8Bytes)`
std	`std` (implies `alloc`)	as `alloc`, plus `std::error::Error`

Other(Utf8Bytes) needs a heap, so it exists only at the alloc / std tier. At the no-alloc tier the same enums are closed and their FromStr returns the vocabulary’s own error instead: an error beats a wrong value, and collapsing an unknown name onto a named variant would be a wrong value. The wire shape is the same at every tier (a slug either way) — only the openness differs.

Since 0.5.0 the error type follows the tier too. Where the escape arm exists the parse cannot fail, so FromStr::Err is Infallible and a caller can discharge it with an irrefutable let Ok(x) = s.parse::<T>();. The vocabulary’s own Parse*Error stays exported and is still what the no-alloc tier returns. This applies to the vocabularies compiled at every tier — the colour enums, PixelFormat, and the frame orientation enums. Vocabularies that only exist at the alloc tier have had Err = Infallible all along.

Every gate on an alloc-tier item is spelled any(feature = "std", feature = "alloc") rather than bare feature = "alloc", so the item cannot evaporate for a dependant that turns on std alone.

Re-exports§

pub use source::PixelSink;
pub use source::SourceFormat;

Modules§

audioalloc or std: Audio-stream descriptor vocabulary — channel layout (the name) and channel layout description (the structure: order, mask, per-channel list), sample / container format, bit-rate mode, EBU R128 loudness, fingerprint, embedded metadata tags + cover art. Requires the alloc feature (std includes it) for the Other(Utf8Bytes) escape arms and the Vec<u8> payloads.
capturealloc or std: EXIF / capture-metadata vocabulary — capture device, geographic location (with ISO-6709 parse/format). Requires the alloc feature (std includes it) because the constituent types lean on Utf8Bytes / std::string::String for their text surface. EXIF / capture-metadata vocabulary — capture device, geographic location (with ISO-6709 parse/format), and (future) capture-time, lens, exposure (ISO/aperture/shutter).
codecalloc or std: Stream-descriptor codec/format/layout vocabulary for video, audio, and subtitle tracks. Requires the alloc feature (std includes it) for the Other(Utf8Bytes) escape arms. Stream-descriptor codec vocabulary for video, audio, subtitle, data, and attachment tracks.
color: Color metadata: enums for matrix, primaries, transfer, range, and chroma location — all closed-form per ITU-T H.273.
containeralloc or std: Top-level multimedia container-format vocabulary. Requires the alloc feature (std includes it) for the Other(Utf8Bytes) escape arm. Multimedia container-format vocabulary — top-level (video + audio) containers.
disposition: FFmpeg AV_DISPOSITION_* bitflags shared across all track types (video / audio / subtitle). [TrackDisposition] — FFmpeg AV_DISPOSITION_* bitflags shared across video, audio, and subtitle tracks.
frame: Frame primitives + the typed source-format *Frame<'a, BE> borrow types.
imagealloc or std: Still-image vocabulary — standard photo formats (jpeg, png, heif, …) plus a curated camera-RAW family (dng, cr2, nef, arw, …). Requires the alloc feature (std includes it) for the Other(Utf8Bytes) escape arm — same tier as container and audio, which this household’s own module doc explains it exists to sit beside. Still-image vocabulary — standard photo formats plus the camera-RAW family.
langalloc or std: BCP 47 language identity — the three validated subtag types (lang::Language, lang::ScriptSubtag, lang::Region) and the lossless whole tag that composes them (lang::LanguageId), read against the two registries vendored under xtask/vendor/ and generated into lang::registry.
pixel_format: Pixel format identifier — comprehensive coverage of FFmpeg’s AVPixelFormat enum plus Bayer mosaic and cinema-RAW formats.
quickcheck_helpersquickcheck: fn(&mut quickcheck::Gen) -> T helpers consumed by the per-type #[quickcheck(arbitrary = "…")] attributes on each descriptor’s quickcheck-richderive::Arbitrary derive. The derive emits the actual impl quickcheck::Arbitrary for T blocks; this module owns the bodies. Same surface as [arbitrary_impls] (44 descriptor-vocabulary types) but the two are independent — quickcheck does not bridge through arbitrary. fn(g: &mut quickcheck::Gen) -> T helpers — one per descriptor type — referenced via container-level #[quickcheck(arbitrary = "…")] on each type’s quickcheck_richderive::Arbitrary derive.
source: Source pixel-format kernels — marker ZSTs, per-row borrow types (*Row<'a>), per-format Sink subtraits, and walker fns that iterate crate::frame::*Frame<'a, BE> row-by-row dispatching to a PixelSink.
subtitlealloc or std: Subtitle-stream descriptor vocabulary — file / demuxer format (subtitle::Format) and track-origin axis (subtitle::TrackOrigin). Requires the alloc feature (std includes it) for both types’ Other(Utf8Bytes) escape arms. Subtitle-stream descriptor vocabulary — format + track origin.

Macros§

marker: Generates the canonical marker quartet (struct + new() + Sealed + SourceFormat) for a source-format marker type.