Crate mediaframe

Expand description

mediaframe

A common media-stream descriptor vocabulary for media processing pipelines — codec / pixel-format / colour / frame metadata. Video + audio + subtitle codec vocabularies ship today; the frame / pixel-format / colour types currently cover video, with audio + subtitle descriptor types added incrementally.

LoC

§Overview

A common media-stream descriptor vocabulary for media processing pipelines. The codec module covers video + audio + subtitle codec identifiers; the pixel-format / colour / frame / source modules cover the video pipeline today, with audio + subtitle descriptor types added incrementally. Pure data types: no SIMD, no decoder, no codec implementation, no math — just the shared spine that a color-conversion library, a decoder backend, and a frame consumer can all speak to without agreeing on anything heavier.

§What it provides

codec — VideoCodec, AudioCodec, SubtitleCodec stream- descriptor enums covering every codec FFmpeg n8.1 knows under each media type, plus an Other(SmolStr) lossless escape for codecs added upstream before the table is regenerated. Generated by cargo xtask gen-codec from the vendored FFmpeg table — see xtask. Requires alloc (gated behind any(feature = "std", feature = "alloc") for the Other(SmolStr) arm).
color — ITU-T H.273 colour-metadata enums (color::Matrix, color::Primaries, color::Transfer, color::DynamicRange, color::ChromaLocation) bundled into color::Info, with FFmpeg-exact code points and a lossless Unknown(u32) arm on each. Plus DcpTargetGamut (DCI-XYZ target-gamut selection), Rotation, HDR static side- data (ContentLightLevel, ChromaCoord, MasteringDisplay, HdrStaticMetadata per SMPTE ST 2086 / FFmpeg HDR10), and DolbyVisionConfig (FFmpeg AVDOVIDecoderConfigurationRecord). Colour-enum numbering is CI-checked against the pinned FFmpeg header by cargo xtask check.
cfa — Bayer color-filter-array description (BayerPattern).
pixel_format — single PixelFormat enum covering every pixel format in FFmpeg n8.1’s AVPixelFormat (254 variants excluding GPU-resident HW formats) plus cinema-RAW additions. Coverage is verified by cargo xtask check against vendored pixfmt.h slugs — see xtask.
frame — structural primitives (Dimensions, Rect, Plane<B>), exact-ratio building blocks (Rational, FrameRate, SampleAspectRatio as a Rational newtype), stream- descriptor metadata (FieldOrder, StereoMode — both with lossless Unknown(u32)), the runtime-tagged VideoFrame<P, B>, and the orthogonal TimestampedFrame<F> wrapper bundling mediatime::Timestamp PTS + duration around any inner frame shape. Plus per-format typed *Frame<'a, BE> zero-copy borrow views + *FrameError validation (feature-gated).
source — per-format marker ZSTs (Yuv420p, Nv12, Rgb24, …), *Row<'a> borrows, *Sink subtraits, and *_to walker fns that iterate Frame → Row → PixelSink. The walker macro generates the marker / Row / Sink / walker quartet uniformly. Marker construction is Foo::new() (private () field locks shape evolution to additive changes).
buffa — optional buffa wire serialization (hand-written Message / DefaultInstance impls, no codegen) for the colour / frame / HDR vocabulary so downstream proto schemas can extern-map .mediaframe.v1 → ::mediaframe. Off by default — enable with --features buffa.
serde — optional serde::{Serialize, Deserialize} for the whole descriptor vocabulary. Wire shape by type:
- Open codec / format enums (codec::*, container::Format, subtitle::Format, audio::{ChannelLayout, ContainerFormat}) — the as_str() slug, unknown slugs ride Other.
- FFmpeg-coded enums with an Unknown(u32) arm (colour, pixel-format, frame coded enums, TrackDisposition) — the to_u32() integer; unknown codes round-trip via Unknown (no slug form).
- Strictly-closed coded enums (subtitle::TrackOrigin, audio::BitRateMode) — the to_u32() integer, but unknown codes are rejected as serde errors rather than collapsing to the default.
- audio::SampleFormat (both Unknown(u32) and Other(SmolStr)) — bespoke: human-readable formats emit a string for named/Other and a number for Unknown; binary formats use a tagged {Code, Slug} wire.
- lang::Language — its BCP-47 string. Validated structs (GeoLocation / Fingerprint / CoverArt) deserialize through their checking constructors.
Orthogonal to the capability tiers (no-alloc Copy types included). Off by default — enable with --features serde.
arbitrary — optional arbitrary::Arbitrary for the descriptor vocabulary (fuzzing / property tests), hand-written through each type’s public constructors so try_new-validated types come out valid and open Other(_) arms are exercised. Generated values are wire-canonical (every value survives a serde round-trip). Std-only — the arbitrary crate itself is std-based — so this serves host-side fuzzing, not an embedded target. Off by default — enable with --features arbitrary.
quickcheck — optional native quickcheck::Arbitrary for the same surface, via the quickcheck-richderive derive (each type carries #[quickcheck(arbitrary = "…")] pointing at a fn(&mut Gen) -> Self helper). Independent of arbitrary. Std-only. Off by default — enable with --features quickcheck.
PixelSink + SourceFormat sealed traits re-exported at the crate root.

§Installation

[dependencies]
# Lean — codec + color + pixel_format + frame primitives.
# Adds `mediatime` + `derive_more` + `smol_str` (codec `Other` arm).
mediaframe = "0.1"

Opt into typed *Frame<'a> borrow views + the per-format source::* walker quartet per family:

mediaframe = { version = "0.1", features = ["yuv-planar", "rgb"] }

Or take everything via the umbrella:

mediaframe = { version = "0.1", features = ["frame"] }

§Per-family feature flags

Enable only the families your pipeline actually consumes — each flag pulls in just the matching *Frame validators, *Row borrow types, marker ZSTs, walker fn, and Sink subtraits. The frame umbrella enables all of them at once.

Feature	Formats
`yuv-planar`	`Yuv420p` / `422p` / `444p` / `440p` / `411p` / `410p` + 9-16 bit
`yuv-semi-planar`	`Nv12` / `16` / `21` / `24` / `42`, `P010` / `210` / `410` family
`yuva`	YUVA planar 8-bit + 9-16 bit
`yuv-packed`	`Yuyv422`, `Uyvy422`, `Yvyu422`, `Uyyvyy411`
`yuv-444-packed`	`V410`, `Xv30`, `Xv36`, `Ayuv64`, `Vuya`, `Vuyx`, `V30X`
`y2xx`	`Y210` / `Y212` / `Y216`
`v210`	`V210`
`rgb`	`Rgb24` / `Bgr24` / `Rgba` / `Bgra` + 10-bit + 16-bit
`rgb-float`	`Rgbf32` / `Rgbf16` + `Rgbaf16`/`f32`
`rgb-legacy`	`Rgb444` / `555` / `565` + Bgr counterparts
`gbr`	`Gbrp` / `Gbrap` + 9-16 bit + float
`gray`	`Gray8` / 9-16 bit / f32, `Ya8` / `Ya16`
`bayer`	Bayer 8 / 10 / 12 / 14 / 16-bit × 4 patterns
`xyz`	`Xyz12` (DCI-XYZ)
`mono`	`Monoblack` / `Monowhite` / `Pal8`
`frame`	umbrella — enables every sub-feature above

Deps pulled in by family features:

thiserror — every per-family feature (for *FrameError).
half — rgb-float, gbr, gray (for half::f16).
derive_more try_unwrap / unwrap — yuv-444-packed, y2xx.

§`no_std`

# Pure no_std — just enums, marker ZSTs, structural primitives.
mediaframe = { version = "0.1", default-features = false }

# no_std + alloc — adds Vec-using helpers and tests.
mediaframe = { version = "0.1", default-features = false, features = ["alloc"] }

The color, cfa, and pixel_format modules work without alloc (pure enums / Copy types). The codec module is gated on any(feature = "std", feature = "alloc") because of its Other(SmolStr) arm. Per-family frame / source features work under no_std + alloc.

§xtask

cargo xtask sync fetches the pinned FFmpeg release tag (currently n8.1) and refreshes the vendored tables under xtask/vendor/:

ffmpeg-pixfmts.txt — every AV_PIX_FMT_<NAME> slug from libavutil/pixfmt.h.
ffmpeg-color.txt — every colour-enum code point from libavutil/pixfmt.h (matrix / primaries / transfer / range / chroma location).
ffmpeg-codecs.txt — every codec identifier under media types video / audio / subtitle from libavcodec/codec_desc.c.

cargo xtask gen-codec regenerates src/codec.rs from ffmpeg-codecs.txt — one VideoCodec / AudioCodec / SubtitleCodec enum variant per FFmpeg codec.

cargo xtask check diffs the vendored tables against the in-tree enums and fails on any missing variant or numbering drift: PixelFormat::as_str() vs ffmpeg-pixfmts.txt, colour-enum code points vs ffmpeg-color.txt, and the codec enum variants vs ffmpeg-codecs.txt. CI runs this so every enum stays exhaustive against the pinned FFmpeg version.

Vendoring only the slug / code-point lists (not the LGPL FFmpeg headers verbatim) sidesteps the license question.

§License

mediaframe is under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE, LICENSE-MIT for details.

Re-exports§

pub use source::PixelSink;
pub use source::SourceFormat;

Modules§

audiostd or alloc: Audio-stream descriptor vocabulary — channel layout, sample / container format, bit-rate mode, EBU R128 loudness, fingerprint, embedded metadata tags + cover art. Requires the alloc feature (std includes it) for the Other(SmolStr) escape arms and the Vec<u8> payloads. Audio-stream descriptor vocabulary — channel layout, sample / container format, bit-rate mode, EBU R128 loudness, fingerprint, embedded metadata tags + cover art.
capturestd or alloc: EXIF / capture-metadata vocabulary — capture device, geographic location (with ISO-6709 parse/format). Requires the alloc feature (std includes it) because the constituent types lean on SmolStr / std::string::String for their text surface. EXIF / capture-metadata vocabulary — capture device, geographic location (with ISO-6709 parse/format), and (future) capture-time, lens, exposure (ISO/aperture/shutter).
codecstd or alloc: Stream-descriptor codec/format/layout vocabulary for video, audio, and subtitle tracks. Requires the alloc feature (std includes it) for the Other(SmolStr) escape arms. Stream-descriptor codec vocabulary for video, audio, and subtitle tracks.
color: Color metadata: enums for matrix, primaries, transfer, range, and chroma location — all closed-form per ITU-T H.273.
containerstd or alloc: Top-level multimedia container-format vocabulary. Requires the alloc feature (std includes it) for the Other(SmolStr) escape arm. Multimedia container-format vocabulary — top-level (video + audio) containers.
disposition: FFmpeg AV_DISPOSITION_* bitflags shared across all track types (video / audio / subtitle). [TrackDisposition] — FFmpeg AV_DISPOSITION_* bitflags shared across video, audio, and subtitle tracks.
frame: Frame primitives + the typed source-format *Frame<'a, BE> borrow types.
langstd or alloc: Validated BCP-47 language tag wrapping icu_locale_core subtags (Copy, heap-free representation; to_bcp47() -> String and Display need the allocator). BCP-47 language tag — language + optional script + optional region, validated by icu_locale_core.
pixel_format: Pixel format identifier — comprehensive coverage of FFmpeg’s AVPixelFormat enum plus Bayer mosaic and cinema-RAW formats.
quickcheck_helpersquickcheck: fn(&mut quickcheck::Gen) -> T helpers consumed by the per-type #[quickcheck(arbitrary = "…")] attributes on each descriptor’s quickcheck-richderive::Arbitrary derive. The derive emits the actual impl quickcheck::Arbitrary for T blocks; this module owns the bodies. Same surface as [arbitrary_impls] (39 descriptor-vocabulary types) but the two are independent — quickcheck does not bridge through arbitrary. fn(g: &mut quickcheck::Gen) -> T helpers — one per descriptor type — referenced via container-level #[quickcheck(arbitrary = "…")] on each type’s quickcheck_richderive::Arbitrary derive.
source: Source pixel-format kernels — marker ZSTs, per-row borrow types (*Row<'a>), per-format Sink subtraits, and walker fns that iterate crate::frame::*Frame<'a, BE> row-by-row dispatching to a PixelSink.
subtitlestd or alloc: Subtitle-stream descriptor vocabulary — file / demuxer format (subtitle::Format) and track-origin axis (subtitle::TrackOrigin). Requires the alloc feature (std includes it) for the subtitle::Format’s Other(SmolStr) escape arm. Subtitle-stream descriptor vocabulary — format + track origin.

Macros§

marker: Generates the canonical marker quartet (struct + new() + Sealed + SourceFormat) for a source-format marker type.