Crate mediadecode

Expand description

mediadecode

Generic, no_std-friendly type-and-trait spine for media decoders.

LoC

The backend-agnostic core of the mediadecode workspace. Defines the unified Packet / Frame types, VideoAdapter / AudioAdapter / SubtitleAdapter traits, the matching push-style *StreamDecoder traits that concrete decoder backends implement, and the two tiers on either side of them: the Demuxer session that produces packets and the AudioResampler seam that reshapes decoded audio.

This crate ships no decoder code and no FFmpeg dependency. It’s no_std-clean (with optional alloc / std features) and zero heavy deps — downstream crates (colconv, scenesdetect, …) program against this vocabulary regardless of which backend produced the bytes. Adapter implementations live in sibling crates such as mediadecode-ffmpeg.

§What’s in the box

Pixel and sample formats — PixelFormat (~270 variants covering every FFmpeg n9.0 AVPixelFormat slug plus cinema-RAW additions; sourced from videoframe and re-exported here so consumers keep their mediadecode::PixelFormat import). Unknown(u32) preserves the raw wire identifier for lossless round-trip via from_u32 / to_u32. H.273-aligned color enums (ColorMatrix, ColorPrimaries, ColorTransfer, ColorRange, ChromaLocation) and BayerPattern for RAW are similarly re-exported from videoframe.
Generic packet / frame types — VideoPacket<A, B>, AudioPacket<A, B>, SubtitlePacket<A, B>, VideoFrame<A, B>, AudioFrame<A, B>, SubtitleFrame<A, B> and ImageFrame<A, B> parameterized over an adapter’s per-item extras type A and buffer type B. Plane<B> is the generic plane carrier. ImageFrame is the still-image household: no pts, no duration, because a still is not on the timeline — the same fact AttachmentPacket states on the packet side.
The D-seat amputation contract — the one law a backend’s buffer type B must obey: owned, Send + Sync, cheap to clone (a refcount bump), with no backend-internal lifetime crossing the seam. This crate names no carrier and pins no bound past AsRef<[u8]>; the contract is written out in full on the adapter module.
Adapter traits — VideoAdapter, AudioAdapter, SubtitleAdapter, ImageAdapter. A backend implements these on a zero-sized type to fix A and B once for the whole pipeline.
Decoder traits — VideoStreamDecoder, AudioStreamDecoder, SubtitleDecoder, ImageDecoder. The two *Stream* faces are push-style (send_packet / receive_frame / send_eof / flush), mirroring FFmpeg’s decoder API while staying backend-agnostic; the other two are not, and their names say so — a subtitle cue and a still image each come out of exactly the packet that went in.
The demux tier — Demuxer, the pull session over an opened container (tracks() / next_packet() / seek()), the five-arm DemuxedPacket envelope, the TrackInfo / TrackParams / TrackKind table, and the DemuxAdapter vocabulary bundle. Opening is each backend’s own, so the trait covers only the opened session. A session keeps its track table for life: tracks() is a non-destructive read that hands out TrackHandles — the backend’s own row carrier, Arc / Rc / a plain borrow — so a consumer that needs a row past the pull loop clones a refcount rather than taking the table away from the session that classifies against it. A row carries the identity the container declares about a track, its language among it — as the file wrote it, unfolded, because the registries that reconcile an MKV’s ger with an MP4’s deu belong to whoever owns the language vocabulary.
The resample seam — AudioResampler, the AudioStreamDecoder push pair one tier along, converting rate, sample format and channel layout between a source spec read off TrackInfo and a target spec that is always the caller’s options.
Time primitives — re-exported Timebase / Timestamp / TimeRange from mediatime, so consumers don’t need a separate dep.

§API style

Mirrors the mediatime idioms the rest of the findit-studio workspace uses:

All public fields are private; access is via field() getters, consuming with_field(value) builders, and in-place set_field(value) mutators that return &mut Self.
const fn everywhere the field type allows.
Panicking constructors come with try_* fallible counterparts (empty / try_empty, clone / try_clone, …).
Errors via thiserror over the stable core::error::Error, so failures still implement the Error trait under --no-default-features.

§Cargo features

Feature	Default	Effect
`std`	yes	Enable the standard library and `mediatime/default`.
`alloc`	—	Enable owning collections (`Vec`, `String`) without `std`.
`serde`	—	Every type below on the wire, plus `mediatime`.
`arbitrary`	—	`Arbitrary` impls for fuzzing, same coverage.
`quickcheck`	—	The same coverage again, for `quickcheck`.

The three optional matrices cover the same type, and its wire shape follows what it is:

Type	Tier	serde wire shape
`packet::PacketFlags`	any	the raw bits, as a number

A bit set travels as a number, because every bit pattern is meaningful and bits this build has no constant for still have to survive the round trip.

The audio channel-layout vocabulary this crate used to own — ChannelLayoutKind, AudioChannelOrderKind, AudioChannelSpec, AudioChannelLayout — now lives in mediaframe::audio, whose wire shapes are documented there. AudioFrame’s channel-layout parameter is generic, so this crate names no channel type at all.

no_std builds: disable defaults and pick alloc if you need Vec-backed payloads:

[dependencies]
mediadecode = { version = "0.8", default-features = false, features = ["alloc"] }

§Usage

This crate defines the surface; concrete decoding happens in adapter crates. A backend-agnostic consumer programs against the traits, and both faces answer states: send_packet answers Sent — accepted or must drain — and receive_frame answers Received — a frame, needs input, or ended. Err means a fault and nothing else:

use mediadecode::{
  Received, Sent,
  adapter::VideoAdapter,
  decoder::VideoStreamDecoder,
  frame::VideoFrame,
  packet::VideoPacket,
};

type Frame<D> = VideoFrame<
  <<D as VideoStreamDecoder>::Adapter as VideoAdapter>::PixelFormat,
  <<D as VideoStreamDecoder>::Adapter as VideoAdapter>::FrameExtra,
  <D as VideoStreamDecoder>::Buffer,
>;

/// Feeds one packet and delivers every frame it made ready.
/// Answers `true` once the stream is over.
fn decode_one<D: VideoStreamDecoder>(
  decoder: &mut D,
  packet: &VideoPacket<
    <D::Adapter as VideoAdapter>::PacketExtra,
    D::Buffer,
  >,
  dst: &mut Frame<D>,
  mut on_frame: impl FnMut(&Frame<D>),
) -> Result<bool, D::Error> {
  // Offer until the decoder takes it. `MustDrain` promises nothing was
  // consumed, so the *same* packet is re-offered after a drain.
  loop {
    // `?` gives up on a real failure — and only on a real failure.
    match decoder.send_packet(packet)? {
      Sent::Accepted => break,
      Sent::MustDrain => {
        while let Received::Frame = decoder.receive_frame(dst)? {
          on_frame(dst);
        }
      }
    }
  }
  loop {
    match decoder.receive_frame(dst)? {
      Received::Frame => on_frame(dst),
      Received::NeedsInput => return Ok(false),
      Received::Ended => return Ok(true),
    }
  }
}

The compiler is what makes this correct: a consumer that forgets end-of-stream does not compile, a receive-side failure can no longer be mistaken for a drained decoder, and back pressure no longer has to be guessed at by offering every packet twice.

For an end-to-end example using the FFmpeg adapter, see mediadecode-ffmpeg.

§Build requirements

Rust ≥ 1.95, edition 2024.
No system dependencies — mediadecode is FFmpeg-free and builds anywhere Rust does, including no_std targets with optional alloc.

§License

mediadecode is under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE, LICENSE-MIT for details.

Re-exports§

pub use rhythm::Received;
pub use rhythm::Sent;

Modules§

adapter: Adapter traits — the per-kind backend “vocabulary.”
cfa: Color-filter-array (Bayer) descriptions: re-exported from mediaframe::frame (mediaframe 0.2 dropped the cfa module and moved BayerPattern under frame::bayer, re-exported via frame::*).
color: Color metadata: re-exported from mediaframe::color.
decoder: Decoder traits — push-style streams (FFmpeg / WebCodecs / ProRes RAW via VTDecompressionSession), pull-style frame sources (R3D / BRAW / ARRIRAW / X-OCN / Canon RAW Light), and the one-shot ImageDecoder.
demuxer: The demux tier — the track table, the five-arm packet envelope, and the Demuxer session face.
frame: Frame types and supporting building blocks.
futurefuture: Async variants of the decoder / frame-source traits — gated behind the future feature.
packet: Compressed Packet types and PacketFlags.
pixel_format: Pixel format identifier: re-exported from mediaframe::pixel_format.
resampler: The resample seam — sample-rate, sample-format and channel-layout conversion for decoded audio.
rhythm: The push rhythm’s vocabulary — what a submission answered (Sent) and what a drain answered (Received), shared by every decoder session and by the resampler.
subtitle: Decoded subtitle payload.

Structs§

TimeRange: A half-open time range [start, end) in a given Timebase.
Timebase: A media timebase represented as a rational number: a non-negative numerator over a strictly positive denominator.
Timestamp: A presentation timestamp, expressed as a PTS value in units of an associated Timebase.

Enums§

PixelFormat: Pixel format identifier covering FFmpeg + Bayer + cinema-RAW.