Struct VideoFrame 

pub struct VideoFrame {
    pub pts: Option<i64>,
    pub planes: Vec<VideoPlane>,
}

Uncompressed video frame.

Stream-level properties (pixel format, width, height, time base) are NOT carried per-frame — read them from the stream’s CodecParameters. Frames stay lightweight because real-time playback moves thousands per second per stream.

Side-channels

VideoFrame (like VideoPlane) is a fully-public struct built by struct literal throughout the codec crates, so per-frame metadata cannot be added as new fields without breaking every constructor. Instead, optional metadata rides in-band as side-channel entries at the tail of planes: VideoPlane values whose shape is impossible for an image plane, which makes them unambiguous. Every record has non-empty data and a stride that no image plane can have — either 0 (an image plane’s data is stride × rows long, so a zero stride forces empty data) or a value above isize::MAX (stride × rows with any non-zero row count would exceed what a Vec can hold). The whole stride > isize::MAX band is reserved for side-channel tags. Six record kinds exist, distinguished by their stride tag:

• Palette — stride == 0. Carries the color table for palette-indexed content (PixelFormat::Pal8); see palette / set_palette.
• Palette alpha — stride == usize::MAX - 3. Carries one alpha byte per palette entry for tables with transparent or translucent entries (a GIF transparent index, a PNG tRNS chunk on a colour-type-3 image, TGA / BMP alpha palettes); entries the record does not cover are opaque. Only meaningful next to a palette record; see palette_alpha / set_palette_alpha and the combined palette_rgba / set_palette_rgba.
• Per-plane significant bits — stride == usize::MAX. Carries mixed per-plane bit depths (e.g. 12-bit luma with 10-bit chroma from a wavelet codec’s custom signal range); see significant_bits / set_significant_bits.
• Colour signal — stride == usize::MAX - 1. Carries a ColorSignal (sample range + H.273 primaries / transfer / matrix) for producers whose signal is per-picture or that have no stream object to put it on; see color_signal / set_color_signal.
• Layer identity — stride == usize::MAX - 2. Carries a LayerIdentity (layer / view / access-unit) for frames of multi-layer or multi-view streams; see layer / set_layer.
• Metadata blobs — stride == usize::MAX - 4. Carries a list of MetadataBlobs (ICC profile, Exif, XMP, … — see crate::blob) for pictures whose metadata is per-picture: a multi-page TIFF’s per-page Exif, HEIF burst items with their own profile. Wire form encode_blobs / decode_blobs; see blobs / set_blobs. A blob that applies to the whole stream belongs on CodecParameters::blobs instead.

The records compose: a frame can carry any subset at once, in any order, within the trailing run of side-channel-shaped entries. The typed accessors find each record by its stride tag regardless of order, and image_planes / image_plane_count exclude the whole trailing run. Frames without any attached side-channel are byte-for-byte identical to what they always were.

Consumers that index planes directly (rather than through image_planes) see the records as extra trailing entries; producers attaching a record to every frame of a stream should therefore prefer the stream-level home (CodecParameters) for data that does not vary per picture, and attach per-frame records when the value is genuinely per-picture or no stream object exists.

Fields

pts: Option<i64>

Presentation timestamp in the stream’s time base; None if unknown.

planes: Vec<VideoPlane>

One entry per plane (e.g., 3 for Yuv420P). Each entry is (stride, bytes).

May additionally end with side-channel entries (palette, palette alpha, per-plane significant bits, colour signal, layer identity, metadata blobs — see the type-level docs). Code that wants only pixel planes should iterate image_planes instead of this field.

Implementations

impl VideoFrame

pub fn palette(&self) -> Option<&[u8]>

The frame’s attached palette, if any.

Returns the raw bytes of the palette side-channel (see the type-level docs): packed 3-byte RGB entries, entry i at bytes 3*i .. 3*i + 3 in R, G, B order. A full Pal8 table is 256 entries (768 bytes), but producers may attach fewer when the source image declares a shorter table; indices at or beyond len / 3 are undefined by this frame and up to the consumer’s missing-entry policy (typically black).

pub fn palette_rgb(&self, index: u8) -> Option<[u8; 3]>

The RGB triplet for palette entry index, or None when no palette is attached or the attached table is too short to cover index. Sugar over palette for per-pixel lookups.

pub fn set_palette(&mut self, rgb: Vec<u8>)

Attach (or replace) the frame’s palette side-channel.

rgb is packed 3-byte RGB entries — see palette for the exact layout; pass a length that is a multiple of 3 (up to 768 bytes for a full 256-entry Pal8 table). The bytes are stored verbatim. An empty rgb removes any attached palette instead (the sentinel requires non-empty data), leaving the frame exactly as it was before any palette was attached.

pub fn with_palette(self, rgb: Vec<u8>) -> Self

Builder-style counterpart to set_palette for construction chains: VideoFrame { pts, planes }.with_palette(rgb).

pub fn take_palette(&mut self) -> Option<Vec<u8>>

Detach and return the frame’s palette side-channel, if any. Afterwards the frame carries no palette (any other side-channel record is left in place — including a palette-alpha record, which then reads as None until a palette is attached again).

pub fn palette_alpha(&self) -> Option<&[u8]>

The frame’s attached palette-alpha record, if any.

Returns the raw bytes of the palette-alpha side-channel (see the type-level docs): byte i is the alpha of palette entry i (0 transparent, 255 opaque), in the same entry order as palette. The record may be shorter than the palette — a GIF with one transparent index needs only index + 1 bytes — and every entry it does not cover is opaque.

The record is only meaningful next to a palette: it reads as None when no palette is attached, and a malformed record longer than the palette’s entry count also reads as None (consumers fall back to opaque). Validation happens at read time, so palette and alpha may be attached in either order.

pub fn palette_rgba(&self, index: u8) -> Option<[u8; 4]>

The RGBA quadruplet for palette entry index: the RGB triplet from the palette record and the alpha from the palette-alpha record, or 255 (opaque) when no alpha record is attached, it does not cover index, or it is malformed. None when no palette is attached or the palette is too short to cover index — exactly when palette_rgb is None. The one lookup a Pal8 → RGBA expander needs.

pub fn set_palette_alpha(&mut self, alpha: Vec<u8>)

Attach (or replace) the frame’s palette-alpha side-channel.

alpha holds one byte per palette entry, in entry order — see palette_alpha for the semantics (may be shorter than the palette; uncovered entries are opaque). The bytes are stored verbatim; the length is checked against the palette when read, not here, so the two records may be attached in either order. An empty alpha removes any attached record instead (the sentinel requires non-empty data). Other side-channel records — the palette included — are unaffected.

pub fn with_palette_alpha(self, alpha: Vec<u8>) -> Self

Builder-style counterpart to set_palette_alpha for construction chains: VideoFrame { pts, planes }.with_palette(rgb).with_palette_alpha(alpha).

pub fn take_palette_alpha(&mut self) -> Option<Vec<u8>>

Detach and return the frame’s palette-alpha side-channel, if any — the bytes palette_alpha would have reported, so a record that read as None (no palette, or longer than the palette) is removed but returns None. Afterwards the frame carries no palette alpha (the palette and every other record are left in place).

pub fn set_palette_rgba(&mut self, rgba: &[[u8; 4]])

Attach (or replace) both palette records at once from RGBA entries: entry i of rgba becomes bytes 3*i .. 3*i + 3 of the palette record and byte i of the palette-alpha record. The alpha record is written even when every entry is opaque (stored verbatim, no trimming). An empty rgba removes both records.

let mut f = VideoFrame { pts: None, planes: vec![VideoPlane { stride: 1, data: vec![1] }] };
f.set_palette_rgba(&[[0, 0, 0, 0], [255, 255, 255, 255]]);
assert_eq!(f.palette_rgba(0), Some([0, 0, 0, 0]));
assert_eq!(f.palette_rgba(1), Some([255, 255, 255, 255]));
assert_eq!(f.palette_rgba(2), None);
assert_eq!(f.image_plane_count(), 1);

pub fn with_palette_rgba(self, rgba: &[[u8; 4]]) -> Self

Builder-style counterpart to set_palette_rgba.

pub fn significant_bits(&self) -> Option<&[u8]>

The frame’s attached per-plane significant-bits record, if any.

Returns the raw bytes of the significant-bits side-channel (see the type-level docs): byte k is the number of significant bits in the samples of image plane k, in plane order. This lets a producer express mixed per-plane depths that no single PixelFormat variant can name — e.g. a wavelet codec’s custom signal range with 12-bit luma and 10-bit chroma, stored on a Yuv444P12Le surface with an attached record of [12, 10, 10].

Semantics

• Values are LSB-anchored: a plane with b significant bits keeps its sample values in the low b bits of each storage word, with the upper bits zero — the same convention as this crate’s partial-depth formats (Gray10Le, Yuv420P10Le, Gbrp12Le, …, each documented as “uses the low N bits of a 16-bit word”). Full-scale for b significant bits is (1 << b) - 1.
• Each value must satisfy 1 ≤ b ≤ 8 × storage-word-bytes of the frame’s pixel format (so at most 8 for byte-sized planes, 16 for LE-16-bit-word planes). The record refines the storage format’s significant depth; it never changes the storage word size or plane geometry.
• A record shorter than the image-plane count (or a missing record) leaves the uncovered planes at the pixel format’s own documented depth. Bytes are stored verbatim; out-of-range values are a producer bug and consumers may clamp or reject them.

pub fn plane_significant_bits(&self, plane: usize) -> Option<u8>

The significant-bit count for image plane plane, or None when no record is attached or the attached record is too short to cover plane (fall back to the pixel format’s own depth). Sugar over significant_bits for per-plane lookups.

pub fn set_significant_bits(&mut self, bits: Vec<u8>)

Attach (or replace) the frame’s per-plane significant-bits side-channel.

bits holds one byte per image plane, in plane order — see significant_bits for the exact semantics (LSB-anchored values, 1 ≤ b ≤ storage word bits). The bytes are stored verbatim. An empty bits removes any attached record instead (the sentinel requires non-empty data), leaving the frame exactly as it was before any record was attached. Any attached palette is unaffected.

pub fn with_significant_bits(self, bits: Vec<u8>) -> Self

Builder-style counterpart to set_significant_bits for construction chains: VideoFrame { pts, planes }.with_significant_bits(bits).

pub fn take_significant_bits(&mut self) -> Option<Vec<u8>>

Detach and return the frame’s per-plane significant-bits side-channel, if any. Afterwards the frame carries no significant-bits record (any attached palette is left in place).

pub fn color_signal(&self) -> Option<ColorSignal>

The frame’s attached colour-signal description, if any.

Decoded from the colour-signal side-channel record (see the type-level docs; wire form in ColorSignal::to_bytes). A per-frame description refines the stream-level CodecParameters::color_signal: consumers resolve frame.color_signal().unwrap_or_default() .or(params.color_signal) and then apply their own policy to whatever is still unspecified. A malformed (too short) record reads as None.

pub fn set_color_signal(&mut self, signal: ColorSignal)

Attach (or replace) the frame’s colour-signal side-channel. Other side-channel records are unaffected.

pub fn with_color_signal(self, signal: ColorSignal) -> Self

Builder-style counterpart to set_color_signal for construction chains: VideoFrame { pts, planes }.with_color_signal(sig).

pub fn take_color_signal(&mut self) -> Option<ColorSignal>

Detach and return the frame’s colour-signal side-channel, if any. Afterwards the frame carries no colour signal (other records are left in place).

pub fn layer(&self) -> Option<LayerIdentity>

The frame’s attached layer / view identity, if any.

Decoded from the layer-identity side-channel record (see the type-level docs; wire form in LayerIdentity::to_bytes). Single-layer decoders attach nothing; consumers treat None as “base layer, no view”. A malformed (too short) record reads as None.

pub fn layer_or_base(&self) -> LayerIdentity

The frame’s layer identity, or the base layer when none is attached. Sugar over layer for consumers that handle single- and multi-layer streams alike.

pub fn set_layer(&mut self, layer: LayerIdentity)

Attach (or replace) the frame’s layer-identity side-channel. Other side-channel records are unaffected.

pub fn with_layer(self, layer: LayerIdentity) -> Self

Builder-style counterpart to set_layer for construction chains: VideoFrame { pts, planes }.with_layer(LayerIdentity::new(1)).

pub fn take_layer(&mut self) -> Option<LayerIdentity>

Detach and return the frame’s layer-identity side-channel, if any. Afterwards the frame carries no layer identity (other records are left in place).

pub fn blobs(&self) -> Vec<MetadataBlob>

The frame’s attached metadata blobs, decoded from the blobs side-channel record (see the type-level docs; wire form in encode_blobs). Empty when no record is attached — and when the record is malformed, so a consumer never sees half a list. Decodes on every call: read it once per frame.

A per-frame blob refines the stream-level CodecParameters::blobs of the same kind: resolve frame.blob(&k).or_else(|| params.blob(&k).cloned()).

pub fn blob(&self, kind: &BlobKind) -> Option<MetadataBlob>

The first attached blob of kind, if any (see blobs).

pub fn set_blobs(&mut self, blobs: Vec<MetadataBlob>)

Attach (or replace) the frame’s metadata-blobs side-channel with blobs, in the given order. An empty list removes any attached record instead (the sentinel requires non-empty data). Other side-channel records are unaffected.

pub fn push_blob(&mut self, blob: MetadataBlob)

Append one blob to the frame’s metadata-blobs record (creating it when absent; a malformed existing record is replaced by the single new blob). Blobs of the same kind are kept in insertion order; nothing is replaced.

pub fn with_blob(self, blob: MetadataBlob) -> Self

Builder-style counterpart to push_blob for construction chains: VideoFrame { pts, planes }.with_blob(MetadataBlob::new(BlobKind::EXIF, exif)).

pub fn with_blobs(self, blobs: Vec<MetadataBlob>) -> Self

Builder-style counterpart to set_blobs.

pub fn take_blobs(&mut self) -> Vec<MetadataBlob>

Detach and return the frame’s metadata blobs, if any (empty for no record or a malformed one — which is removed all the same). Afterwards the frame carries no blobs record (other records are left in place).

pub fn image_planes(&self) -> &[VideoPlane]

The frame’s image planes — planes with the trailing side-channel entries (palette, palette alpha, significant bits, colour signal, layer identity, metadata blobs) excluded. Prefer this over indexing planes directly in code that handles side-channel-capable frames.

pub fn image_plane_count(&self) -> usize

Number of image planes (excludes every side-channel entry). Matches the stream pixel format’s plane_count for well-formed frames.

Trait Implementations

impl Clone for VideoFrame

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for VideoFrame

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.