Expand description
Conversion helpers from FFmpeg AVFrame / AVPacket to the
mediadecode types parameterized by crate::Ffmpeg and
FfmpegBytes.
Every plane is copied once, here, out of FFmpeg’s
AVBufferRef and into Rust-owned memory — the
D-seat amputation contract. Through 0.8 the video path
exported a refcounted view into libavcodec’s own allocation
whenever the stride happened to be tight, and copied only when it
was padded; a consumer therefore inherited an FFmpeg lifetime it
could not see, on some frames and not others. 0.9 copies both
branches. What is unchanged is the shape each branch produces —
a tight plane keeps the decoder’s linesize as its stride, a
padded one is compacted to row_bytes — because that geometry is
what consumers read, and the amputation is about ownership, not
about relaying out the picture.
§Header fields: the validation-order census
Every number in this module comes out of an AVFrame a file chose
the contents of, and each one is answerable to two questions —
what judges it, and what reads it first. When the second
precedes the first, the judgement is being made against a value its
own consumer has already laundered, which is not a judgement. That
is not hypothetical: it is how a declared -1 channel count reached
a ceiling as a legitimate-looking 0, having been floored by the
very helper the ceiling was supposed to run before.
So the order is censused rather than assumed. Every raw header field these three paths read, with its validator and its first consumer:
| path | field | validator | first consumer | order |
|---|---|---|---|---|
| audio | nb_samples | < 0 → InvalidSampleCount | the byte product | validator first |
| audio | ch_layout.nb_channels | < 0, > 255, == 0 with samples → UnsupportedChannelCount | channel_layout_description_from_raw_ptr | was inverted — hoisted |
| audio | format | bytes_per_sample() → UnsupportedSampleFormat | is_planar(), for the plane count | validator first |
| audio | linesize[0] | < 0, and == 0 with samples → InvalidPlaneLayout | allocated_per_plane | validator first |
| audio | sample_rate | none — censused metadata | AudioFrame::new | no geometry rides it |
| audio | data[i] | null check, then the backing-buffer proof | the copy | validator first |
| picture | width / height | < 0 → InvalidDimensions | copy_out_planes’ pixel ceiling | was inverted — hoisted |
| picture | format | is_deliverable → unsupported-format | plane_geometry | validator first |
| picture | linesize[i] | <= 0 and < row_bytes[i] → InvalidPlaneLayout | its own pass, after the budget and before any copy | validator first |
| picture | crop_* | checked_add per pair, then sum < extent | the rect | validator first |
| picture | nb_side_data, entry size (still road) | the entry cap and FrameLimits::max_image_side_data_bytes | the plane copy, then the side-data copy | was inverted — hoisted ahead of copy_out_planes |
| picture | colour enums, pict_type | the raw i32 fold, which is total | the fold’s own output | the fold is the validator |
| packet | flags (AV_PKT_FLAG_TRUSTED) | crate::buffer::TrustedPayload, both legs | the payload copy | validator first |
§The open-C-enum sweep, including this crate’s own code
The same discipline, applied to entry points rather than fields: a value read out of FFmpeg memory as a closed Rust enum is undefined behaviour before any comparison on it can run, and FFmpeg extends these enums in ABI-compatible releases.
| caller | entry point | enum | closed by |
|---|---|---|---|
| image / video / audio / subtitle open | Decoder::{video,audio,subtitle}() | AVCodecID, AVMediaType | find_decoder (raw u32) + ensure_codec_type (raw i32) |
track build, attachment classify, resampler spec, Debug | Parameters::medium() | AVMediaType | boundary::media_kind_of, a total fold |
| the pixel-format census | av_pix_fmt_desc_get_id | AVPixelFormat | local c_int shim |
| the pixel-format census | av_image_get_buffer_size | AVPixelFormat | local c_int shim |
| the sample-format census | av_get_bytes_per_sample | AVSampleFormat | local c_int shim |
| HW format negotiation | get_format callback list | AVPixelFormat | walked as *const i32 |
§The dimension-vocabulary sweep
A frame has more than one extent, and a judge that reads the wrong
one is not a judge. AVFrame.width/.height are the display
dims; what gets allocated is the coded extent on the software
road and the frames-context pool on the hardware one. On a
cropped stream they diverge without limit — measured on this build,
an h264 clip carrying SPS cropping shows 32x32 display over a
1920x1088 coded surface, a 2040x gap.
Every site that reads a dimension, and which vocabulary it needs:
| site | reads | sizes what | verdict |
|---|---|---|---|
judge_buffer | AVFrame.width/height at get_buffer2 | the software allocation’s cost | correct: measured, libavcodec hands this hook the frame at coded extent (1920x1088, aligned 1920x1090, 2,092,831 bytes), and the footprint prices those aligned dims against max_frame_bytes. Logical extent is not this seat’s question — max_pixels is enforced by ff_set_dimensions against the raw dims, which is the semantics it has |
get_hw_format | AVCodecContext.coded_width/height | the hardware pool | correct, and new: the display dims max_pixels was checked against are blind to it |
judge_hw_transfer | the frames-context pool dims | the transfer’s CPU destination | was display — repriced |
estimate_transfer_bytes | the frames-context pool dims | the probe’s pending budget | correct already, and its doc named this trap first |
drain_into_pending (two sites) | AVFrame.width/height | nothing — log fields only | benign |
VideoDecoder::width/height | the decoder’s display dims | nothing; a public accessor | correct — display is what a caller is asking for |
copy_out_planes | the converted frame’s own extent | the plane copy | correct — a decoded CPU frame’s extent is its allocation |
The pattern worth keeping: the extent to judge is the one the allocator will use, and it is never assumed — it is read from whatever structure the allocation is sized from. Where that structure cannot be read, the judge fails closed, because an unprovable extent is not a small one.
And the capstone the whole series arrives at, which generalises both tables above:
A judge must dominate the allocator’s arithmetic, not the payload’s.
Every ceiling here answers “may this be allocated?”, so the number
it compares has to be what the allocator will take — not what the
bytes nominally weigh, not what a tight layout would cost, and not
what the header displays. The two differ by under one percent on
ordinary frames, which is precisely why every under-pricing defect
in this release hid behind a shape big enough for the slack not to
show: nv12 16x16 is 384 bytes of pixels and a 1,792-byte
allocation, a one-sample eight-channel planar frame is 16 bytes of
samples and 768 allocated, and yuv420p 1920x1080 is 3,110,400
against 3,133,696. See [crate::footprint], where the pricing lives
and where the estimates are verified against real allocations rather
than argued.
The last three rows of the enum table above are the class inside
this crate’s own new code, and the census rows are its sharpest instance: that code
exists precisely to price formats this build’s bindings may not
name, and the binding it called handed those ids back as a closed
AVPixelFormat. Every future format would have become an invalid
enum value on the way into the pricing meant to handle it — the
census would have been undefined behaviour on exactly its reason for
existing. Writing the discipline down was not enough; it had to be
re-applied to the code that enforces it.
The still road’s side-data judgement is the same lesson one level
up, about passes rather than fields: it was correct, and it ran
after copy_out_planes, so an over-budget still had already bought
up to max_frame_bytes of plane copies before its annotations were
totalled. It reads only header fields and allocates nothing, so it
now runs with the other free judgements. Everything a conversion
can refuse is refused before anything it can allocate is
allocated.
The picture road’s byte ceiling is now judged from the geometry alone — the format’s row width times its row count, which no number the frame chose can influence — so it runs before any stride is so much as read. Then every stride is judged, in its own pass, before a single plane is bought: a layout fault is a property of the frame, knowable before any of it is paid for, and discovering it three plane allocations in was how a refused frame still cost three allocations.
The colour row is the shape to copy: a fold that cannot fail and maps everything unknown onto a named “not stated” leaves nothing for an order to get wrong.
Structs§
- Buffer
Acquire Failed - Payload for
ConvertError::BufferAcquireFailed. - Carrier
Alloc Failed - Payload for
ConvertError::CarrierAllocFailed. - Frame
TooLarge - Payload for
ConvertError::FrameTooLarge. - Image
Side Data Entries - Payload for
ConvertError::ImageSideDataEntries. - Image
Side Data TooLarge - Payload for
ConvertError::ImageSideDataTooLarge. - Invalid
Dimensions - Payload for
ConvertError::InvalidDimensions. - Invalid
Plane Layout - Payload for
ConvertError::InvalidPlaneLayout. - Invalid
Sample Count - Payload for
ConvertError::InvalidSampleCount. - TooMany
Pixels - Payload for
ConvertError::TooManyPixels. - Unsupported
Channel Count - Payload for
ConvertError::UnsupportedChannelCount. - Unsupported
Pixel Format - Payload for
ConvertError::UnsupportedPixelFormat. - Unsupported
Sample Format - Payload for
ConvertError::UnsupportedSampleFormat.
Enums§
- Convert
Error - Errors from
av_frame_to_video_frame.
Functions§
- audio_
frame_ from - Safe wrapper around
av_frame_to_audio_frametaking a borrowedffmpeg::frame::Audio. - av_
frame_ ⚠to_ audio_ frame - [
av_frame_to_audio_frame_as] on the view lane. - av_
frame_ ⚠to_ image_ frame - [
av_frame_to_image_frame_as] on the view lane. - av_
frame_ ⚠to_ owned_ audio_ frame av_frame_to_audio_frameon the owned lane, which copies every byte it reads and therefore has no aliasing obligation.- av_
frame_ ⚠to_ owned_ image_ frame av_frame_to_image_frameon the owned lane, which copies every byte it reads and therefore has no aliasing obligation.- av_
frame_ ⚠to_ owned_ video_ frame av_frame_to_video_frameon the owned lane, which copies every byte it reads and therefore has no aliasing obligation.- av_
frame_ ⚠to_ video_ frame - [
av_frame_to_video_frame_as] on the view lane. - av_
subtitle_ ⚠to_ owned_ subtitle_ frame av_subtitle_to_subtitle_frameon the owned lane, which copies every byte it reads and therefore has no aliasing obligation.- av_
subtitle_ ⚠to_ subtitle_ frame - [
av_subtitle_to_subtitle_frame_as] on the view lane. - image_
frame_ from - Safe wrapper around
av_frame_to_image_frametaking a borrowedffmpeg::Frame. - is_
video_ deliverable - Whether the video road can deliver
pix_fmt. - subtitle_
frame_ from - Safe wrapper around
av_subtitle_to_subtitle_frametaking a borrowedffmpeg::Subtitle. - video_
frame_ from - Safe wrapper around
av_frame_to_video_frametaking a borrowedffmpeg::Frame. Recommended entry point for most callers — equivalent to passingframe.as_ptr()to the unsafe variant, but the FFmpeg side keeps the frame alive for the duration of the call so the safety contract is satisfied internally.