Skip to main content

Module convert

Module convert 

Source
Expand description

Conversion helpers from FFmpeg AVFrame / AVPacket to the mediadecode types parameterized by crate::Ffmpeg and FfmpegBytes.

Every plane is copied once, here, out of FFmpeg’s AVBufferRef and into Rust-owned memory — the D-seat amputation contract. Through 0.8 the video path exported a refcounted view into libavcodec’s own allocation whenever the stride happened to be tight, and copied only when it was padded; a consumer therefore inherited an FFmpeg lifetime it could not see, on some frames and not others. 0.9 copies both branches. What is unchanged is the shape each branch produces — a tight plane keeps the decoder’s linesize as its stride, a padded one is compacted to row_bytes — because that geometry is what consumers read, and the amputation is about ownership, not about relaying out the picture.

§Header fields: the validation-order census

Every number in this module comes out of an AVFrame a file chose the contents of, and each one is answerable to two questions — what judges it, and what reads it first. When the second precedes the first, the judgement is being made against a value its own consumer has already laundered, which is not a judgement. That is not hypothetical: it is how a declared -1 channel count reached a ceiling as a legitimate-looking 0, having been floored by the very helper the ceiling was supposed to run before.

So the order is censused rather than assumed. Every raw header field these three paths read, with its validator and its first consumer:

pathfieldvalidatorfirst consumerorder
audionb_samples< 0InvalidSampleCountthe byte productvalidator first
audioch_layout.nb_channels< 0, > 255, == 0 with samples → UnsupportedChannelCountchannel_layout_description_from_raw_ptrwas inverted — hoisted
audioformatbytes_per_sample()UnsupportedSampleFormatis_planar(), for the plane countvalidator first
audiolinesize[0]< 0, and == 0 with samples → InvalidPlaneLayoutallocated_per_planevalidator first
audiosample_ratenone — censused metadataAudioFrame::newno geometry rides it
audiodata[i]null check, then the backing-buffer proofthe copyvalidator first
picturewidth / height< 0InvalidDimensionscopy_out_planes’ pixel ceilingwas inverted — hoisted
pictureformatis_deliverable → unsupported-formatplane_geometryvalidator first
picturelinesize[i]<= 0 and < row_bytes[i]InvalidPlaneLayoutits own pass, after the budget and before any copyvalidator first
picturecrop_*checked_add per pair, then sum < extentthe rectvalidator first
picturenb_side_data, entry size (still road)the entry cap and FrameLimits::max_image_side_data_bytesthe plane copy, then the side-data copywas inverted — hoisted ahead of copy_out_planes
picturecolour enums, pict_typethe raw i32 fold, which is totalthe fold’s own outputthe fold is the validator
packetflags (AV_PKT_FLAG_TRUSTED)crate::buffer::TrustedPayload, both legsthe payload copyvalidator first

§The open-C-enum sweep, including this crate’s own code

The same discipline, applied to entry points rather than fields: a value read out of FFmpeg memory as a closed Rust enum is undefined behaviour before any comparison on it can run, and FFmpeg extends these enums in ABI-compatible releases.

callerentry pointenumclosed by
image / video / audio / subtitle openDecoder::{video,audio,subtitle}()AVCodecID, AVMediaTypefind_decoder (raw u32) + ensure_codec_type (raw i32)
track build, attachment classify, resampler spec, DebugParameters::medium()AVMediaTypeboundary::media_kind_of, a total fold
the pixel-format censusav_pix_fmt_desc_get_idAVPixelFormatlocal c_int shim
the pixel-format censusav_image_get_buffer_sizeAVPixelFormatlocal c_int shim
the sample-format censusav_get_bytes_per_sampleAVSampleFormatlocal c_int shim
HW format negotiationget_format callback listAVPixelFormatwalked as *const i32

§The dimension-vocabulary sweep

A frame has more than one extent, and a judge that reads the wrong one is not a judge. AVFrame.width/.height are the display dims; what gets allocated is the coded extent on the software road and the frames-context pool on the hardware one. On a cropped stream they diverge without limit — measured on this build, an h264 clip carrying SPS cropping shows 32x32 display over a 1920x1088 coded surface, a 2040x gap.

Every site that reads a dimension, and which vocabulary it needs:

sitereadssizes whatverdict
judge_bufferAVFrame.width/height at get_buffer2the software allocation’s costcorrect: measured, libavcodec hands this hook the frame at coded extent (1920x1088, aligned 1920x1090, 2,092,831 bytes), and the footprint prices those aligned dims against max_frame_bytes. Logical extent is not this seat’s question — max_pixels is enforced by ff_set_dimensions against the raw dims, which is the semantics it has
get_hw_formatAVCodecContext.coded_width/heightthe hardware poolcorrect, and new: the display dims max_pixels was checked against are blind to it
judge_hw_transferthe frames-context pool dimsthe transfer’s CPU destinationwas display — repriced
estimate_transfer_bytesthe frames-context pool dimsthe probe’s pending budgetcorrect already, and its doc named this trap first
drain_into_pending (two sites)AVFrame.width/heightnothing — log fields onlybenign
VideoDecoder::width/heightthe decoder’s display dimsnothing; a public accessorcorrect — display is what a caller is asking for
copy_out_planesthe converted frame’s own extentthe plane copycorrect — a decoded CPU frame’s extent is its allocation

The pattern worth keeping: the extent to judge is the one the allocator will use, and it is never assumed — it is read from whatever structure the allocation is sized from. Where that structure cannot be read, the judge fails closed, because an unprovable extent is not a small one.

And the capstone the whole series arrives at, which generalises both tables above:

A judge must dominate the allocator’s arithmetic, not the payload’s.

Every ceiling here answers “may this be allocated?”, so the number it compares has to be what the allocator will take — not what the bytes nominally weigh, not what a tight layout would cost, and not what the header displays. The two differ by under one percent on ordinary frames, which is precisely why every under-pricing defect in this release hid behind a shape big enough for the slack not to show: nv12 16x16 is 384 bytes of pixels and a 1,792-byte allocation, a one-sample eight-channel planar frame is 16 bytes of samples and 768 allocated, and yuv420p 1920x1080 is 3,110,400 against 3,133,696. See [crate::footprint], where the pricing lives and where the estimates are verified against real allocations rather than argued.

The last three rows of the enum table above are the class inside this crate’s own new code, and the census rows are its sharpest instance: that code exists precisely to price formats this build’s bindings may not name, and the binding it called handed those ids back as a closed AVPixelFormat. Every future format would have become an invalid enum value on the way into the pricing meant to handle it — the census would have been undefined behaviour on exactly its reason for existing. Writing the discipline down was not enough; it had to be re-applied to the code that enforces it.

The still road’s side-data judgement is the same lesson one level up, about passes rather than fields: it was correct, and it ran after copy_out_planes, so an over-budget still had already bought up to max_frame_bytes of plane copies before its annotations were totalled. It reads only header fields and allocates nothing, so it now runs with the other free judgements. Everything a conversion can refuse is refused before anything it can allocate is allocated.

The picture road’s byte ceiling is now judged from the geometry alone — the format’s row width times its row count, which no number the frame chose can influence — so it runs before any stride is so much as read. Then every stride is judged, in its own pass, before a single plane is bought: a layout fault is a property of the frame, knowable before any of it is paid for, and discovering it three plane allocations in was how a refused frame still cost three allocations.

The colour row is the shape to copy: a fold that cannot fail and maps everything unknown onto a named “not stated” leaves nothing for an order to get wrong.

Structs§

BufferAcquireFailed
Payload for ConvertError::BufferAcquireFailed.
CarrierAllocFailed
Payload for ConvertError::CarrierAllocFailed.
FrameTooLarge
Payload for ConvertError::FrameTooLarge.
ImageSideDataEntries
Payload for ConvertError::ImageSideDataEntries.
ImageSideDataTooLarge
Payload for ConvertError::ImageSideDataTooLarge.
InvalidDimensions
Payload for ConvertError::InvalidDimensions.
InvalidPlaneLayout
Payload for ConvertError::InvalidPlaneLayout.
InvalidSampleCount
Payload for ConvertError::InvalidSampleCount.
TooManyPixels
Payload for ConvertError::TooManyPixels.
UnsupportedChannelCount
Payload for ConvertError::UnsupportedChannelCount.
UnsupportedPixelFormat
Payload for ConvertError::UnsupportedPixelFormat.
UnsupportedSampleFormat
Payload for ConvertError::UnsupportedSampleFormat.

Enums§

ConvertError
Errors from av_frame_to_video_frame.

Functions§

audio_frame_from
Safe wrapper around av_frame_to_audio_frame taking a borrowed ffmpeg::frame::Audio.
av_frame_to_audio_frame
[av_frame_to_audio_frame_as] on the view lane.
av_frame_to_image_frame
[av_frame_to_image_frame_as] on the view lane.
av_frame_to_owned_audio_frame
av_frame_to_audio_frame on the owned lane, which copies every byte it reads and therefore has no aliasing obligation.
av_frame_to_owned_image_frame
av_frame_to_image_frame on the owned lane, which copies every byte it reads and therefore has no aliasing obligation.
av_frame_to_owned_video_frame
av_frame_to_video_frame on the owned lane, which copies every byte it reads and therefore has no aliasing obligation.
av_frame_to_video_frame
[av_frame_to_video_frame_as] on the view lane.
av_subtitle_to_owned_subtitle_frame
av_subtitle_to_subtitle_frame on the owned lane, which copies every byte it reads and therefore has no aliasing obligation.
av_subtitle_to_subtitle_frame
[av_subtitle_to_subtitle_frame_as] on the view lane.
image_frame_from
Safe wrapper around av_frame_to_image_frame taking a borrowed ffmpeg::Frame.
is_video_deliverable
Whether the video road can deliver pix_fmt.
subtitle_frame_from
Safe wrapper around av_subtitle_to_subtitle_frame taking a borrowed ffmpeg::Subtitle.
video_frame_from
Safe wrapper around av_frame_to_video_frame taking a borrowed ffmpeg::Frame. Recommended entry point for most callers — equivalent to passing frame.as_ptr() to the unsafe variant, but the FFmpeg side keeps the frame alive for the duration of the call so the safety contract is satisfied internally.