<div align="center">
<h1>mediadecode-ffmpeg</h1>
</div>
<div align="center">
FFmpeg adapter for the [`mediadecode`](../mediadecode) abstraction
layer, built on top of
[`ffmpeg-next`](https://crates.io/crates/ffmpeg-next).
[<img alt="github" src="https://img.shields.io/badge/github-findit--ai/mediadecode-8da0cb?style=for-the-badge&logo=Github" height="22">][Github-url]
<img alt="LoC" src="https://img.shields.io/endpoint?url=https%3A%2F%2Fgist.githubusercontent.com%2Fal8n%2F327b2a8aef9003246e45c6e47fe63937%2Fraw%2Fmediadecode-ffmpeg" height="22">
[<img alt="Build" src="https://img.shields.io/github/actions/workflow/status/findit-ai/mediadecode/ci-ffmpeg.yml?logo=Github-Actions&style=for-the-badge" height="22">][CI-url]
[<img alt="codecov" src="https://img.shields.io/codecov/c/gh/findit-ai/mediadecode?style=for-the-badge&logo=codecov" height="22">][codecov-url]
[<img alt="docs.rs" src="https://img.shields.io/badge/docs.rs-mediadecode--ffmpeg-66c2a5?style=for-the-badge&labelColor=555555&logo=data:image/svg+xml;base64,PHN2ZyByb2xlPSJpbWciIHhtbG5zPSJodHRwOi8vd3d3LnczLm9yZy8yMDAwL3N2ZyIgdmlld0JveD0iMCAwIDUxMiA1MTIiPjxwYXRoIGZpbGw9IiNmNWY1ZjUiIGQ9Ik00ODguNiAyNTAuMkwzOTIgMjE0VjEwNS41YzAtMTUtOS4zLTI4LjQtMjMuNC0zMy43bC0xMDAtMzcuNWMtOC4xLTMuMS0xNy4xLTMuMS0yNS4zIDBsLTEwMCAzNy41Yy0xNC4xIDUuMy0yMy40IDE4LjctMjMuNCAzMy43VjIxNGwtOTYuNiAzNi4yQzkuMyAyNTUuNSAwIDI2OC45IDAgMjgzLjlWMzk0YzAgMTMuNiA3LjcgMjYuMSAxOS45IDMyLjJsMTAwIDUwYzEwLjEgNS4xIDIyLjEgNS4xIDMyLjIgMGwxMDMuOS01MiAxMDMuOSA1MmMxMC4xIDUuMSAyMi4xIDUuMSAzMi4yIDBsMTAwLTUwYzEyLjItNi4xIDE5LjktMTguNiAxOS45LTMyLjJWMjgzLjljMC0xNS05LjMtMjguNC0yMy40LTMzLjd6TTM1OCAyMTQuOGwtODUgMzEuOXYtNjguMmw4NS0zN3Y3My4zek0xNTQgMTA0LjFsMTAyLTM4LjIgMTAyIDM4LjJ2LjZsLTEwMiA0MS40LTEwMi00MS40di0uNnptODQgMjkxLjFsLTg1IDQyLjV2LTc5LjFsODUtMzguOHY3NS40em0wLTExMmwtMTAyIDQxLjQtMTAyLTQxLjR2LS42bDEwMi0zOC4yIDEwMiAzOC4ydi42em0yNDAgMTEybC04NSA0Mi41di03OS4xbDg1LTM4Ljh2NzUuNHptMC0xMTJsLTEwMiA0MS40LTEwMi00MS40di0uNmwxMDItMzguMiAxMDIgMzguMnYuNnoiPjwvcGF0aD48L3N2Zz4K" height="20">][doc-url]
[<img alt="crates.io" src="https://img.shields.io/crates/v/mediadecode-ffmpeg?style=for-the-badge&logo=data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBlbmNvZGluZz0iaXNvLTg4NTktMSI/Pg0KPCEtLSBHZW5lcmF0b3I6IEFkb2JlIElsbHVzdHJhdG9yIDE5LjAuMCwgU1ZHIEV4cG9ydCBQbHVnLUluIC4gU1ZHIFZlcnNpb246IDYuMDAgQnVpbGQgMCkgIC0tPg0KPHN2ZyB2ZXJzaW9uPSIxLjEiIGlkPSJMYXllcl8xIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHhtbG5zOnhsaW5rPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5L3hsaW5rIiB4PSIwcHgiIHk9IjBweCINCgkgdmlld0JveD0iMCAwIDUxMiA1MTIiIHhtbDpzcGFjZT0icHJlc2VydmUiPg0KPGc+DQoJPGc+DQoJCTxwYXRoIGQ9Ik0yNTYsMEwzMS41MjgsMTEyLjIzNnYyODcuNTI4TDI1Niw1MTJsMjI0LjQ3Mi0xMTIuMjM2VjExMi4yMzZMMjU2LDB6IE0yMzQuMjc3LDQ1Mi41NjRMNzQuOTc0LDM3Mi45MTNWMTYwLjgxDQoJCQlsMTU5LjMwMyw3OS42NTFWNDUyLjU2NHogTTEwMS44MjYsMTI1LjY2MkwyNTYsNDguNTc2bDE1NC4xNzQsNzcuMDg3TDI1NiwyMDIuNzQ5TDEwMS44MjYsMTI1LjY2MnogTTQzNy4wMjYsMzcyLjkxMw0KCQkJbC0xNTkuMzAzLDc5LjY1MVYyNDAuNDYxbDE1OS4zMDMtNzkuNjUxVjM3Mi45MTN6IiBmaWxsPSIjRkZGIi8+DQoJPC9nPg0KPC9nPg0KPGc+DQo8L2c+DQo8Zz4NCjwvZz4NCjxnPg0KPC9nPg0KPGc+DQo8L2c+DQo8Zz4NCjwvZz4NCjxnPg0KPC9nPg0KPGc+DQo8L2c+DQo8Zz4NCjwvZz4NCjxnPg0KPC9nPg0KPGc+DQo8L2c+DQo8Zz4NCjwvZz4NCjxnPg0KPC9nPg0KPGc+DQo8L2c+DQo8Zz4NCjwvZz4NCjxnPg0KPC9nPg0KPC9zdmc+DQo=" height="22">][crates-url]
[<img alt="crates.io" src="https://img.shields.io/crates/d/mediadecode-ffmpeg?color=critical&logo=data:image/svg+xml;base64,PD94bWwgdmVyc2lvbj0iMS4wIiBzdGFuZGFsb25lPSJubyI/PjwhRE9DVFlQRSBzdmcgUFVCTElDICItLy9XM0MvL0RURCBTVkcgMS4xLy9FTiIgImh0dHA6Ly93d3cudzMub3JnL0dyYXBoaWNzL1NWRy8xLjEvRFREL3N2ZzExLmR0ZCI+PHN2ZyB0PSIxNjQ1MTE3MzMyOTU5IiBjbGFzcz0iaWNvbiIgdmlld0JveD0iMCAwIDEwMjQgMTAyNCIgdmVyc2lvbj0iMS4xIiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHAtaWQ9IjM0MjEiIGRhdGEtc3BtLWFuY2hvci1pZD0iYTMxM3guNzc4MTA2OS4wLmkzIiB3aWR0aD0iNDgiIGhlaWdodD0iNDgiIHhtbG5zOnhsaW5rPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5L3hsaW5rIj48ZGVmcz48c3R5bGUgdHlwZT0idGV4dC9jc3MiPjwvc3R5bGU+PC9kZWZzPjxwYXRoIGQ9Ik00NjkuMzEyIDU3MC4yNHYtMjU2aDg1LjM3NnYyNTZoMTI4TDUxMiA3NTYuMjg4IDM0MS4zMTIgNTcwLjI0aDEyOHpNMTAyNCA2NDAuMTI4QzEwMjQgNzgyLjkxMiA5MTkuODcyIDg5NiA3ODcuNjQ4IDg5NmgtNTEyQzEyMy45MDQgODk2IDAgNzYxLjYgMCA1OTcuNTA0IDAgNDUxLjk2OCA5NC42NTYgMzMxLjUyIDIyNi40MzIgMzAyLjk3NiAyODQuMTYgMTk1LjQ1NiAzOTEuODA4IDEyOCA1MTIgMTI4YzE1Mi4zMiAwIDI4Mi4xMTIgMTA4LjQxNiAzMjMuMzkyIDI2MS4xMkM5NDEuODg4IDQxMy40NCAxMDI0IDUxOS4wNCAxMDI0IDY0MC4xOTJ6IG0tMjU5LjItMjA1LjMxMmMtMjQuNDQ4LTEyOS4wMjQtMTI4Ljg5Ni0yMjIuNzItMjUyLjgtMjIyLjcyLTk3LjI4IDAtMTgzLjA0IDU3LjM0NC0yMjQuNjQgMTQ3LjQ1NmwtOS4yOCAyMC4yMjQtMjAuOTI4IDIuOTQ0Yy0xMDMuMzYgMTQuNC0xNzguMzY4IDEwNC4zMi0xNzguMzY4IDIxNC43MiAwIDExNy45NTIgODguODMyIDIxNC40IDE5Ni45MjggMjE0LjRoNTEyYzg4LjMyIDAgMTU3LjUwNC03NS4xMzYgMTU3LjUwNC0xNzEuNzEyIDAtODguMDY0LTY1LjkyLTE2NC45MjgtMTQ0Ljk2LTE3MS43NzZsLTI5LjUwNC0yLjU2LTUuODg4LTMwLjk3NnoiIGZpbGw9IiNmZmZmZmYiIHAtaWQ9IjM0MjIiIGRhdGEtc3BtLWFuY2hvci1pZD0iYTMxM3guNzc4MTA2OS4wLmkwIiBjbGFzcz0iIj48L3BhdGg+PC9zdmc+&style=for-the-badge" height="22">][crates-url]
<img alt="license" src="https://img.shields.io/badge/License-Apache%202.0/MIT-blue.svg?style=for-the-badge&fontColor=white&logoColor=f5c076&logo=data:image/svg+xml;base64,PCFET0NUWVBFIHN2ZyBQVUJMSUMgIi0vL1czQy8vRFREIFNWRyAxLjEvL0VOIiAiaHR0cDovL3d3dy53My5vcmcvR3JhcGhpY3MvU1ZHLzEuMS9EVEQvc3ZnMTEuZHRkIj4KDTwhLS0gVXBsb2FkZWQgdG86IFNWRyBSZXBvLCB3d3cuc3ZncmVwby5jb20sIFRyYW5zZm9ybWVkIGJ5OiBTVkcgUmVwbyBNaXhlciBUb29scyAtLT4KPHN2ZyBmaWxsPSIjZmZmZmZmIiBoZWlnaHQ9IjgwMHB4IiB3aWR0aD0iODAwcHgiIHZlcnNpb249IjEuMSIgaWQ9IkNhcGFfMSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIiB4bWxuczp4bGluaz0iaHR0cDovL3d3dy53My5vcmcvMTk5OS94bGluayIgdmlld0JveD0iMCAwIDI3Ni43MTUgMjc2LjcxNSIgeG1sOnNwYWNlPSJwcmVzZXJ2ZSIgc3Ryb2tlPSIjZmZmZmZmIj4KDTxnIGlkPSJTVkdSZXBvX2JnQ2FycmllciIgc3Ryb2tlLXdpZHRoPSIwIi8+Cg08ZyBpZD0iU1ZHUmVwb190cmFjZXJDYXJyaWVyIiBzdHJva2UtbGluZWNhcD0icm91bmQiIHN0cm9rZS1saW5lam9pbj0icm91bmQiLz4KDTxnIGlkPSJTVkdSZXBvX2ljb25DYXJyaWVyIj4gPGc+IDxwYXRoIGQ9Ik0xMzguMzU3LDBDNjIuMDY2LDAsMCw2Mi4wNjYsMCwxMzguMzU3czYyLjA2NiwxMzguMzU3LDEzOC4zNTcsMTM4LjM1N3MxMzguMzU3LTYyLjA2NiwxMzguMzU3LTEzOC4zNTcgUzIxNC42NDgsMCwxMzguMzU3LDB6IE0xMzguMzU3LDI1OC43MTVDNzEuOTkyLDI1OC43MTUsMTgsMjA0LjcyMywxOCwxMzguMzU3UzcxLjk5MiwxOCwxMzguMzU3LDE4IHMxMjAuMzU3LDUzLjk5MiwxMjAuMzU3LDEyMC4zNTdTMjA0LjcyMywyNTguNzE1LDEzOC4zNTcsMjU4LjcxNXoiLz4gPHBhdGggZD0iTTE5NC43OTgsMTYwLjkwM2MtNC4xODgtMi42NzctOS43NTMtMS40NTQtMTIuNDMyLDIuNzMyYy04LjY5NCwxMy41OTMtMjMuNTAzLDIxLjcwOC0zOS42MTQsMjEuNzA4IGMtMjUuOTA4LDAtNDYuOTg1LTIxLjA3OC00Ni45ODUtNDYuOTg2czIxLjA3Ny00Ni45ODYsNDYuOTg1LTQ2Ljk4NmMxNS42MzMsMCwzMC4yLDcuNzQ3LDM4Ljk2OCwyMC43MjMgYzIuNzgyLDQuMTE3LDguMzc1LDUuMjAxLDEyLjQ5NiwyLjQxOGM0LjExOC0yLjc4Miw1LjIwMS04LjM3NywyLjQxOC0xMi40OTZjLTEyLjExOC0xNy45MzctMzIuMjYyLTI4LjY0NS01My44ODItMjguNjQ1IGMtMzUuODMzLDAtNjQuOTg1LDI5LjE1Mi02NC45ODUsNjQuOTg2czI5LjE1MiA2NC45ODYsNjQuOTg1LDY0Ljk4NmMyMi4yODEsMCw0Mi43NTktMTEuMjE4LDU0Ljc3OC0zMC4wMDkgQzIwMC4yMDgsMTY5LjE0NywxOTguOTg1LDE2My41ODIsMTk0Ljc5OCwxNjAuOTAzeiIvPiA8L2c+IDwvZz4KDTwvc3ZnPg==" height="22">
</div>
Implements `mediadecode`'s `VideoAdapter` / `AudioAdapter` /
`SubtitleAdapter` traits and the matching push-style `*StreamDecoder`
traits. Frame payloads are zero-copy refcounted views over FFmpeg's
`AVBufferRef` via the [`FfmpegBuffer`] type — receiving a frame does
not memcpy the pixel data.
`FfmpegVideoStreamDecoder` mirrors the `send_packet` / `receive_frame`
shape of `ffmpeg::decoder::Video`, auto-probes the host's HW backends,
and falls through to a software decoder when none open. Audio and
subtitles use parallel `FfmpegAudioStreamDecoder` /
`FfmpegSubtitleStreamDecoder` types.
## Backends
`FfmpegVideoStreamDecoder::open` walks this probe order, opening the
first backend that accepts the stream:
| Target | Probe order |
| ------------------- | --------------------------------- |
| macOS / iOS / tvOS | VideoToolbox → software |
| Linux | VAAPI → CUDA → software |
| Windows | D3D11VA → CUDA → software |
| other | software |
Output frames are CPU-side, downloaded with `av_hwframe_transfer_data`
(NV12 for 8-bit, P010/P012/P016/P210/P212/P216/P410/P412/P416 for
10/12/16-bit). Pixel-format conversion is intentionally out of scope
— downstream
[`colconv`](https://github.com/findit-ai/colconv) handles it.
If every HW backend opens but later fails at decode time and the
software backend is also unavailable, the error surfaces as
`VideoDecodeError::Decode(Error::AllBackendsFailed(p))` carrying any
packets the decoder had already accepted from the demuxer (accessible
via `p.unconsumed_packets()` / `p.into_unconsumed_packets()`) — so
non-seekable callers (live streams, pipes, network sources) can replay
them through their own software decoder without re-demuxing.
## Usage
```rust,no_run
use ffmpeg_next as ffmpeg;
use ffmpeg::{format, media};
use mediadecode::{Timebase, decoder::VideoStreamDecoder};
use mediadecode_ffmpeg::{
Error as FfmpegError, FfmpegVideoStreamDecoder, VideoDecodeError,
empty_video_frame, video_packet_from_ffmpeg,
};
fn main() -> Result<(), Box<dyn std::error::Error>> {
ffmpeg::init()?;
let path = std::env::args().nth(1).expect("usage: <input-file>");
let mut input = format::input(&path)?;
let stream = input.streams().best(media::Type::Video).unwrap();
let stream_index = stream.index();
let time_base = Timebase::new(
stream.time_base().numerator() as u32,
std::num::NonZeroU32::new(stream.time_base().denominator() as u32).unwrap(),
);
// Probes HW backends in order, falls back to software.
let mut decoder = match FfmpegVideoStreamDecoder::open(stream.parameters(), time_base) {
Ok(d) => d,
Err(FfmpegError::AllBackendsFailed(p)) => {
// No backend at all could open this stream — including software.
// `unconsumed_packets` is empty at open-time. Caller decides.
let _unconsumed_packets = p.into_unconsumed_packets();
return Ok(());
}
Err(e) => return Err(e.into()),
};
let mut frame = empty_video_frame();
for (s, av_packet) in input.packets() {
if s.index() != stream_index { continue; }
let Some(pkt) = video_packet_from_ffmpeg(&av_packet) else { continue };
match decoder.send_packet(&pkt) {
Ok(()) => {}
Err(VideoDecodeError::Decode(FfmpegError::AllBackendsFailed(p))) => {
// Runtime exhaustion: rescued packets are the bytes the decoder
// already consumed from `input`. Replay them through your own
// software decoder before the current packet so non-seekable
// sources recover cleanly.
let _unconsumed_packets = p.into_unconsumed_packets();
return Ok(());
}
Err(e) => return Err(e.into()),
}
while decoder.receive_frame(&mut frame).is_ok() {
// frame.pixel_format(), frame.width(), frame.height(),
// frame.planes() — zero-copy views over AVBufferRef.
}
}
decoder.send_eof()?;
while decoder.receive_frame(&mut frame).is_ok() { /* drain */ }
Ok(())
}
```
Audio and subtitle decoding share the shape — see
[`examples/decode_via_trait.rs`](examples/decode_via_trait.rs) and
[`tests/audio_subtitle_via_trait.rs`](tests/audio_subtitle_via_trait.rs)
for end-to-end demuxer-driven runs that cover all three streams.
## Public surface map
- **Decoders**: `FfmpegVideoStreamDecoder`,
`FfmpegAudioStreamDecoder`, `FfmpegSubtitleStreamDecoder`. Plus
their error types: `VideoDecodeError`, `AudioDecodeError`,
`SubtitleDecodeError`.
- **Type aliases**: `VideoPacket`, `AudioPacket`, `SubtitlePacket`,
`VideoFrame`, `AudioFrame`, `SubtitleFrame` — the `mediadecode`
generic types pre-parameterized with this crate's adapter / buffer /
extras, so you don't have to spell them out.
- **Buffer**: `FfmpegBuffer` — refcounted view over an `AVBufferRef`
with safe constructors (`empty`, `from_packet`, `try_*`
panic-free counterparts).
- **Boundary helpers**: `video_packet_from_ffmpeg`,
`audio_packet_from_ffmpeg`, `subtitle_packet_from_ffmpeg` — convert a
borrowed `ffmpeg::Packet` into the matching `mediadecode` packet
without copying the compressed payload.
- **Empty-frame builders**: `empty_video_frame`, `empty_audio_frame`,
`empty_subtitle_frame` — well-formed destinations for `receive_frame`.
## Running tests and benches
The integration test and benchmark expect a real video file. Set
`MEDIADECODE_SAMPLE_VIDEO` to enable them:
```sh
MEDIADECODE_SAMPLE_VIDEO=/path/to/clip.mp4 cargo test
MEDIADECODE_SAMPLE_VIDEO=/path/to/clip.mp4 cargo test --test hw_smoke -- --ignored
MEDIADECODE_SAMPLE_VIDEO=/path/to/clip.mp4 cargo bench
```
Without the env var the integration tests skip with a notice; unit
tests run unconditionally.
## Build requirements
- A system FFmpeg ≥ **5.1** linkable via `pkg-config` (we reference
`AV_PIX_FMT_P212LE` / `AV_PIX_FMT_P412LE`, which were added in 5.1).
Tested against 8.1. Verify with `ffmpeg -hwaccels` that your build
has the backends you expect compiled in
(e.g. `videotoolbox` on macOS, `vaapi` / `cuda` on Linux,
`d3d11va` / `cuda` on Windows).
- Rust ≥ **1.95**, edition 2024.
## License
`mediadecode-ffmpeg` is under the terms of both the MIT license and the
Apache License (Version 2.0).
See [LICENSE-APACHE](LICENSE-APACHE), [LICENSE-MIT](LICENSE-MIT) for details.
Copyright (c) 2026 FinDIT Studio authors.
[Github-url]: https://github.com/findit-ai/mediadecode
[CI-url]: https://github.com/findit-ai/mediadecode/actions/workflows/ci.yml
[codecov-url]: https://app.codecov.io/gh/findit-ai/mediadecode/
[doc-url]: https://docs.rs/mediadecode-ffmpeg
[crates-url]: https://crates.io/crates/mediadecode-ffmpeg