Crate openh264[−][src]
Expand description
OpenH264 Rust API
Idiomatic and low-level bindings for OpenH264, converting* between these two in Rust:
*High-level wrapped decoder and encoder.
Example API
Here we convert the last image of a H264 stream to a RGB byte array.
use openh264::decoder::Decoder;
let mut decoder = Decoder::new()?;
let mut rgb_out = [0; 512 * 512 * 3];
let h264_in = include_bytes!("../tests/data/multi_512x512.h264");
// Decode to YUV, then convert and write RGB.
decoder.decode_no_delay(&h264_in[..])?.write_rgb8(&mut rgb_out)?;
Platform Support
Test results on various platforms:
Platform | Compiled | Unit Tested |
---|---|---|
x86_64-pc-windows-msvc | ✅ | ✅ |
x86_64-unknown-linux-gnu | ✅ | ✅ |
x86_64-apple-darwin | ✅ | ✅ |
aarch64-linux-android | 🆗1 | - |
wasm32-unknown-unknown | ❌1,2 | - |
✅ works out of the box; 🆗 the usual shenanigans required; ❌ not supported.
1 via cargo build --target <platform>
, needs CXX
set and libc++_shared.so
.
2 unclear if could ever work, investigation welcome
Performance
Tested on a i9-9900K, Windows 10, single threaded decoding:
test decode_yuv_single_1920x1080 ... bench: 9,243,380 ns/iter (+/- 497,200)
test decode_yuv_single_512x512_cabac ... bench: 1,841,775 ns/iter (+/- 53,211)
test decode_yuv_single_512x512_cavlc ... bench: 2,076,030 ns/iter (+/- 7,287)
test whole_decoder ... bench: 2,874,107 ns/iter (+/- 62,643)
test result: ok. 0 passed; 0 failed; 0 ignored; 5 measured; 0 filtered out; finished in 14.26s
Running unittests (target\release\deps\yuv2rgb-5a3aaabbb6bf3e8a.exe)
running 2 tests
test convert_yuv_to_rgb_1920x1080 ... bench: 7,226,290 ns/iter (+/- 110,871)
test convert_yuv_to_rgb_512x512 ... bench: 907,340 ns/iter (+/- 28,296)
If you want to improve these numbers you can submit PRs that
- better enable autovectorization converting YUV to RGB,
-
conditionally enable assembly in
build.rs
for OpenH264.
Compile Features
backtrace
- Enable backtraces on errors (requires nightly)
FAQ
- How does
openh264-sys2
differ fromopenh264-sys
?
We directly ship OpenH264 source code and provide simple, hand-crafted compilation via cc
in build.rs
. Ouropenh264-sys2
crate should compile via cargo build
out of the box on most platforms, and cross-compile via cargo build --target ...
as
long as the environment variable CXX
is properly set.
- I need to fix an important OpenH264 security hole, how can I update the library?
Cisco’s OpenH264 library is contained in openh264-sys2/upstream
. Updating is (almost, see below) as simple as pulling their latest source,
copying it into that directory, and manually removing all “resource” files. We probably should have a script to strip that folder automatically …
- I heard Rust is super-safe, will this make decoding my videos safe too?
No. Below a thin Rust layer we rely on a very complex C library, and an equally complex standard. Apart from Rust being a much nicer language to work with, depending on this project will give you no additional safety guarantees as far as video handling is concerned. FYI, this is not making a statement about OpenH264, but about the realities of securing +50k lines of C against attacks.
- Feature X is missing or broken, will you fix it?
Right now I only have time to implement what I need. However, I will gladly accept PRs either extending the APIs, or fixing bugs; see below.
OpenH264 Patches Applied
Ideally the embedded upstream should be pristine. That said, the following patches have been applied to fix Valgrind issues and crashes on some platforms:
decoder.cpp
- removedif (pCtx->pDstInfo) pCtx->pDstInfo->iBufferStatus = 0;
which seems to write to previously deallocated memory.
Help with upstreaming them would be appreciated.
Contributing
PRs are very welcome. Feel free to submit PRs and fixes right away. You can open issues if you want to discuss things, but due to time restrictions on my side the project will have to rely on people contributing.
Especially needed:
- BT.601 / BT.709 YUV <-> RGB Conversion
- Enabling of platform specific assembly (without breaking or complicating build)
- Faster YUV to RGB conversion
- Have script to automatically update / import OpenH264 source (or submodule?)
- WASM investigation (either patch, or evidence it can’t be fixed)
- Submit patches upstream
- Feedback which platforms successfully built on
License
Modules
Converts NAL packets to YUV images.
Converts YUV / RGB images to NAL packets.
Handles conversions, e.g., between RGB and YUV.
Structs
Error struct if something goes wrong.