Skip to main content

Crate threers_codec

Crate threers_codec 

Source
Expand description

Pure-Rust media codecs: video and audio encoders, decoders, muxers and demuxers, with hardware acceleration where the platform has it and no C dependencies anywhere else.

Everything builds for wasm32 as well as native and depends on nothing beyond std (rayon with parallel). Outside the opt-in hardware modules (videotoolbox, nvidia, vaapi) there are no OS, process or filesystem calls: encoders return Vec<u8>, and the caller owns I/O. It is the media half of threers, which re-exports it as threers::codec, split out so programs that only encode or decode media do not link a renderer.

§Start here

prelude has the types most programs need. Three entry points cover the common jobs:

use threers_codec::prelude::*;

// Encode RGBA frames to MP4 — on the hardware encoder where there is one.
let mut enc = Mp4Encoder::new(640, 360, EncodeOptions::default()).unwrap();
enc.push_rgba(&vec![128; 640 * 360 * 4]).unwrap();
std::fs::write("out.mp4", enc.finish().unwrap()).unwrap();

// Decode any audio file this crate reads to planar f32.
let pcm = audio_file::read(&std::fs::read("song.flac").unwrap()).unwrap();
println!("{} channels at {} Hz", pcm.channels.len(), pcm.sample_rate);

// Decode a video track (feature `decode`), again hardware first.
let file = std::fs::read("clip.mp4").unwrap();
let track = demux_video(&file).unwrap();
let mut dec = Decoder::new(&track).unwrap();
for s in &track.samples {
    dec.push(&file[s.offset as usize..][..s.size as usize], s.pts, s.dts, s.duration as i64)
        .unwrap();
    if let Some(_bgra) = dec.take(s.pts) {}
}

§What is here

KindModules
Video encodersh264 (CAVLC/CABAC, intra + inter), hevc (intra, alpha), vp9 (intra + inter, alpha)
Video decodersdecode: H.264, HEVC, VP9, AV1 — bit-exact against the reference decoders (feature decode)
Audio encodersaac::encoder (AAC-LC to raw units or ADTS, 8 to 96 kHz, mono and stereo) and aac::m4a (the same in an .m4a, with the priming edit list); mp3::encoder (MPEG-1 Layer III, 32 to 320 kbit/s, with a gapless Info frame)
Audio decodersaac (LC, HE-AAC SBR + PS), mp3, opus (SILK, CELT, hybrid), vorbis, flac, pcm (WAV/RF64, AIFF); audio_file sniffs and dispatches
Animated imagesgif (encode and decode, feature gif), apng; animation picks one for a browser
Containersmp4 / mp4_stream (write), mp4_demux (read), webm (write), mkv_demux (read, plus IVF), ogg (read)
Hardwarehw: one encoder and one decoder over VideoToolbox, NVENC/NVDEC, VAAPI and the native codecs; videotoolbox, videotoolbox_decode (macOS), nvidia (Linux, Windows) and vaapi (Linux) directly
Rate controlrate: a quantizer or a bitrate target
Conversionswr (resampling and sample formats), ffcompat (swscale, .mov/.mkv readers, the shapes ffmpeg’s tools produce)
Captionscaptions: tracks and cues, SRT and WebVTT, muxed by mp4 and webm as soft subtitles
Primitivesbitstream (bit writer, Exp-Golomb, emulation prevention), contrast (WCAG luminance)

§Deep access

Every stage of every codec is a public module, so the building blocks are there for research, tooling and tests as well as the high-level calls: an entropy coder (h264::cabac, hevc::cabac, vp9::bool_coder, decode::av1::msac), a transform (hevc::transform, decode::av1::itx), a loop filter (h264::deblock, decode::vp9::lpf), a parameter-set parser (decode::h264::params, decode::hevc::params), an MDCT (aac::dsp, opus::celt). Each module’s docs say which section of which specification it implements. These inner modules follow the structure of their specifications rather than a stability promise; the high-level types re-exported from each codec’s root and from prelude are the stable surface.

§Hardware acceleration

hw::Mp4Encoder and decode::Decoder use VideoToolbox on macOS (the videotoolbox feature), an NVIDIA GPU’s NVENC and NVDEC engines on Linux and Windows (the nvidia feature), and VA drivers — AMD, Intel — on Linux (the vaapi feature), and fall back to the pure-Rust codecs when the feature is off, the machine has no such hardware, or the hardware refuses a stream. The NVIDIA and VA libraries are loaded at run time, so a binary built with those features still runs, natively, where they are missing. The software paths use every core with parallel: frames of an all-intra stream are independent, so batches of them encode at once with byte-identical output (see set_frame_memory_budget).

§Features

FeatureDefaultWhat it adds
decodeonThe video decoders in decode (H.264, HEVC, VP9, AV1)
gifonGIF89a encode and decode (gif), and GIF in animation
parallelFrame-parallel encoding, and AV1 decoding across frames, tiles and loop filters, on rayon
videotoolboxVideoToolbox encode and decode on macOS (links Apple’s frameworks; inert elsewhere)
nvidiaNVENC encode and NVDEC decode on Linux and Windows (driver loaded at run time; implies decode; inert elsewhere)
vaapiVAAPI HEVC encode and H.264/HEVC/VP9/AV1 decode on Linux (libva loaded at run time; implies decode; inert elsewhere)

Encoders, audio decoders, muxers, demuxers and captions are always compiled; none of them has a dependency to make optional.

Re-exports§

pub use animation::encode_animation_rgba;
pub use animation::encode_animation_rgba_with_progress;
pub use animation::format_animation_progress;
pub use animation::AnimationEncodeError;
pub use animation::AnimationEncodeOptions;
pub use animation::AnimationExportPhase;
pub use animation::AnimationExportProgress;
pub use animation::BrowserCodec;
pub use apng::ApngEncoder;
pub use bitstream::emulation_prevention;
pub use bitstream::rbsp_trailing_bits;
pub use bitstream::BitWriter;
pub use gif::DecodedFrame;Deprecatedgif
pub use gif::decode_gif;gif
pub use gif::encode_gif;gif
pub use gif::DisposalMethod;gif
pub use gif::DisposalMode;gif
pub use gif::GifDecoder;gif
pub use gif::GifEncoder;gif
pub use gif::GifError;gif
pub use gif::GifFrame;gif
pub use gif::GifFrameMeta;gif
pub use gif::GifInfo;gif
pub use gif::GifOptions;gif
pub use gif::GifVersion;gif
pub use gif::GifWriter;gif
pub use gif::LzwClearMode;gif
pub use gif::PaletteMode;gif
pub use gif::QuantizerKind;gif
pub use vp9::encode_intra_frame;
pub use vp9::encode_intra_gray;
pub use vp9::Reconstruction;
pub use webm::encode_gray_webm;
pub use webm::encode_webm;
pub use webm::mux_webm;
pub use webm::WebmCodec;
pub use webm::WebmFrame;
pub use webm::WebmParams;
pub use threers_captions as captions;

Modules§

aac
Pure-Rust AAC-LC and HE-AAC v1/v2 decoding that reproduces ffmpeg n7.1 (-c:a aac, the native float decoder, aarch64 build): the decoder (AacDecoder), the MP4 track and edit-list handling of its mov demuxer (demux_aac), and decode_mp4, which yields the planar float samples ffmpeg -i in.mp4 -f f32le -c:a pcm_f32le would.
animation
High-level encoders for browser-friendly animated image/video containers.
apng
APNG (Animated PNG) encoder — a universally web-supported transparent animation format (pure Rust, wasm-safe).
audio_file
One entry point for audio files, decoded the way libsndfile (soundfile.read) returns them: planar float, signed integers scaled by 1 / 2^(bits-1). Formats: WAV/RF64, AIFF/AIFF-C, FLAC (native and in Ogg), Vorbis in Ogg and MP3, plus AAC in MP4/M4A and Opus in WebM/Matroska (which libsndfile cannot read; decoded as ffmpeg 7.1 does). Recognised by content, not extension.
av1decode
A native AV1 encoder: key frames only.
bitstream
Bit-level output primitives shared by the native codecs.
bytes
Bytes shared between readers without copying: a buffer in memory, or a file mapped into it.
contrast
WCAG luminance and contrast, from threers_captions. WCAG 2.x’s colour contrast: relative luminance and contrast ratio, for the UI’s accessibility checks and caption legibility alike.
decode
Native, dependency-free H.264, HEVC, VP9 and AV1 decoding — the software twin of videotoolbox_decode::VideoToolboxDecoder.
ffcompat
ffmpeg-compatible behaviour, without ffmpeg: the values and pictures the ffmpeg n7.1 CLI produces from an MP4/MOV, computed from the file.
flac
FLAC decoding (native .flac streams and FLAC-in-Ogg via super::ogg): STREAMINFO, frames with constant / verbatim / fixed / LPC subframes, wasted bits, Rice and escaped residuals, and the three stereo decorrelations, with the frame CRC-8 / CRC-16 checked. Lossless, so the integer samples are the encoder’s exactly.
gifgif
GIF89a animated GIF encoder — pure Rust, wasm-safe.
h264
Native, from-scratch H.264 / AVC encoder (no ffmpeg, no C bindings).
hevc
Native, from-scratch HEVC / H.265 encoder (no ffmpeg, no C bindings).
hw
Hardware acceleration with a software fallback, behind one interface each way.
mkv_demux
Matroska / WebM demuxer (EBML), and the IVF container, for reading.
mp3
MPEG-1/2/2.5 Layer III decoding, ported from mpg123 1.32.3 as libsndfile 1.2.2 (soundfile) runs it on arm64: float output, gapless, the NEON64 decoder. mpg123 is built with -ffast-math, so the float kernels are generated from the reference binary’s machine code (built, run on cpu::Cpu).
mp4
Minimal ISOBMFF / MP4 muxer for H.264 and HEVC video tracks, optionally with a tx3g timed-text (subtitle) track. Pure Rust, wasm-safe.
mp4_demux
Reading an MP4 back apart — the mirror of mp4.
mp4_stream
Incremental MP4 writing for live capture, as opposed to the batch muxers in crate::mp4.
nvidia64-bit and nvidia and (Linux or Windows)
NVIDIA’s hardware video engines — NVDEC for decoding, NVENC for encoding — reached through the driver’s own libraries, loaded when first asked for.
ogg
Ogg pages and packets as libogg delivers them: packets reassembled across pages, each tagged with the page’s granule position when it is the last packet completed on that page (-1 otherwise), a running packet number, and the end-of-stream flag on the final packet of an EOS page. Only the first logical stream (the first BOS page’s serial) is followed.
opus
Opus, decoded as ffmpeg n7.1’s native decoder does (libavcodec/opus, the decoder the ffmpeg CLI picks: “opus (native)”), bit for bit on aarch64: SILK (resampled to 48 kHz by its own libswresample context, filter_size 16), CELT (av_tx’s PFA inverse MDCT over the NEON power-of-two codelets), hybrid mode with the CELT delay FIFO, redundancy frames, multistream channel mapping (families 0, 1, 2, 255) and the OpusHead output gain. decode_webm drives it the way the CLI does for a WebM/Matroska file: pre-skip from the header, DiscardPadding trimmed off the end, and the decoder drained at EOF.
pcm
Uncompressed PCM files: WAV (RIFF/RF64, PCM 8–32-bit, float 32/64, WAVE_FORMAT_EXTENSIBLE) and AIFF / AIFF-C (NONE, twos, sowt, fl32, fl64), read to float the way libsndfile (soundfile.read) normalises them — signed int / 2^(bits-1), unsigned 8-bit (v - 128) / 128 — and 16-bit WAV writing.
prelude
The types most programs need, in one import.
rate
Average-bitrate control for the all-intra encoders.
swr
What the ffmpeg n7.1 CLI does to decoded float audio for -f s16le -c:a pcm_s16le -ar RATE -ac 2: the auto-inserted aresample filter’s libswresample context (default options, aarch64 NEON build).
v4l2Linux and v4l2
Hardware H.264 encoding through a V4L2 memory-to-memory encoder (feature v4l2, Linux): the Raspberry Pi’s bcm2835-codec (/dev/video11 on a Pi 4), or any stateful V4L2 encoder with a multi-planar YU12 → H264 interface. I420 in, H.264 Annex B out, no CPU spent encoding (a 1280x720 frame in ~11 ms of wall time on a Pi 4, a 320x180 one in ~2.4 ms).
vaapiLinux and vaapi
VAAPI: hardware decoding and encoding on Linux GPUs with a VA driver — AMD (Mesa radeonsi), Intel (iHD, i965) and others.
vorbis
Vorbis I decoding, ported from libvorbis 1.3.7 (the decoder inside libsndfile / soundfile): codebooks (_book_unquantize, the packed entry search), floor 1, residues 0/1/2, mapping 0 with polar coupling, mdct_backward, and vorbis_synthesis_blockin’s lapping and granule trimming, driven the way libsndfile’s vorbis_read_sample drives it. Floor 0 (pre-1.0 streams) is refused.
vp9
Native, from-scratch VP9 encoder (no libvpx, no C bindings) — the codec that rides inside crate::webm for a fully dependency-free .webm.
webm
Minimal WebM (Matroska / EBML) muxer for a single video track (pure Rust, wasm-safe).

Constants§

DEFAULT_FRAME_MEMORY_BUDGET
Default working-set budget for frame-parallel encoding.

Functions§

frame_memory_budget
How much memory frame-parallel encoding may hold in flight.
map_frames_pub
map_frames, public so benchmarks and callers with their own frame source can use the same batching the export paths do.
set_frame_memory_budget
Set how much memory frame-parallel encoding may hold in flight.