Expand description
Pure-Rust media codecs: video and audio encoders, decoders, muxers and demuxers, with hardware acceleration where the platform has it and no C dependencies anywhere else.
Everything builds for wasm32 as well as native and depends on nothing
beyond std (rayon with parallel). Outside the opt-in hardware
modules (videotoolbox, nvidia, vaapi) there are no OS, process or
filesystem calls: encoders return Vec<u8>, and the caller owns I/O. It
is the media half of threers, which
re-exports it as threers::codec, split out so programs that only encode
or decode media do not link a renderer.
§Start here
prelude has the types most programs need. Three entry points cover
the common jobs:
use threers_codec::prelude::*;
// Encode RGBA frames to MP4 — on the hardware encoder where there is one.
let mut enc = Mp4Encoder::new(640, 360, EncodeOptions::default()).unwrap();
enc.push_rgba(&vec![128; 640 * 360 * 4]).unwrap();
std::fs::write("out.mp4", enc.finish().unwrap()).unwrap();
// Decode any audio file this crate reads to planar f32.
let pcm = audio_file::read(&std::fs::read("song.flac").unwrap()).unwrap();
println!("{} channels at {} Hz", pcm.channels.len(), pcm.sample_rate);
// Decode a video track (feature `decode`), again hardware first.
let file = std::fs::read("clip.mp4").unwrap();
let track = demux_video(&file).unwrap();
let mut dec = Decoder::new(&track).unwrap();
for s in &track.samples {
dec.push(&file[s.offset as usize..][..s.size as usize], s.pts, s.dts, s.duration as i64)
.unwrap();
if let Some(_bgra) = dec.take(s.pts) {}
}§What is here
| Kind | Modules |
|---|---|
| Video encoders | h264 (CAVLC/CABAC, intra + inter), hevc (intra, alpha), vp9 (intra + inter, alpha) |
| Video decoders | decode: H.264, HEVC, VP9, AV1 — bit-exact against the reference decoders (feature decode) |
| Audio decoders | aac (LC, HE-AAC SBR + PS), mp3, opus (SILK, CELT, hybrid), vorbis, flac, pcm (WAV/RF64, AIFF); audio_file sniffs and dispatches |
| Animated images | gif (encode and decode, feature gif), apng; animation picks one for a browser |
| Containers | mp4 / mp4_stream (write), mp4_demux (read), webm (write), mkv_demux (read, plus IVF), ogg (read) |
| Hardware | hw: one encoder and one decoder over VideoToolbox, NVENC/NVDEC, VAAPI and the native codecs; videotoolbox, videotoolbox_decode (macOS), nvidia (Linux, Windows) and vaapi (Linux) directly |
| Rate control | rate: a quantizer or a bitrate target |
| Conversion | swr (resampling and sample formats), ffcompat (swscale, .mov/.mkv readers, the shapes ffmpeg’s tools produce) |
| Captions | captions: tracks and cues, SRT and WebVTT, muxed by mp4 and webm as soft subtitles |
| Primitives | bitstream (bit writer, Exp-Golomb, emulation prevention), contrast (WCAG luminance) |
§Deep access
Every stage of every codec is a public module, so the building blocks are
there for research, tooling and tests as well as the high-level calls: an
entropy coder (h264::cabac, hevc::cabac, vp9::bool_coder,
decode::av1::msac), a transform (hevc::transform,
decode::av1::itx), a loop filter (h264::deblock,
decode::vp9::lpf), a parameter-set parser (decode::h264::params,
decode::hevc::params), an MDCT (aac::dsp, opus::celt). Each
module’s docs say which section of which specification it implements.
These inner modules follow the structure of their specifications rather
than a stability promise; the high-level types re-exported from each
codec’s root and from prelude are the stable surface.
§Hardware acceleration
hw::Mp4Encoder and decode::Decoder use VideoToolbox on macOS (the
videotoolbox feature), an NVIDIA GPU’s NVENC and NVDEC engines on
Linux and Windows (the nvidia feature), and VA drivers — AMD, Intel —
on Linux (the vaapi feature), and fall back to the
pure-Rust codecs when the feature is off, the machine has no such
hardware, or the hardware refuses a stream. The NVIDIA and VA libraries
are loaded at run time, so a binary built with those features still runs,
natively, where they are missing.
The software paths use every core with parallel: frames of an all-intra
stream are independent, so batches of them encode at once with
byte-identical output (see set_frame_memory_budget).
§Features
| Feature | Default | What it adds |
|---|---|---|
decode | on | The video decoders in decode (H.264, HEVC, VP9, AV1) |
gif | on | GIF89a encode and decode (gif), and GIF in animation |
parallel | Frame-parallel encoding on rayon | |
videotoolbox | VideoToolbox encode and decode on macOS (links Apple’s frameworks; inert elsewhere) | |
nvidia | NVENC encode and NVDEC decode on Linux and Windows (driver loaded at run time; implies decode; inert elsewhere) | |
vaapi | VAAPI HEVC encode and H.264/HEVC/VP9/AV1 decode on Linux (libva loaded at run time; implies decode; inert elsewhere) |
Encoders, audio decoders, muxers, demuxers and captions are always compiled; none of them has a dependency to make optional.
Re-exports§
pub use animation::encode_animation_rgba;pub use animation::encode_animation_rgba_with_progress;pub use animation::format_animation_progress;pub use animation::AnimationEncodeError;pub use animation::AnimationEncodeOptions;pub use animation::AnimationExportPhase;pub use animation::AnimationExportProgress;pub use animation::BrowserCodec;pub use apng::ApngEncoder;pub use bitstream::emulation_prevention;pub use bitstream::rbsp_trailing_bits;pub use bitstream::BitWriter;pub use gif::decode_gif;gifpub use gif::encode_gif;gifpub use gif::DisposalMethod;gifpub use gif::DisposalMode;gifpub use gif::GifDecoder;gifpub use gif::GifEncoder;gifpub use gif::GifError;gifpub use gif::GifFrame;gifpub use gif::GifFrameMeta;gifpub use gif::GifInfo;gifpub use gif::GifOptions;gifpub use gif::GifVersion;gifpub use gif::GifWriter;gifpub use gif::LzwClearMode;gifpub use gif::PaletteMode;gifpub use gif::QuantizerKind;gifpub use gif::DecodedFrame;Deprecated gifpub use vp9::encode_intra_frame;pub use vp9::encode_intra_gray;pub use vp9::Reconstruction;pub use webm::encode_gray_webm;pub use webm::encode_webm;pub use webm::mux_webm;pub use webm::WebmCodec;pub use webm::WebmFrame;pub use webm::WebmParams;
Modules§
- aac
- Pure-Rust AAC-LC and HE-AAC v1/v2 decoding that reproduces ffmpeg n7.1 (
-c:a aac, the native float decoder, aarch64 build): the decoder (AacDecoder), the MP4 track and edit-list handling of its mov demuxer (demux_aac), anddecode_mp4, which yields the planar float samplesffmpeg -i in.mp4 -f f32le -c:a pcm_f32lewould. - animation
- High-level encoders for browser-friendly animated image/video containers.
- apng
- APNG (Animated PNG) encoder — a universally web-supported transparent animation format (pure Rust, wasm-safe).
- audio_
file - One entry point for audio files, decoded the way libsndfile
(
soundfile.read) returns them: planar float, signed integers scaled by1 / 2^(bits-1). Formats: WAV/RF64, AIFF/AIFF-C, FLAC (native and in Ogg), Vorbis in Ogg and MP3, plus AAC in MP4/M4A and Opus in WebM/Matroska (which libsndfile cannot read; decoded as ffmpeg 7.1 does). Recognised by content, not extension. - bitstream
- Bit-level output primitives shared by the native codecs.
- captions
- The caption data model: timed-text tracks and cues, their style, and
SubRip / WebVTT parsing and writing. The muxers in
cratewrite these tracks as soft subtitle streams. - contrast
- WCAG 2.x’s colour contrast: relative luminance and contrast ratio, for the UI’s accessibility checks and caption legibility alike.
- decode
- Native, dependency-free H.264, HEVC, VP9 and AV1 decoding — the software
twin of
videotoolbox_decode::VideoToolboxDecoder. - ffcompat
- ffmpeg-compatible behaviour, without ffmpeg: the values and pictures the ffmpeg n7.1 CLI produces from an MP4/MOV, computed from the file.
- flac
- FLAC decoding (native
.flacstreams and FLAC-in-Ogg viasuper::ogg): STREAMINFO, frames with constant / verbatim / fixed / LPC subframes, wasted bits, Rice and escaped residuals, and the three stereo decorrelations, with the frame CRC-8 / CRC-16 checked. Lossless, so the integer samples are the encoder’s exactly. - gif
gif - GIF89a animated GIF encoder — pure Rust, wasm-safe.
- h264
- Native, from-scratch H.264 / AVC encoder (no ffmpeg, no C bindings).
- hevc
- Native, from-scratch HEVC / H.265 encoder (no ffmpeg, no C bindings).
- hw
- Hardware acceleration with a software fallback, behind one interface each way.
- mkv_
demux - Matroska / WebM demuxer (EBML), and the IVF container, for reading.
- mp3
- MPEG-1/2/2.5 Layer III decoding, ported from mpg123 1.32.3 as libsndfile
1.2.2 (
soundfile) runs it on arm64: float output, gapless, the NEON64 decoder. mpg123 is built with-ffast-math, so the float kernels are generated from the reference binary’s machine code (built, run oncpu::Cpu). - mp4
- Minimal ISOBMFF / MP4 muxer for H.264 and HEVC video tracks, optionally with
a
tx3gtimed-text (subtitle) track. Pure Rust, wasm-safe. - mp4_
demux - Reading an MP4 back apart — the mirror of
mp4. - mp4_
stream - Incremental MP4 writing for live capture, as opposed to the batch muxers
in
crate::mp4. - nvidia
64-bit and nvidiaand (Linux or Windows) - NVIDIA’s hardware video engines — NVDEC for decoding, NVENC for encoding — reached through the driver’s own libraries, loaded when first asked for.
- ogg
- Ogg pages and packets as libogg delivers them: packets reassembled
across pages, each tagged with the page’s granule position when it is
the last packet completed on that page (
-1otherwise), a running packet number, and the end-of-stream flag on the final packet of an EOS page. Only the first logical stream (the first BOS page’s serial) is followed. - opus
- Opus, decoded as ffmpeg n7.1’s native decoder does (
libavcodec/opus, the decoder the ffmpeg CLI picks: “opus (native)”), bit for bit on aarch64: SILK (resampled to 48 kHz by its own libswresample context,filter_size 16), CELT (av_tx’s PFA inverse MDCT over the NEON power-of-two codelets), hybrid mode with the CELT delay FIFO, redundancy frames, multistream channel mapping (families 0, 1, 2, 255) and the OpusHead output gain.decode_webmdrives it the way the CLI does for a WebM/Matroska file: pre-skip from the header,DiscardPaddingtrimmed off the end, and the decoder drained at EOF. - pcm
- Uncompressed PCM files: WAV (RIFF/RF64, PCM 8–32-bit, float 32/64,
WAVE_FORMAT_EXTENSIBLE) and AIFF / AIFF-C (NONE,twos,sowt,fl32,fl64), read to float the way libsndfile (soundfile.read) normalises them — signedint / 2^(bits-1), unsigned 8-bit(v - 128) / 128— and 16-bit WAV writing. - prelude
- The types most programs need, in one import.
- rate
- Average-bitrate control for the all-intra encoders.
- swr
- What the ffmpeg n7.1 CLI does to decoded float audio for
-f s16le -c:a pcm_s16le -ar RATE -ac 2: the auto-insertedaresamplefilter’s libswresample context (default options, aarch64 NEON build). - vaapi
Linux and vaapi - VAAPI: hardware decoding and encoding on Linux GPUs with a VA driver —
AMD (Mesa
radeonsi), Intel (iHD,i965) and others. - vorbis
- Vorbis I decoding, ported from libvorbis 1.3.7 (the decoder inside
libsndfile /
soundfile): codebooks (_book_unquantize, the packed entry search), floor 1, residues 0/1/2, mapping 0 with polar coupling,mdct_backward, andvorbis_synthesis_blockin’s lapping and granule trimming, driven the way libsndfile’svorbis_read_sampledrives it. Floor 0 (pre-1.0 streams) is refused. - vp9
- Native, from-scratch VP9 encoder (no libvpx, no C bindings) — the codec that
rides inside
crate::webmfor a fully dependency-free.webm. - webm
- Minimal WebM (Matroska / EBML) muxer for a single video track (pure Rust, wasm-safe).
Constants§
- DEFAULT_
FRAME_ MEMORY_ BUDGET - Default working-set budget for frame-parallel encoding.
Functions§
- frame_
memory_ budget - How much memory frame-parallel encoding may hold in flight.
- map_
frames_ pub map_frames, public so benchmarks and callers with their own frame source can use the same batching the export paths do.- set_
frame_ memory_ budget - Set how much memory frame-parallel encoding may hold in flight.