1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
//! Pure-Rust media codecs: video and audio encoders, decoders, muxers and
//! demuxers, with hardware acceleration where the platform has it and no C
//! dependencies anywhere else.
//!
//! Everything builds for `wasm32` as well as native and depends on nothing
//! beyond `std` (`rayon` with `parallel`). Outside the opt-in hardware
//! modules (`videotoolbox`, `nvidia`, `vaapi`) there are no OS, process or
//! filesystem calls: encoders return `Vec<u8>`, and the caller owns I/O. It
//! is the media half of [threers](https://crates.io/crates/threers), which
//! re-exports it as `threers::codec`, split out so programs that only encode
//! or decode media do not link a renderer.
//!
//! # Start here
//!
//! [`prelude`] has the types most programs need. Three entry points cover
//! the common jobs:
//!
//! ```no_run
//! # #[cfg(feature = "decode")] {
//! use threers_codec::prelude::*;
//!
//! // Encode RGBA frames to MP4 — on the hardware encoder where there is one.
//! let mut enc = Mp4Encoder::new(640, 360, EncodeOptions::default()).unwrap();
//! enc.push_rgba(&vec![128; 640 * 360 * 4]).unwrap();
//! std::fs::write("out.mp4", enc.finish().unwrap()).unwrap();
//!
//! // Decode any audio file this crate reads to planar f32.
//! let pcm = audio_file::read(&std::fs::read("song.flac").unwrap()).unwrap();
//! println!("{} channels at {} Hz", pcm.channels.len(), pcm.sample_rate);
//!
//! // Decode a video track (feature `decode`), again hardware first.
//! let file = std::fs::read("clip.mp4").unwrap();
//! let track = demux_video(&file).unwrap();
//! let mut dec = Decoder::new(&track).unwrap();
//! for s in &track.samples {
//! dec.push(&file[s.offset as usize..][..s.size as usize], s.pts, s.dts, s.duration as i64)
//! .unwrap();
//! if let Some(_bgra) = dec.take(s.pts) {}
//! }
//! # }
//! ```
//!
//! # What is here
//!
//! | Kind | Modules |
//! |------|---------|
//! | Video encoders | [`h264`] (CAVLC/CABAC, intra + inter), [`hevc`] (intra, alpha), [`vp9`] (intra + inter, alpha) |
//! | Video decoders | [`decode`]: H.264, HEVC, VP9, AV1 — bit-exact against the reference decoders (feature `decode`) |
//! | Audio decoders | [`aac`] (LC, HE-AAC SBR + PS), [`mp3`], [`opus`] (SILK, CELT, hybrid), [`vorbis`], [`flac`], [`pcm`] (WAV/RF64, AIFF); [`audio_file`] sniffs and dispatches |
//! | Animated images | [`gif`] (encode and decode, feature `gif`), [`apng`]; [`animation`] picks one for a browser |
//! | Containers | [`mp4`] / [`mp4_stream`] (write), [`mp4_demux`] (read), [`webm`] (write), [`mkv_demux`] (read, plus IVF), [`ogg`] (read) |
//! | Hardware | [`hw`]: one encoder and one decoder over VideoToolbox, NVENC/NVDEC, VAAPI and the native codecs; `videotoolbox`, `videotoolbox_decode` (macOS), `nvidia` (Linux, Windows) and `vaapi` (Linux) directly |
//! | Rate control | [`rate`]: a quantizer or a bitrate target |
//! | Conversion | [`swr`] (resampling and sample formats), [`ffcompat`] (`swscale`, `.mov`/`.mkv` readers, the shapes ffmpeg's tools produce) |
//! | Captions | [`captions`]: tracks and cues, SRT and WebVTT, muxed by `mp4` and `webm` as soft subtitles |
//! | Primitives | [`bitstream`] (bit writer, Exp-Golomb, emulation prevention), [`contrast`] (WCAG luminance) |
//!
//! # Deep access
//!
//! Every stage of every codec is a public module, so the building blocks are
//! there for research, tooling and tests as well as the high-level calls: an
//! entropy coder ([`h264::cabac`], [`hevc::cabac`], [`vp9::bool_coder`],
//! [`decode::av1::msac`]), a transform ([`hevc::transform`],
//! [`decode::av1::itx`]), a loop filter ([`h264::deblock`],
//! [`decode::vp9::lpf`]), a parameter-set parser ([`decode::h264::params`],
//! [`decode::hevc::params`]), an MDCT ([`aac::dsp`], [`opus::celt`]). Each
//! module's docs say which section of which specification it implements.
//! These inner modules follow the structure of their specifications rather
//! than a stability promise; the high-level types re-exported from each
//! codec's root and from [`prelude`] are the stable surface.
//!
//! # Hardware acceleration
//!
//! [`hw::Mp4Encoder`] and [`decode::Decoder`] use VideoToolbox on macOS (the
//! `videotoolbox` feature), an NVIDIA GPU's NVENC and NVDEC engines on
//! Linux and Windows (the `nvidia` feature), and VA drivers — AMD, Intel —
//! on Linux (the `vaapi` feature), and fall back to the
//! pure-Rust codecs when the feature is off, the machine has no such
//! hardware, or the hardware refuses a stream. The NVIDIA and VA libraries
//! are loaded at run time, so a binary built with those features still runs,
//! natively, where they are missing.
//! The software paths use every core with `parallel`: frames of an all-intra
//! stream are independent, so batches of them encode at once with
//! byte-identical output (see [`set_frame_memory_budget`]).
//!
//! # Features
//!
//! | Feature | Default | What it adds |
//! |---------|---------|--------------|
//! | `decode` | **on** | The video decoders in [`decode`] (H.264, HEVC, VP9, AV1) |
//! | `gif` | **on** | GIF89a encode and decode ([`gif`]), and GIF in [`animation`] |
//! | `parallel` | | Frame-parallel encoding on `rayon` |
//! | `videotoolbox` | | VideoToolbox encode and decode on macOS (links Apple's frameworks; inert elsewhere) |
//! | `nvidia` | | NVENC encode and NVDEC decode on Linux and Windows (driver loaded at run time; implies `decode`; inert elsewhere) |
//! | `vaapi` | | VAAPI HEVC encode and H.264/HEVC/VP9/AV1 decode on Linux (libva loaded at run time; implies `decode`; inert elsewhere) |
//!
//! Encoders, audio decoders, muxers, demuxers and captions are always
//! compiled; none of them has a dependency to make optional.
// The codecs follow their specifications' notation: loops over indices that
// address several parallel arrays, argument lists that mirror the spec's
// processes, and the reference decoders' own nested types.
pub use ;
pub use ApngEncoder;
pub use ;
pub use ;
pub use DecodedFrame;
pub use ;
pub use ;
// --------------------------------------------------------------------------
// Frame-level parallelism
// --------------------------------------------------------------------------
/// How many frames to hold in flight at once.
///
/// Every frame of an all-intra stream is independent, so the only thing bounding
/// the batch is memory: an 8K frame is 50 MB of input and the encoder allocates
/// a reconstruction of its own alongside it. Encoding every frame at once would
/// undo the streaming work that got export memory down in the first place, so
/// the batch is whichever is smaller — the thread count, or what fits in a
/// fixed working-set budget.
pub
/// How many frames the memory budget allows in flight, irrespective of how many
/// cores there are.
///
/// Rate control sizes its groups from this rather than from
/// [`frames_in_flight`], because the group is how often the controller gets to
/// correct itself — and a clip must not encode to a different size on a machine
/// with more cores.
pub
/// Roughly what one frame costs while it is being encoded: the input planes,
/// the reconstruction, and the padded copies the encoder makes of both.
pub
/// Default working-set budget for frame-parallel encoding.
///
/// Deliberately conservative: it binds only at 4K and above, and only there
/// because a frame is large enough that holding a core's worth of them is real
/// memory. See [`set_frame_memory_budget`].
pub const DEFAULT_FRAME_MEMORY_BUDGET: usize = 256 << 20;
static FRAME_MEMORY_BUDGET: AtomicUsize =
new;
/// How much memory frame-parallel encoding may hold in flight.
/// Set how much memory frame-parallel encoding may hold in flight.
///
/// Frames of an all-intra stream are independent, so the only thing limiting how
/// many encode at once is memory — about 4.5 bytes per pixel per frame in
/// flight, which is 149 MB at 8K. The default of 256 MB keeps peak usage close
/// to where the streaming work left it, at the cost of capping 8K to two frames
/// at a time.
///
/// Raising it is the cheapest speedup available at high resolution and it costs
/// nothing in bitrate: measured on a 4-frame 8K clip, going from 256 MB to 1 GB
/// took the encode from 49.9 s to 33.0 s with byte-identical output. Tiling the
/// picture would reach the same parallelism at a bitrate cost; this does not.
///
/// ```
/// // Let an 8K export use eight frames at a time.
/// threers_codec::set_frame_memory_budget(1024 << 20);
/// ```
/// Encode a batch of frames, in order, using every available thread.
///
/// The output is byte-identical to mapping `encode` over the frames one at a
/// time — the `parallel` feature changes how long it takes and nothing else.
pub
pub
/// `map_frames`, public so benchmarks and callers with their own frame source
/// can use the same batching the export paths do.
/// [`map_frames`] where the work needs to know its own index — tiles do, to
/// tell the last one from the rest.
pub
pub