1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
//! Pure-Rust media codecs: video and audio encoders, decoders, muxers and
//! demuxers, with hardware acceleration where the platform has it and no C
//! dependencies anywhere else.
//!
//! Everything builds for `wasm32` as well as native and depends on nothing
//! beyond `std` (`rayon` with `parallel`). Outside the opt-in hardware
//! modules (`videotoolbox`, `nvidia`, `vaapi`) there are no OS, process or
//! filesystem calls: encoders return `Vec<u8>`, and the caller owns I/O. It
//! is the media half of [threers](https://crates.io/crates/threers), which
//! re-exports it as `threers::codec`, split out so programs that only encode
//! or decode media do not link a renderer.
//!
//! # Start here
//!
//! [`prelude`] has the types most programs need. Three entry points cover
//! the common jobs:
//!
//! ```no_run
//! # #[cfg(feature = "decode")] {
//! use threers_codec::prelude::*;
//!
//! // Encode RGBA frames to MP4 — on the hardware encoder where there is one.
//! let mut enc = Mp4Encoder::new(640, 360, EncodeOptions::default()).unwrap();
//! enc.push_rgba(&vec![128; 640 * 360 * 4]).unwrap();
//! std::fs::write("out.mp4", enc.finish().unwrap()).unwrap();
//!
//! // Decode any audio file this crate reads to planar f32.
//! let pcm = audio_file::read(&std::fs::read("song.flac").unwrap()).unwrap();
//! println!("{} channels at {} Hz", pcm.channels.len(), pcm.sample_rate);
//!
//! // Decode a video track (feature `decode`), again hardware first.
//! let file = std::fs::read("clip.mp4").unwrap();
//! let track = demux_video(&file).unwrap();
//! let mut dec = Decoder::new(&track).unwrap();
//! for s in &track.samples {
//! dec.push(&file[s.offset as usize..][..s.size as usize], s.pts, s.dts, s.duration as i64)
//! .unwrap();
//! if let Some(_bgra) = dec.take(s.pts) {}
//! }
//! # }
//! ```
//!
//! # What is here
//!
//! | Kind | Modules |
//! |------|---------|
//! | Video encoders | [`h264`] (CAVLC/CABAC, intra + inter), [`hevc`] (intra, alpha), [`vp9`] (intra + inter, alpha) |
//! | Video decoders | [`decode`]: H.264, HEVC, VP9, AV1 — bit-exact against the reference decoders (feature `decode`) |
//! | Audio encoders | [`aac::encoder`] (AAC-LC to raw units or ADTS, 8 to 96 kHz, mono and stereo) and [`aac::m4a`] (the same in an `.m4a`, with the priming edit list); [`mp3::encoder`] (MPEG-1 Layer III, 32 to 320 kbit/s, with a gapless `Info` frame) |
//! | Audio decoders | [`aac`] (LC, HE-AAC SBR + PS), [`mp3`], [`opus`] (SILK, CELT, hybrid), [`vorbis`], [`flac`], [`pcm`] (WAV/RF64, AIFF); [`audio_file`] sniffs and dispatches |
//! | Animated images | [`gif`] (encode and decode, feature `gif`), [`apng`]; [`animation`] picks one for a browser |
//! | Containers | [`mp4`] / [`mp4_stream`] (write), [`mp4_demux`] (read), [`webm`] (write), [`mkv_demux`] (read, plus IVF), [`ogg`] (read) |
//! | Hardware | [`hw`]: one encoder and one decoder over VideoToolbox, NVENC/NVDEC, VAAPI and the native codecs; `videotoolbox`, `videotoolbox_decode` (macOS), `nvidia` (Linux, Windows) and `vaapi` (Linux) directly |
//! | Rate control | [`rate`]: a quantizer or a bitrate target |
//! | Conversion | [`swr`] (resampling and sample formats), [`ffcompat`] (`swscale`, `.mov`/`.mkv` readers, the shapes ffmpeg's tools produce) |
//! | Captions | [`captions`]: tracks and cues, SRT and WebVTT, muxed by `mp4` and `webm` as soft subtitles |
//! | Primitives | [`bitstream`] (bit writer, Exp-Golomb, emulation prevention), [`contrast`] (WCAG luminance) |
//!
//! # Deep access
//!
//! Every stage of every codec is a public module, so the building blocks are
//! there for research, tooling and tests as well as the high-level calls: an
//! entropy coder ([`h264::cabac`], [`hevc::cabac`], [`vp9::bool_coder`],
//! [`decode::av1::msac`]), a transform ([`hevc::transform`],
//! [`decode::av1::itx`]), a loop filter ([`h264::deblock`],
//! [`decode::vp9::lpf`]), a parameter-set parser ([`decode::h264::params`],
//! [`decode::hevc::params`]), an MDCT ([`aac::dsp`], [`opus::celt`]). Each
//! module's docs say which section of which specification it implements.
//! These inner modules follow the structure of their specifications rather
//! than a stability promise; the high-level types re-exported from each
//! codec's root and from [`prelude`] are the stable surface.
//!
//! # Hardware acceleration
//!
//! [`hw::Mp4Encoder`] and [`decode::Decoder`] use VideoToolbox on macOS (the
//! `videotoolbox` feature), an NVIDIA GPU's NVENC and NVDEC engines on
//! Linux and Windows (the `nvidia` feature), and VA drivers — AMD, Intel —
//! on Linux (the `vaapi` feature), and fall back to the
//! pure-Rust codecs when the feature is off, the machine has no such
//! hardware, or the hardware refuses a stream. The NVIDIA and VA libraries
//! are loaded at run time, so a binary built with those features still runs,
//! natively, where they are missing.
//! The software paths use every core with `parallel`: frames of an all-intra
//! stream are independent, so batches of them encode at once with
//! byte-identical output (see [`set_frame_memory_budget`]).
//!
//! # Features
//!
//! | Feature | Default | What it adds |
//! |---------|---------|--------------|
//! | `decode` | **on** | The video decoders in [`decode`] (H.264, HEVC, VP9, AV1) |
//! | `gif` | **on** | GIF89a encode and decode ([`gif`]), and GIF in [`animation`] |
//! | `parallel` | | Frame-parallel encoding, and AV1 decoding across frames, tiles and loop filters, on `rayon` |
//! | `videotoolbox` | | VideoToolbox encode and decode on macOS (links Apple's frameworks; inert elsewhere) |
//! | `nvidia` | | NVENC encode and NVDEC decode on Linux and Windows (driver loaded at run time; implies `decode`; inert elsewhere) |
//! | `vaapi` | | VAAPI HEVC encode and H.264/HEVC/VP9/AV1 decode on Linux (libva loaded at run time; implies `decode`; inert elsewhere) |
//!
//! Encoders, audio decoders, muxers, demuxers and captions are always
//! compiled; none of them has a dependency to make optional.
// The codecs follow their specifications' notation: loops over indices that
// address several parallel arrays, argument lists that mirror the spec's
// processes, and the reference decoders' own nested types.
/// The caption data model, from [`threers_captions`].
pub use threers_captions as captions;
/// WCAG luminance and contrast, from [`threers_captions`].
pub use contrast;
// The decoder hands pictures over in `decode`'s types, so it needs `decode` too.
pub use ;
pub use ApngEncoder;
pub use ;
pub use DecodedFrame;
pub use ;
pub use ;
pub use ;
// --------------------------------------------------------------------------
// Frame-level parallelism
// --------------------------------------------------------------------------
/// How many frames to hold in flight at once.
///
/// Every frame of an all-intra stream is independent, so the only thing bounding
/// the batch is memory: an 8K frame is 50 MB of input and the encoder allocates
/// a reconstruction of its own alongside it. Encoding every frame at once would
/// undo the streaming work that got export memory down in the first place, so
/// the batch is whichever is smaller — the thread count, or what fits in a
/// fixed working-set budget.
pub
/// How many frames the memory budget allows in flight, irrespective of how many
/// cores there are.
///
/// Rate control sizes its groups from this rather than from
/// [`frames_in_flight`], because the group is how often the controller gets to
/// correct itself — and a clip must not encode to a different size on a machine
/// with more cores.
pub
/// Roughly what one frame costs while it is being encoded: the input planes,
/// the reconstruction, and the padded copies the encoder makes of both.
pub
/// Default working-set budget for frame-parallel encoding.
///
/// Deliberately conservative: it binds only at 4K and above, and only there
/// because a frame is large enough that holding a core's worth of them is real
/// memory. See [`set_frame_memory_budget`].
pub const DEFAULT_FRAME_MEMORY_BUDGET: usize = 256 << 20;
static FRAME_MEMORY_BUDGET: AtomicUsize =
new;
/// How much memory frame-parallel encoding may hold in flight.
/// Set how much memory frame-parallel encoding may hold in flight.
///
/// Frames of an all-intra stream are independent, so the only thing limiting how
/// many encode at once is memory — about 4.5 bytes per pixel per frame in
/// flight, which is 149 MB at 8K. The default of 256 MB keeps peak usage close
/// to where the streaming work left it, at the cost of capping 8K to two frames
/// at a time.
///
/// Raising it is the cheapest speedup available at high resolution and it costs
/// nothing in bitrate: measured on a 4-frame 8K clip, going from 256 MB to 1 GB
/// took the encode from 49.9 s to 33.0 s with byte-identical output. Tiling the
/// picture would reach the same parallelism at a bitrate cost; this does not.
///
/// ```
/// // Let an 8K export use eight frames at a time.
/// threers_codec::set_frame_memory_budget(1024 << 20);
/// ```
/// Encode a batch of frames, in order, using every available thread.
///
/// The output is byte-identical to mapping `encode` over the frames one at a
/// time — the `parallel` feature changes how long it takes and nothing else.
pub
pub
/// `map_frames`, public so benchmarks and callers with their own frame source
/// can use the same batching the export paths do.
/// [`map_frames`] where the work needs to know its own index — tiles do, to
/// tell the last one from the rest.
pub
pub
/// Run `f` on each of `items`, on every available thread with `parallel`.
/// Callers give each item its own disjoint piece of the output (a band of
/// rows, say), so the result is the same whatever the schedule.
///
/// On wasm this needs a build with `+atomics` and a thread pool the page has
/// started (`wasm-bindgen-rayon`); without atomics it runs in order, as a
/// plain wasm32 build has no threads to spawn.
pub
/// Run `f` on the thread pool, returning at once.
pub
pub
/// Whether the caller can block while the pool does work for it: there must
/// be a pool, the caller must not be one of its threads, and with one thread
/// there would be nothing to gain.
pub
pub
/// How many frames to decode at once by default.
pub
pub
/// Whether [`for_each_owned`] runs its items on more than one thread, for
/// callers whose parallel path has a setup cost worth skipping otherwise.
pub
pub