1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
//! # Silero VAD
//!
//! The Silero voice-activity detector as a burn module: a small recurrent
//! model that gives, for each chunk of audio (512 samples at 16 kHz, 256
//! at 8 kHz), the probability that it holds speech, carrying its state
//! from one chunk to the next.
//!
//! - [`blocks`]: the model, [`SileroVad`], for one sample rate; the two-rate
//! [`SileroVadCollection`] a checkpoint holds; and the [`SileroVadContext`]
//! that carries what a stream needs between chunks (the tail of the last
//! chunk and the recurrent state). The model holds no stream state, so one
//! loaded model serves any number of streams, a context each.
//! - [`pretrained`] (features `store_burnpack` and `cache`, both default): how
//! the weights arrive.
//! [`default_silero_factory`](pretrained::default_silero_factory) is the
//! index: with the `silero-weights` feature, one row, `bundled:silero/vad`,
//! the burnpack linked into the binary and written into the cache under its
//! digest on first use. The loaders in [`pretrained::load`] read the same
//! bytes with no cache at all, for a binary that wants nothing else.
//!
//! The ONNX reference this was transliterated from, and the cross-checks
//! against it, live in the `silero-model-validation` crate. There is
//! *some* bug in the burn CUDA backend, which we see by model divergence
//! from the golden tests on that backend alone.
//!
//! # Example
//!
//! The default pathway, over any backend: a cache, the factory, the
//! collection by name, the branch for the stream's rate, and a context fed
//! chunk by chunk.
//!
//! ```rust,no_run
//! # #[cfg(all(feature = "store_burnpack", feature = "cache"))] {
//! use std::sync::Arc;
//!
//! use bunsen::{
//! data::pretrained::{
//! PretrainedCache,
//! PretrainedCacheOptions,
//! },
//! errors::BunsenResult,
//! kits::speech::silero_vad::{
//! SileroVad,
//! SileroVadCollection,
//! SileroVadContextConfig,
//! SileroVadMeta,
//! pretrained::default_silero_factory,
//! },
//! };
//! use burn::{
//! prelude::Backend,
//! tensor::{
//! ElementConversion,
//! Tensor,
//! },
//! };
//!
//! /// The bundled model, both rates, on `device`: from the cache, or
//! /// written into it from the binary on first use.
//! fn load_vad<B: Backend>(
//! device: &B::Device
//! ) -> BunsenResult<Arc<SileroVadCollection<B>>> {
//! let cache = PretrainedCache::new(PretrainedCacheOptions::default())?;
//! Ok(default_silero_factory()?
//! .load::<B>("bundled:silero/vad", &cache, device)?
//! .handle)
//! }
//!
//! /// One stream through one branch: mono samples at the branch's rate,
//! /// a chunk at a time, each answered with the probability it holds
//! /// speech.
//! fn speech_probabilities<B: Backend>(
//! vad: &SileroVad<B>,
//! samples: &[f32],
//! device: &B::Device,
//! ) -> Vec<f32> {
//! // One stream at the branch's rate, and the tail the model looks back
//! // over: 64 samples at 16 kHz, 32 at 8 kHz.
//! let mut ctx =
//! SileroVadContextConfig::new(vad.sample_rate()).init(vad, device);
//! let mut probabilities = Vec::new();
//! for chunk in samples.chunks_exact(vad.chunk_size()) {
//! let chunk: Tensor<B, 2> =
//! Tensor::<B, 1>::from_floats(chunk, device).unsqueeze();
//! let (probability, next) = vad.context_forward(chunk, ctx);
//! ctx = next;
//! probabilities.push(probability.into_scalar().elem::<f32>());
//! }
//! probabilities
//! }
//!
//! fn run<B: Backend>(
//! device: &B::Device,
//! samples: &[f32],
//! ) -> BunsenResult<()> {
//! let vad = load_vad::<B>(device)?;
//! let branch = vad.expect_branch(16000);
//! for (i, p) in speech_probabilities(branch, samples, device)
//! .iter()
//! .enumerate()
//! {
//! let at = i * branch.chunk_size();
//! println!("{at:>8}: {p:.3}");
//! }
//! Ok(())
//! }
//! # }
//! ```
//!
//! The Whisper stream driver's real-time emission presets take the model
//! the same way, and turn its probabilities into speech regions through a
//! [`VoiceActivityFilterConfig`](crate::kits::speech::whisper::driver::VoiceActivityFilterConfig),
//! attached with
//! [`with_vad`](crate::kits::speech::whisper::driver::WhisperStreamDriver::with_vad):
//! the 16 kHz branch, cloned out of the collection. The Whisper kit's
//! [other ways in](crate::kits::speech::whisper#other-ways-in) show it
//! whole.
pub use *;