bunsen 0.40.0

bunsen is a batteries included common library for burn
Documentation
//! # Whisper
//!
//! `OpenAI`'s Whisper speech-recognition model as a burn module, with what
//! it takes to use one: an index of pretrained checkpoints, a decoder, and
//! a stream driver that turns audio pushed in chunks into timed transcript
//! segments.
//!
//! - [`blocks`]: the model, [`Whisper`], and its configs ([`WhisperApiConfig`]
//!   describes it as upstream's `ModelDimensions` does; [`WhisperGeometry`] is
//!   the part a checkpoint fixes).
//! - [`pretrained`]: names to loaded models.
//!   [`default_whisper_factory`](pretrained::default_whisper_factory) is the
//!   index: `openai/base`, `large`, `well-known:openai/tiny.en`, and
//!   `hf:openai/whisper-large-v3` for a Hugging Face repo. It resolves a name
//!   to a [`WhisperBundle`](driver::WhisperBundle): the model at the precision
//!   it ships in, its token layout, and its vocabulary.
//! - [`decode`]: one window of audio to tokens, with upstream's temperature
//!   fallback.
//! - [`logit_filters`]: what the decode may and may not emit.
//! - [`driver`]: [`WhisperStreamDriver`](driver::WhisperStreamDriver) over a
//!   bundle, and the [context](driver::WhisperStreamContext) it opens on a
//!   stream: samples in,
//!   [`TranscriptEvent`](crate::kits::speech::asr_driver::TranscriptEvent)s
//!   out.
//!
//! # Lifecycle
//!
//! 1. A [`PretrainedCache`](crate::data::pretrained::PretrainedCache): where
//!    checkpoints and vocabularies land, pinned by digest.
//! 2. The factory,
//!    [`default_whisper_factory`](pretrained::default_whisper_factory): a name
//!    to a row of the index, and the kit's hook to build it.
//! 3. A [`WhisperBundle`](driver::WhisperBundle) behind an `Arc`, from the
//!    factory's [`load_bundle`](crate::data::pretrained::PretrainedFactory::load_bundle):
//!    the model, its token layout and its vocabulary.
//! 4. A [`WhisperStreamDriverConfig`](driver::WhisperStreamDriverConfig):
//!    language, task, timestamps, search, and the
//!    [`EmissionPolicy`](crate::kits::speech::asr_driver::EmissionPolicy).
//! 5. A [`WhisperStreamDriver`](driver::WhisperStreamDriver) over the bundle,
//!    from [`init_from_bundle`](driver::WhisperStreamDriverConfig::init_from_bundle):
//!    shared, and never mutated.
//! 6. A [`WhisperStreamContext`](driver::WhisperStreamContext) per stream, from
//!    [`new_context`](driver::WhisperStreamDriver::new_context): all of that
//!    stream's state.
//! 7. [`TranscriptEvent`](crate::kits::speech::asr_driver::TranscriptEvent)s,
//!    from [`write_read`](driver::WhisperStreamContext::write_read) as samples
//!    arrive and [`end_read`](driver::WhisperStreamContext::end_read) at the
//!    end.
//!
//! Stream state is injected, never owned by the model, so one driver over
//! one loaded model serves several streams in one process; the [`driver`]
//! docs explain the split, the emission rules, the stream clock, the clamp
//! policy and the voice-activity regions.
//!
//! # Example
//!
//! The default pathway, as `bunsen-asr family whisper` walks it, over any
//! backend: a cache, the factory, a bundle by name, a driver over it, and a
//! context fed a stream. The first run fetches `openai/base` (145 MB,
//! digest-checked) and its vocabulary into the cache; later runs find them
//! there.
//!
//! ```rust,no_run
//! # #[cfg(all(feature = "store_pytorch", feature = "cache"))] {
//! use bunsen::{
//!     data::pretrained::{
//!         PretrainedCache,
//!         PretrainedCacheOptions,
//!     },
//!     errors::BunsenResult,
//!     kits::speech::{
//!         asr_driver::{
//!             LevelPolicy,
//!             StreamClock,
//!             TranscriptSegment,
//!         },
//!         whisper::{
//!             driver::{
//!                 WhisperStreamDriver,
//!                 WhisperStreamDriverConfig,
//!             },
//!             pretrained::default_whisper_factory,
//!         },
//!     },
//! };
//! use burn::tensor::Device;
//!
//! /// The named model as a driver on `device`, with the defaults: the
//! /// language detected from the first window, a transcription, greedy,
//! /// and the offline emission policy, which decodes whole windows as
//! /// they fill and commits each once.
//! fn load_driver(
//!     name: &str,
//!     device: &Device,
//! ) -> BunsenResult<WhisperStreamDriver> {
//!     let cache = PretrainedCache::new(PretrainedCacheOptions::default())?;
//!
//!     // A name to a bundle: the checkpoint read through the reader its
//!     // row names, checked against the geometry the name promises, and
//!     // the vocabulary its token layout selects.
//!     let bundle =
//!         default_whisper_factory()?.load_bundle(name, &cache, device)?;
//!
//!     WhisperStreamDriverConfig::new().init_from_bundle(bundle, device)
//! }
//!
//! /// One stream through the driver: mono samples at the model's rate
//! /// (16 kHz), pushed as a live loop would feed them; the segments as
//! /// they became final.
//! fn transcribe(
//!     driver: &WhisperStreamDriver,
//!     samples: &[f32],
//! ) -> BunsenResult<Vec<TranscriptSegment>> {
//!     // A bare stream: a clock from zero at the model's rate, and the
//!     // running level (the loudest heard so far) as each window's
//!     // dynamic-range reference.
//!     let mut ctx = driver.new_context(
//!         StreamClock::uniform(driver.sample_rate()),
//!         LevelPolicy::Running,
//!     )?;
//!
//!     let mut segments = Vec::new();
//!     for block in samples.chunks(driver.sample_rate() / 10) {
//!         for event in ctx.write_read(block)? {
//!             segments.push(event.segment().clone());
//!         }
//!     }
//!     // The end of the stream: whatever is pending is decoded and
//!     // committed.
//!     for event in ctx.end_read()? {
//!         segments.push(event.segment().clone());
//!     }
//!     Ok(segments)
//! }
//!
//! fn run(
//!     device: &Device,
//!     samples: &[f32],
//! ) -> BunsenResult<()> {
//!     let driver = load_driver("openai/base", device)?;
//!     for segment in transcribe(&driver, samples)? {
//!         println!(
//!             "[{:7.2} --> {:7.2}] {}",
//!             segment.start,
//!             segment.end,
//!             segment.text.as_deref().unwrap_or("")
//!         );
//!     }
//!     Ok(())
//! }
//! # }
//! ```
//!
//! # Other ways in
//!
//! The same driver serves every other way to a bundle. A checkpoint on disk
//! is a given map, with the same hook the factory would attach. A Hugging
//! Face repo is a ref of the `hf` provider, resolved through the cache (the
//! hub's file listing is fetched once and kept). A vocabulary of one's own
//! is an overlay on the resolved model. And the real-time emission presets
//! decode on speech endpoints, so they want a voice-activity model, which
//! the Silero kit's factory provides the same way; `bunsen-asr` attaches
//! it with [`with_vad`](driver::WhisperStreamDriver::with_vad) as
//! here.
//!
//! ```rust,no_run
//! # #[cfg(all(feature = "store_pytorch", feature = "cache"))] {
//! use std::{
//!     path::Path,
//!     sync::Arc,
//! };
//!
//! use bunsen::{
//!     data::pretrained::{
//!         Deferred,
//!         PretrainedCache,
//!         ResourceMap,
//!     },
//!     errors::BunsenResult,
//!     kits::speech::{
//!         asr_driver::{
//!             EmissionPolicy,
//!             VoiceActivityFilterConfig,
//!         },
//!         silero_vad::pretrained::default_silero_factory,
//!         whisper::{
//!             driver::{
//!                 WhisperBundle,
//!                 WhisperStreamDriver,
//!                 WhisperStreamDriverConfig,
//!             },
//!             pretrained::{
//!                 CHECKPOINT,
//!                 VOCABULARY,
//!                 WhisperConstruct,
//!                 default_whisper_factory,
//!             },
//!         },
//!     },
//! };
//! use burn::tensor::Device;
//!
//! /// A checkpoint on disk: a given map, read by the reader its file
//! /// calls for, with the vocabulary its layout selects.
//! fn load_from_path(
//!     path: &Path,
//!     cache: &PretrainedCache,
//!     device: &Device,
//! ) -> BunsenResult<Arc<WhisperBundle>> {
//!     Deferred::<WhisperConstruct>::from_map(ResourceMap::given(
//!         "mine", CHECKPOINT, path,
//!     ))?
//!     .load_bundle(cache, device)
//! }
//!
//! /// A Hugging Face repo: `transformers`' safetensors, one file or shards.
//! fn load_from_hub(
//!     cache: &PretrainedCache,
//!     device: &Device,
//! ) -> BunsenResult<Arc<WhisperBundle>> {
//!     default_whisper_factory()?.load_bundle(
//!         "hf:openai/whisper-large-v3",
//!         cache,
//!         device,
//!     )
//! }
//!
//! /// A named model with a vocabulary file of one's own over its row's.
//! fn load_with_vocabulary(
//!     vocabulary: &Path,
//!     cache: &PretrainedCache,
//!     device: &Device,
//! ) -> BunsenResult<Arc<WhisperBundle>> {
//!     default_whisper_factory()?
//!         .resolve("openai/tiny.en", cache)?
//!         .with_overlay(ResourceMap::given(
//!             "--vocab", VOCABULARY, vocabulary,
//!         ))?
//!         .load_bundle(cache, device)
//! }
//!
//! /// Conservative real time: decodes on speech endpoints as well as full
//! /// windows, with upstream's 16 kHz Silero model as the voice-activity
//! /// gate.
//! fn conservative_driver(
//!     bundle: Arc<WhisperBundle>,
//!     cache: &PretrainedCache,
//!     device: &Device,
//! ) -> BunsenResult<WhisperStreamDriver> {
//!     let vad = default_silero_factory()?
//!         .load("silero/vad-16k", cache, device)?
//!         .handle;
//!     // The 16 kHz branch, and the filter's 16 kHz defaults: the model's
//!     // rate.
//!     WhisperStreamDriverConfig::new()
//!         .with_emission(EmissionPolicy::conservative())
//!         .init_from_bundle(bundle, device)?
//!         .with_vad(
//!             Arc::new(vad.expect_branch(16_000).clone()),
//!             VoiceActivityFilterConfig::default(),
//!         )
//! }
//! # }
//! ```

pub mod blocks;
pub mod decode;
pub mod driver;
pub mod logit_filters;
pub mod pretrained;

#[doc(inline)]
pub use blocks::{
    Whisper,
    WhisperApiConfig,
    WhisperGeometry,
    WhisperMeta,
    WhisperStructureConfig,
};
#[doc(inline)]
pub use decode::*;