Expand description
Audio intelligence + call recording (PROCESSOR-DESIGN §6.1).
This module is the home for the audio-intelligence processors that sit on
the hot media path inside the cascaded pipeline — VAD, end-of-turn, noise
filters, and mixers — alongside the existing call recorder. The analyzer
traits (VadAnalyzer/TurnAnalyzer) are frozen in crate::service; the
ONNX (ort) impls compile only under the vad-ort feature so flowcat-core
stays dependency-light by default.
Module map:
vad—VadAnalyzerimpls (Silero behindvad-ort, a mock always).turn—TurnAnalyzerimpls (Smart-Turn behindvad-ort, a mock always).strategy— turn start/stop/mute + interruption strategies (pure logic).filter— RNNoise/Koala audio-filter processors (Krisp deferred).mixer— audio mixers.
§Call recording (existing)
Tap both legs (caller + bot), mono-mix, render to WAV bytes.
The recorder accumulates audio from each leg and produces a single mono WAV
(via hound) suitable for upload to the object store (see DESIGN.md “Audio path”).
The two legs typically arrive at different sample rates (caller ≈ 8 kHz μ-law
decoded, bot ≈ 24 kHz from Gemini Live), so each leg is buffered with its own
rate and resampled to the recorder’s common sample_rate only at render time.
Mixing is a plain per-sample sum clamped back into i16.
Re-exports§
pub use filter::AudioFilter;pub use filter::AudioFilterProcessor;pub use filter::IdentityFilter;pub use mixer::AudioMixer;pub use mixer::MixerProcessor;pub use mixer::SilenceMixer;pub use mixer::SoundfileMixer;pub use strategy::AlwaysInterrupt;pub use strategy::AlwaysMute;pub use strategy::ExternalTurnStart;pub use strategy::FirstSpeechMute;pub use strategy::FunctionCallMute;pub use strategy::InterruptionStrategy;pub use strategy::MinWordsInterrupt;pub use strategy::MinWordsTurnStart;pub use strategy::MuteStrategy;pub use strategy::MuteUntilFirstBotComplete;pub use strategy::SpeechTimeoutStop;pub use strategy::TurnAnalyzerStop;pub use strategy::TurnStartDecision;pub use strategy::TurnStartStrategy;pub use strategy::TurnStopDecision;pub use strategy::TurnStopStrategy;pub use strategy::VadTurnStart;pub use turn::MockTurn;pub use turn::TurnProcessor;pub use turn::TurnSilenceTracker;pub use vad::window_volume;pub use vad::MockVad;pub use vad::VadProcessor;pub use vad::VadStateMachine;
Modules§
- filter
- Input-leg audio-filter processors (noise suppression).
- mixer
- Output-leg audio mixers.
- strategy
- Turn start/stop/mute + interruption strategies (pure framework logic).
- turn
- End-of-turn / semantic-completion analyzer impls + the
TurnProcessor. - vad
- Voice-activity-detector impls + the
VadProcessor.
Structs§
- Audio
Recorder - Accumulates inbound (caller) and outbound (bot) audio on a shared wall-clock timeline and mixes them down to a single mono WAV byte buffer.