rustvani 0.2.2

Voice AI framework for Rust — real-time speech pipelines with STT, LLM, TTS, and Dhara conversation flows
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107

# Voice Activity Detection (VAD)

**Files:** `src/vad/silero_native.rs`, `src/vad/silero_ort.rs`  
**Feature:** `vad-silero` (enabled by default)

Two backends for Silero VAD with the same API. Detects speech start/stop and drives the pipeline via `VADUserStartedSpeaking` / `VADUserStoppedSpeaking` frames.

## Backends

| Backend | Struct | Sample Rates | ONNX Runtime | Memory |
|---|---|---|---|---|
| Native | `SileroVadNative` | 16 kHz only | No | ~5 MB |
| ONNX Runtime | `SileroVadOrt` | 8 kHz + 16 kHz | Yes | Larger |

## Usage

### Native (Zero Dependencies)

```rust
use rustvani::vad::{SileroVadNative, VadBackend, create_vad};

let vad = SileroVadNative::new(16_000).expect("VAD load failed");

// Or use the factory:
let vad = create_vad(VadBackend::Native, 16_000).unwrap();
```

### ONNX Runtime

```rust
use rustvani::vad::{SileroVadOrt, VadBackend, create_vad};

let vad = SileroVadOrt::new(16_000).expect("VAD load failed");

// Or use the factory:
let vad = create_vad(VadBackend::Ort, 16_000).unwrap();
```

### In Transport

```rust
use rustvani::transport::TransportParams;

let transport = BaseTransport::new(TransportParams {
    audio_in_enabled: true,
    audio_in_sample_rate: Some(16_000),
    vad_analyzer: Some(Arc::new(vad)),
    ..Default::default()
});
```

## Configuration

`VadParams` (used by both backends):

| Field | Default | Description |
|---|---|---|
| `confidence` | `0.5` | Speech probability threshold |
| `start_secs` | `0.2` | Duration of speech before `StartedSpeaking` |
| `stop_secs` | `0.8` | Duration of silence before `StoppedSpeaking` |
| `min_volume` | `-45.0` | dFS minimum volume gate |

```rust
use rustvani::vad::VadParams;

let params = VadParams {
    confidence: 0.6,
    start_secs: 0.15,
    stop_secs: 0.5,
    ..Default::default()
};
```

## State Machine

```
Quiet → Starting → Speaking → Stopping → Quiet
```

## SmartTurn (End-of-Turn Prediction)

An optional local ONNX model defers `VADUserStoppedSpeaking` on hesitation pauses. Enable via the turn engine:

```rust
use rustvani::turn::engine::TurnEngine;
```

See `src/turn/` for details.

## Client + Server VAD Coordination

When using the Dioxus frontend integration, the browser runs its own lightweight VAD and pushes `ClientVADUserStartedSpeaking` events into the server pipeline. A CAS toggle ensures exactly one `VADUserStartedSpeaking` is emitted per utterance regardless of which side fires first.

```rust
// Called from your WebSocket handler
transport.push_client_vad_started(&processor, timestamp).await;
transport.push_client_vad_stopped(&processor, timestamp).await;
```

## Cargo Feature

Enabled by default. To disable:

```toml
[dependencies]
rustvani = { version = "0.2", default-features = false, features = ["transport-websocket", "stt-sarvam", "stt-60db", "llm-openai", "tts-deepgram", "tts-sarvam", "dhara"] }
```