gradbot

Core library for building voice AI applications with real-time speech-to-text, LLM processing, and text-to-speech.

Architecture

The library implements a multiplexing loop that coordinates:

• Speech-to-Text (STT): Converts incoming audio to text transcriptions
• LLM: Processes transcribed text and generates responses
• Text-to-Speech (TTS): Converts LLM responses back to audio

Quick Start

use gradbot::{GradbotClients, MsgOut, SessionConfig};

// 1. Create clients (uses environment variables for API keys)
let clients = GradbotClients::new(None, None, None, None, None, None).await?;

// 2. Start a session
let (input, mut output) = clients.start_session(Some(config), Format::OggOpus).await?;

// 3. Spawn a producer to send audio
tokio::spawn(async move {
    input.send_audio(audio_bytes).await.ok();
    // input is dropped when done -> session ends normally
});

// 4. Consume output messages
while let Some(msg) = output.receive().await? {
    match msg {
        MsgOut::Audio { data, .. } => { /* send audio */ }
        MsgOut::TtsText { text, .. } => { /* send caption */ }
        MsgOut::SttText { text, .. } => { /* send transcription */ }
        MsgOut::Event { event, .. } => { /* handle event */ }
    }
}

Environment Variables

• GRADIUM_API_KEY - API key for Gradium STT/TTS services
• GRADIUM_BASE_URL - Base URL for Gradium services (optional, defaults to https://api.gradium.ai)
• LLM_API_KEY - API key for OpenAI-compatible LLM API (falls back to OPENAI_API_KEY)
• LLM_BASE_URL - Base URL for LLM API (optional, defaults to OpenAI's API)
• LLM_MODEL - LLM model name (optional, auto-detected if single model available)

Channel Semantics

• Normal termination: When SessionInputHandle is dropped (client disconnects), output.receive() returns Ok(None).
• Internal error: On any processing error, output.receive() returns Err(e).

Message Flow

┌─────────────┐     MsgIn::Audio      ┌─────────────┐
│  Producer   │ ──────────────────────▶│             │
│   (your     │     MsgIn::Config      │   session   │
│    loop)    │ ──────────────────────▶│   future    │
└─────────────┘                        │             │
                                       │             │
┌─────────────┐     MsgOut::Audio      │             │
│  Consumer   │ ◀──────────────────────│             │
│   (your     │     MsgOut::TtsText    │             │
│    loop)    │ ◀──────────────────────│             │
└─────────────┘     MsgOut::SttText    │             │
                ◀──────────────────────│             │
                    MsgOut::Event      │             │
                ◀──────────────────────└─────────────┘

Timing

All timestamps (start_s, stop_s, time_s) are relative to the start of the session, measured in seconds from when the first audio was received.

• start_s: When this audio/text segment begins
• stop_s: When this audio/text segment ends
• time_s: When this event occurred