Crate gradbot 

gradbot

Core library for building voice AI applications with real-time speech-to-text, LLM processing, and text-to-speech.

Architecture

The library implements a multiplexing loop that coordinates:

• Speech-to-Text (STT): Converts incoming audio to text transcriptions
• LLM: Processes transcribed text and generates responses
• Text-to-Speech (TTS): Converts LLM responses back to audio

Quick Start

use gradbot::{GradbotClients, MsgOut, SessionConfig};

// 1. Create clients (uses environment variables for API keys)
let clients = GradbotClients::new(None, None, None, None, None, None).await?;

// 2. Start a session
let (input, mut output) = clients.start_session(Some(config), Format::OggOpus).await?;

// 3. Spawn a producer to send audio
tokio::spawn(async move {
    input.send_audio(audio_bytes).await.ok();
    // input is dropped when done -> session ends normally
});

// 4. Consume output messages
while let Some(msg) = output.receive().await? {
    match msg {
        MsgOut::Audio { data, .. } => { /* send audio */ }
        MsgOut::TtsText { text, .. } => { /* send caption */ }
        MsgOut::SttText { text, .. } => { /* send transcription */ }
        MsgOut::Event { event, .. } => { /* handle event */ }
    }
}

Environment Variables

• GRADIUM_API_KEY - API key for Gradium STT/TTS services
• GRADIUM_BASE_URL - Base URL for Gradium services (optional, defaults to https://api.gradium.ai)
• LLM_API_KEY - API key for OpenAI-compatible LLM API (falls back to OPENAI_API_KEY)
• LLM_BASE_URL - Base URL for LLM API (optional, defaults to OpenAI’s API)
• LLM_MODEL - LLM model name (optional, auto-detected if single model available)

Channel Semantics

• Normal termination: When SessionInputHandle is dropped (client disconnects), output.receive() returns Ok(None).
• Internal error: On any processing error, output.receive() returns Err(e).

Message Flow

┌─────────────┐     MsgIn::Audio      ┌─────────────┐
│  Producer   │ ──────────────────────▶│             │
│   (your     │     MsgIn::Config      │   session   │
│    loop)    │ ──────────────────────▶│   future    │
└─────────────┘                        │             │
                                       │             │
┌─────────────┐     MsgOut::Audio      │             │
│  Consumer   │ ◀──────────────────────│             │
│   (your     │     MsgOut::TtsText    │             │
│    loop)    │ ◀──────────────────────│             │
└─────────────┘     MsgOut::SttText    │             │
                ◀──────────────────────│             │
                    MsgOut::Event      │             │
                ◀──────────────────────└─────────────┘

Timing

All timestamps (start_s, stop_s, time_s) are relative to the start of the session, measured in seconds from when the first audio was received.

• start_s: When this audio/text segment begins
• stop_s: When this audio/text segment ends
• time_s: When this event occurred

Modules

decoder

encoder

utils

Structs

FlagshipVoice

Flagship voice information: name, voice ID, language, country, gender, and description.

GradbotClients

Shared clients for creating voice AI sessions.

IoFormat

Audio format pair for input (decoding) and output (encoding).

Llm

LlmConfig

SessionConfig

SessionInputHandle

Handle for sending input to a voice session.

SessionOutputHandle

Handle for receiving output from a voice session.

SttClient

ToolCall

A tool call made by the LLM, sent to the client for handling.

ToolCallHandle

Handle given to the client to send tool call results.

ToolDef

Tool definition for the LLM.

TtsClient

Enums

Country

Country/accent of a voice.

Event

Gender

Gender of a voice.

Lang

MsgIn

MsgOut

All outbound messages to the client.

Constants

DEFAULT_FLUSH_FOR_S

DEFAULT_GRADIUM_BASE_URL

Default Gradium API base URL.

FLAGSHIP_VOICES

All available flagship voices.

Functions

flagship_voice

Look up a flagship voice by name.

flagship_voices

Returns all available flagship voices.

run

Create clients and start a session in one call.

start_session

Start a voice AI session.

Type Aliases

ToolResult

A pending tool call being tracked by the session. Result type for tool calls - either success with JSON value or error.