Skip to main content

Crate seeduplex_client

Crate seeduplex_client 

Source
Expand description

Async Rust client for the Volcengine Doubao / Seed end-to-end realtime voice dialogue WebSocket API (/api/v3/realtime/dialogue), plus bidirectional streaming TTS (/api/v3/tts/bidirection) and bidirectional streaming ASR (/api/v3/sauc/bigmodel_async).

This crate covers the binary protocol and session clients only. It does not include microphone capture, AEC, robot tooling, or an external LLM.

Official upstream docs:

§Features

FeatureDefaultPurpose
(none)yesCore WebSocket clients + PCM helpers
opusnoOGG Opus decode helpers in protocol

§Modules

Prefer the namespaced modules for new code:

Flat re-exports at the crate root remain available for compatibility.

§Unified connect shape

All three clients share the same entry points:

  • Client::connect(api_key) — defaults
  • Client::connect_with_config(api_key, config) — custom session/config
  • Client::connect_with_url(url, api_key, resource_id, config) — full control

§Authentication

New Volcengine console credentials:

Legacy App-ID / Access-Key / App-Key headers are not used.

§Two common dialogue usage modes

§1. Full duplex dialogue (model replies)

Send audio or SeeduplexClient::send_text, then consume ServerEvent::ChatText + ServerEvent::AudioData.

When you are not continuously uploading microphone frames, set dialog.extra.input_mod to "text" or "keep_alive" via SessionConfig::with_input_mod, otherwise the server closes with DialogAudioIdleTimeoutError (52000042).

§2. ASR + barge-in + your own reply text (SeeduplexClient::chat_tts_text)

  1. Stream 16 kHz PCM with SeeduplexClient::send_audio
  2. Read ServerEvent::AsrText / ServerEvent::AsrEnded / ServerEvent::ClearCache
  3. After AsrEnded, stream your text with SeeduplexClient::chat_tts_text_segments (event 500)
  4. Play audio only when ServerEvent::TtsSentenceStart has tts_type == "chat_tts_text" (suppress the model’s default chit-chat)

§Bidirectional streaming TTS

One-shot:

let pcm = seeduplex_client::BidirectionalTtsClient::synthesize(
    &std::env::var("DOUBAO_API_KEY").unwrap(),
    "你好",
).await?;

Streaming session control:

use seeduplex_client::{
    BidirectionalTtsClient, BidirectionalTtsConfig, BidirectionalTtsEvent,
};

#[tokio::main]
async fn main() -> seeduplex_client::Result<()> {
    let api_key = std::env::var("DOUBAO_API_KEY").unwrap();
    let config = BidirectionalTtsConfig::new().with_format("pcm");
    let (client, mut events) = BidirectionalTtsClient::connect_with_config(&api_key, config).await?;
    client.start_session().await?;
    while let Some(ev) = events.recv().await {
        match ev {
            BidirectionalTtsEvent::SessionStarted { .. } => {
                client.send_text("你好").await?;
                client.finish_session().await?;
            }
            BidirectionalTtsEvent::AudioData(pcm) => { let _ = pcm; }
            BidirectionalTtsEvent::SessionFinished { .. } => break,
            _ => {}
        }
    }
    client.close().await?;
    Ok(())
}

§Bidirectional streaming ASR

One-shot:

let result = seeduplex_client::BidirectionalAsrClient::recognize(
    &std::env::var("DOUBAO_API_KEY").unwrap(),
    &[0u8; 6400],
).await?;
println!("{}", result.text);

Streaming with optional speaker diarization via BidirectionalAsrConfig::with_speaker_diarization:

use seeduplex_client::{
    BidirectionalAsrClient, BidirectionalAsrConfig, BidirectionalAsrEvent,
};

#[tokio::main]
async fn main() -> seeduplex_client::Result<()> {
    let api_key = std::env::var("DOUBAO_API_KEY").unwrap();
    let config = BidirectionalAsrConfig::new().with_speaker_diarization(true);
    let (client, mut events) = BidirectionalAsrClient::connect_with_config(&api_key, config).await?;
    client.send_audio(&[0u8; 6400]).await?; // 200ms silence placeholder
    client.finish().await?;
    while let Some(ev) = events.recv().await {
        match ev {
            BidirectionalAsrEvent::Final(r) => {
                println!("{}", r.text);
                break;
            }
            _ => {}
        }
    }
    client.close().await?;
    Ok(())
}

§Audio formats

  • Dialogue uplink: 16 kHz, mono, 16-bit PCM, 20 ms frames (320 samples / 640 bytes)
  • Dialogue downlink (crate default config): 24 kHz pcm_s16le
  • Bidirectional TTS downlink (crate default): 24 kHz pcm
  • Bidirectional ASR uplink: 16 kHz pcm_s16le, prefer ≈200 ms packets

§Quick start (dialogue)

use seeduplex_client::{DoubaoClient, ServerEvent, SessionConfig};

#[tokio::main]
async fn main() -> seeduplex_client::Result<()> {
    let api_key = std::env::var("DOUBAO_API_KEY").unwrap();
    let config = SessionConfig::new().with_input_mod("text".into());
    let (client, mut events) = DoubaoClient::connect_with_config(&api_key, config).await?;

    client.send_text("你好").await?;
    while let Some(event) = events.recv().await {
        match event {
            ServerEvent::ChatText(t) => println!("{t}"),
            ServerEvent::AudioData(pcm) => { let _ = pcm; }
            ServerEvent::TtsEnded | ServerEvent::SessionFinished => break,
            _ => {}
        }
    }
    client.close().await?;
    Ok(())
}

See the repository examples: text_query, mic_dialog, say_hello, text_to_file, update_config, asr_chat_tts, bidirectional_tts, and bidirectional_asr.

§TLS note

tokio-tungstenite is built with rustls. Applications typically need to install a CryptoProvider once at startup, for example:

ⓘ
rustls::crypto::ring::default_provider()
    .install_default()
    .expect("rustls CryptoProvider");

Re-exports§

pub use bidirectional_asr::AsrResultPayload;
pub use bidirectional_asr::AsrUtterance;
pub use bidirectional_asr::BidirectionalAsrAudio;
pub use bidirectional_asr::BidirectionalAsrClient;
pub use bidirectional_asr::BidirectionalAsrConfig;
pub use bidirectional_asr::BidirectionalAsrEvent;
pub use bidirectional_asr::BidirectionalAsrRequest;
pub use bidirectional_asr::BidirectionalAsrUser;
pub use bidirectional_asr::ASR_WS_URL_BIDIRECTION;
pub use bidirectional_asr::ASR_WS_URL_NOSTREAM;
pub use bidirectional_asr::DEFAULT_ASR_RESOURCE_ID;
pub use bidirectional_asr::DEFAULT_ASR_WS_URL;
pub use bidirectional_tts::BidirectionalAudioParams;
pub use bidirectional_tts::BidirectionalReqParams;
pub use bidirectional_tts::BidirectionalTtsClient;
pub use bidirectional_tts::BidirectionalTtsConfig;
pub use bidirectional_tts::BidirectionalTtsEvent;
pub use bidirectional_tts::BidirectionalUser;
pub use bidirectional_tts::DEFAULT_TTS_RESOURCE_ID;
pub use bidirectional_tts::DEFAULT_TTS_SPEAKER;
pub use bidirectional_tts::DEFAULT_TTS_WS_URL;
pub use client::AsrConfig;
pub use client::AsrExtra;
pub use client::AudioOutputConfig;
pub use client::DialogConfig;
pub use client::DialogExtra;
pub use client::DoubaoClient;
pub use client::InputAudioConfig;
pub use client::SeeduplexClient;
pub use client::ServerEvent;
pub use client::SessionConfig;
pub use client::TtsConfig;
pub use client::TtsExtra;
pub use client::UserConfig;
pub use error::Error;
pub use error::Result;

Modules§

asr
Bidirectional streaming ASR.
bidirectional_asr
WebSocket bidirectional streaming ASR client (wss://openspeech.bytedance.com/api/v3/sauc/bigmodel_async).
bidirectional_tts
WebSocket bidirectional streaming TTS client (wss://openspeech.bytedance.com/api/v3/tts/bidirection).
client
Async WebSocket client for Volcengine Doubao / Seed end-to-end realtime dialogue.
dialogue
Realtime dialogue client and session types.
error
Typed errors for all Seeduplex / Doubao WebSocket clients.
protocol
Binary protocol encoding and decoding for Volcengine Doubao / Seed end-to-end realtime dialogue (/api/v3/realtime/dialogue).
tts
Bidirectional streaming TTS.

Constants§

DEFAULT_RESOURCE_ID
Default resource ID for end-to-end realtime dialogue.
DEFAULT_SPEAKER
Default built-in TTS speaker used by SessionConfig::new.
DEFAULT_WS_URL
Default realtime dialogue WebSocket URL.