Expand description
Async Rust client for the Volcengine Doubao / Seed end-to-end realtime
voice dialogue WebSocket API (/api/v3/realtime/dialogue), plus
bidirectional streaming TTS (/api/v3/tts/bidirection) and
bidirectional streaming ASR (/api/v3/sauc/bigmodel_async).
This crate covers the binary protocol and session clients only. It does not include microphone capture, AEC, robot tooling, or an external LLM.
Official upstream docs:
- Realtime dialogue: https://www.volcengine.com/docs/6561/1594356
- Bidirectional TTS V3: https://docs.volcengine.com/docs/DoubaoVoice/WebSocketBidirectionalStreaming-V3
- Bidirectional ASR: https://docs.volcengine.com/docs/DoubaoVoice/bidirectional-streaming-automatic-speech-recognition-websocket
§Features
| Feature | Default | Purpose |
|---|---|---|
| (none) | yes | Core WebSocket clients + PCM helpers |
opus | no | OGG Opus decode helpers in protocol |
§Modules
Prefer the namespaced modules for new code:
dialogue— realtime duplex dialogue (SeeduplexClient)tts— bidirectional streaming TTS (BidirectionalTtsClient)asr— bidirectional streaming ASR (BidirectionalAsrClient)
Flat re-exports at the crate root remain available for compatibility.
§Unified connect shape
All three clients share the same entry points:
Client::connect(api_key)— defaultsClient::connect_with_config(api_key, config)— custom session/configClient::connect_with_url(url, api_key, resource_id, config)— full control
§Authentication
New Volcengine console credentials:
- Header
X-Api-Key— API key (required) - Header
X-Api-Resource-Id— service resource (dialogue:DEFAULT_RESOURCE_ID; TTS:DEFAULT_TTS_RESOURCE_ID; ASR:DEFAULT_ASR_RESOURCE_ID) - Header
X-Api-Connect-Id— UUID, added automatically by the client
Legacy App-ID / Access-Key / App-Key headers are not used.
§Two common dialogue usage modes
§1. Full duplex dialogue (model replies)
Send audio or SeeduplexClient::send_text, then consume
ServerEvent::ChatText + ServerEvent::AudioData.
When you are not continuously uploading microphone frames, set
dialog.extra.input_mod to "text" or "keep_alive" via
SessionConfig::with_input_mod, otherwise the server closes with
DialogAudioIdleTimeoutError (52000042).
§2. ASR + barge-in + your own reply text (SeeduplexClient::chat_tts_text)
- Stream 16 kHz PCM with
SeeduplexClient::send_audio - Read
ServerEvent::AsrText/ServerEvent::AsrEnded/ServerEvent::ClearCache - After
AsrEnded, stream your text withSeeduplexClient::chat_tts_text_segments(event 500) - Play audio only when
ServerEvent::TtsSentenceStarthastts_type == "chat_tts_text"(suppress the model’s default chit-chat)
§Bidirectional streaming TTS
One-shot:
let pcm = seeduplex_client::BidirectionalTtsClient::synthesize(
&std::env::var("DOUBAO_API_KEY").unwrap(),
"你好",
).await?;Streaming session control:
use seeduplex_client::{
BidirectionalTtsClient, BidirectionalTtsConfig, BidirectionalTtsEvent,
};
#[tokio::main]
async fn main() -> seeduplex_client::Result<()> {
let api_key = std::env::var("DOUBAO_API_KEY").unwrap();
let config = BidirectionalTtsConfig::new().with_format("pcm");
let (client, mut events) = BidirectionalTtsClient::connect_with_config(&api_key, config).await?;
client.start_session().await?;
while let Some(ev) = events.recv().await {
match ev {
BidirectionalTtsEvent::SessionStarted { .. } => {
client.send_text("你好").await?;
client.finish_session().await?;
}
BidirectionalTtsEvent::AudioData(pcm) => { let _ = pcm; }
BidirectionalTtsEvent::SessionFinished { .. } => break,
_ => {}
}
}
client.close().await?;
Ok(())
}§Bidirectional streaming ASR
One-shot:
let result = seeduplex_client::BidirectionalAsrClient::recognize(
&std::env::var("DOUBAO_API_KEY").unwrap(),
&[0u8; 6400],
).await?;
println!("{}", result.text);Streaming with optional speaker diarization via
BidirectionalAsrConfig::with_speaker_diarization:
use seeduplex_client::{
BidirectionalAsrClient, BidirectionalAsrConfig, BidirectionalAsrEvent,
};
#[tokio::main]
async fn main() -> seeduplex_client::Result<()> {
let api_key = std::env::var("DOUBAO_API_KEY").unwrap();
let config = BidirectionalAsrConfig::new().with_speaker_diarization(true);
let (client, mut events) = BidirectionalAsrClient::connect_with_config(&api_key, config).await?;
client.send_audio(&[0u8; 6400]).await?; // 200ms silence placeholder
client.finish().await?;
while let Some(ev) = events.recv().await {
match ev {
BidirectionalAsrEvent::Final(r) => {
println!("{}", r.text);
break;
}
_ => {}
}
}
client.close().await?;
Ok(())
}§Audio formats
- Dialogue uplink: 16 kHz, mono, 16-bit PCM, 20 ms frames (320 samples / 640 bytes)
- Dialogue downlink (crate default config): 24 kHz
pcm_s16le - Bidirectional TTS downlink (crate default): 24 kHz
pcm - Bidirectional ASR uplink: 16 kHz
pcm_s16le, prefer ≈200 ms packets
§Quick start (dialogue)
use seeduplex_client::{DoubaoClient, ServerEvent, SessionConfig};
#[tokio::main]
async fn main() -> seeduplex_client::Result<()> {
let api_key = std::env::var("DOUBAO_API_KEY").unwrap();
let config = SessionConfig::new().with_input_mod("text".into());
let (client, mut events) = DoubaoClient::connect_with_config(&api_key, config).await?;
client.send_text("你好").await?;
while let Some(event) = events.recv().await {
match event {
ServerEvent::ChatText(t) => println!("{t}"),
ServerEvent::AudioData(pcm) => { let _ = pcm; }
ServerEvent::TtsEnded | ServerEvent::SessionFinished => break,
_ => {}
}
}
client.close().await?;
Ok(())
}See the repository examples: text_query, mic_dialog, say_hello,
text_to_file, update_config, asr_chat_tts, bidirectional_tts,
and bidirectional_asr.
§TLS note
tokio-tungstenite is built with rustls. Applications typically need to
install a CryptoProvider once at startup, for example:
rustls::crypto::ring::default_provider()
.install_default()
.expect("rustls CryptoProvider");Re-exports§
pub use bidirectional_asr::AsrResultPayload;pub use bidirectional_asr::AsrUtterance;pub use bidirectional_asr::BidirectionalAsrAudio;pub use bidirectional_asr::BidirectionalAsrClient;pub use bidirectional_asr::BidirectionalAsrConfig;pub use bidirectional_asr::BidirectionalAsrEvent;pub use bidirectional_asr::BidirectionalAsrRequest;pub use bidirectional_asr::BidirectionalAsrUser;pub use bidirectional_asr::ASR_RECOMMENDED_FRAME_BYTES;pub use bidirectional_asr::ASR_WS_URL_BIDIRECTION;pub use bidirectional_asr::ASR_WS_URL_NOSTREAM;pub use bidirectional_asr::DEFAULT_ASR_RESOURCE_ID;pub use bidirectional_asr::DEFAULT_ASR_WS_URL;pub use bidirectional_tts::BidirectionalAudioParams;pub use bidirectional_tts::BidirectionalReqParams;pub use bidirectional_tts::BidirectionalTtsClient;pub use bidirectional_tts::BidirectionalTtsConfig;pub use bidirectional_tts::BidirectionalTtsEvent;pub use bidirectional_tts::BidirectionalUser;pub use bidirectional_tts::DEFAULT_TTS_RESOURCE_ID;pub use bidirectional_tts::DEFAULT_TTS_SPEAKER;pub use bidirectional_tts::DEFAULT_TTS_WS_URL;pub use client::AsrConfig;pub use client::AsrExtra;pub use client::AudioOutputConfig;pub use client::DialogConfig;pub use client::DialogExtra;pub use client::DoubaoClient;pub use client::InputAudioConfig;pub use client::SeeduplexClient;pub use client::ServerEvent;pub use client::SessionConfig;pub use client::TtsConfig;pub use client::TtsExtra;pub use client::UserConfig;pub use error::Error;pub use error::Result;
Modules§
- asr
- Bidirectional streaming ASR.
- bidirectional_
asr - WebSocket bidirectional streaming ASR client
(
wss://openspeech.bytedance.com/api/v3/sauc/bigmodel_async). - bidirectional_
tts - WebSocket bidirectional streaming TTS client
(
wss://openspeech.bytedance.com/api/v3/tts/bidirection). - client
- Async WebSocket client for Volcengine Doubao / Seed end-to-end realtime dialogue.
- dialogue
- Realtime dialogue client and session types.
- error
- Typed errors for all Seeduplex / Doubao WebSocket clients.
- protocol
- Binary protocol encoding and decoding for Volcengine Doubao / Seed
end-to-end realtime dialogue (
/api/v3/realtime/dialogue). - tts
- Bidirectional streaming TTS.
Constants§
- DEFAULT_
RESOURCE_ ID - Default resource ID for end-to-end realtime dialogue.
- DEFAULT_
SPEAKER - Default built-in TTS speaker used by
SessionConfig::new. - DEFAULT_
WS_ URL - Default realtime dialogue WebSocket URL.