audacity-sdk (Rust)

Rust SDK for the Audacity Investments AI gateway. Exposes an Amazon Bedrock Converse-shaped API so teams migrating off Bedrock can swap the client construction + API key and keep the rest of their code.

Installation

Add to Cargo.toml:

[dependencies]
audacity-sdk = "0.5.1"
tokio = { version = "1", features = ["rt-multi-thread", "macros"] }

Quickstart

Converse (non-streaming)

use audacity_sdk::{Client, ContentBlock, ConversationRole, Message};

#[tokio::main]
async fn main() -> Result<(), audacity_sdk::Error> {
    // Reads AUDACITY_API_KEY from the environment.
    let client = Client::from_env()?;

    let response = client.converse()
        .model_id("gpt-5.4-mini")
        .messages(
            Message::builder()
                .role(ConversationRole::User)
                .content(ContentBlock::Text("Hello!".into()))
                .build()?
        )
        .inference_config(
            audacity_sdk::InferenceConfiguration::builder()
                .max_tokens(500)
                .temperature(0.2)
                .build()
        )
        .send()
        .await?;

    let text = response
        .output().unwrap()
        .as_message().unwrap()
        .content().first().unwrap()
        .as_text().unwrap();

    println!("{text}");
    Ok(())
}

ConverseStream (streaming)

use audacity_sdk::{Client, ContentBlock, ConversationRole, ConverseStreamOutput, Message};

#[tokio::main]
async fn main() -> Result<(), audacity_sdk::Error> {
    let client = Client::from_env()?;

    let mut output = client.converse_stream()
        .model_id("gpt-5.4-mini")
        .messages(
            Message::builder()
                .role(ConversationRole::User)
                .content(ContentBlock::Text("Tell me a joke".into()))
                .build()?
        )
        .send()
        .await?;

    while let Some(event) = output.stream.recv().await? {
        if let ConverseStreamOutput::ContentBlockDelta(e) = event {
            if let audacity_sdk::ContentBlockDeltaPayload::Text(t) = e.delta {
                print!("{t}");
            }
        }
    }
    println!();
    Ok(())
}

OpenAI & Anthropic native formats (pass-through)

The gateway natively serves the OpenAI Chat Completions and Anthropic Messages wire formats, and the SDK exposes both directly — same auth, retry, and error handling, no shape translation. Params are any serde_json::json! object (or your own Serialize struct) sent verbatim, so any field the gateway supports works with no SDK release needed. Responses come back as raw serde_json::Values in the provider's own shape. Both formats work with every gateway model — the gateway bridges the format.

OpenAI format

use serde_json::json;

let client = audacity_sdk::Client::from_env()?;

// Non-streaming: POST /v1/chat/completions, raw OpenAI-shaped response.
let response = client.chat_completions().create(json!({
    "model": "gpt-5.4-mini",
    "messages": [{"role": "user", "content": "Hello!"}],
    "max_tokens": 500,
})).await?;
println!("{}", response["choices"][0]["message"]["content"]);

// Streaming: raw chunk objects; the stream ends at the gateway's
// `data: [DONE]` sentinel (which is not yielded).
let mut stream = client.chat_completions().create_stream(json!({
    "model": "gpt-5.4-mini",
    "messages": [{"role": "user", "content": "Tell me a joke"}],
})).await?;
while let Some(chunk) = stream.recv().await? {
    if let Some(delta) = chunk["choices"][0]["delta"]["content"].as_str() {
        print!("{delta}");
    }
}

Anthropic format

The wire format used by the anthropic SDKs and Claude Code — usable with any gateway model, not just Claude:

use serde_json::json;

// Non-streaming: POST /v1/messages, raw Anthropic-shaped response.
let response = client.messages().create(json!({
    "model": "claude-sonnet-4-5",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Hello!"}],
})).await?;
println!("{}", response["content"][0]["text"]);

// Streaming: raw Anthropic events (message_start … message_stop).
let mut stream = client.messages().create_stream(json!({
    "model": "claude-sonnet-4-5",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "Tell me a joke"}],
})).await?;
while let Some(event) = stream.recv().await? {
    if event["type"] == "content_block_delta" {
        print!("{}", event["delta"]["text"].as_str().unwrap_or_default());
    }
}

// Token counting (free — no inference): POST /v1/messages/count_tokens.
let count = client.messages().count_tokens(json!({
    "model": "claude-sonnet-4-5",
    "messages": [{"role": "user", "content": "How many tokens is this?"}],
})).await?;
println!("{}", count["input_tokens"]);

create_stream sets stream: true on the body for you (everything else passes through untouched); calling create with stream: true in the params is rejected client-side. Streaming failures surface as Error::ModelStreamError: an OpenAI stream that ends without [DONE], an Anthropic stream that ends before message_stop, or a mid-stream transport drop. In-stream error payloads map through the same error taxonomy as non-streaming calls (e.g. rate limits → Error::Throttling).

Images (vision models)

Bedrock-style image content blocks are supported in user messages. Pass raw bytes (base64-encoded for you) or a URL (Audacity extension):

use audacity_sdk::{ContentBlock, ImageBlock, ImageFormat, ImageSource};

let image_bytes = std::fs::read("chart.png")?;

let response = client.converse()
    .model_id("gpt-5.5")
    .messages(
        Message::builder()
            .role(ConversationRole::User)
            .content(ContentBlock::Text("What does this chart show?".into()))
            .content(ContentBlock::Image(ImageBlock {
                format: ImageFormat::Png,
                source: ImageSource::Bytes(image_bytes),
            }))
            .build()?
    )
    .send()
    .await?;

// Or reference a hosted image directly (not available in Bedrock):
// source: ImageSource::Url("https://example.com/photo.jpg".into())

ImageFormat is one of Png, Jpeg, Gif, Webp. Use a vision-capable model.

Video input

Bedrock-style video content blocks are supported in user messages. Pass raw bytes; they are base64-encoded into an inline data URL for you:

use audacity_sdk::{ContentBlock, VideoBlock, VideoFormat, VideoSource};

let video_bytes = std::fs::read("demo.mp4")?;

let response = client.converse()
    .model_id("gemini-2.5-flash")
    .messages(
        Message::builder()
            .role(ConversationRole::User)
            .content(ContentBlock::Text("What happens in this clip?".into()))
            .content(ContentBlock::Video(VideoBlock {
                format: VideoFormat::Mp4,
                source: VideoSource::Bytes(video_bytes),
            }))
            .build()?
    )
    .send()
    .await?;

VideoFormat is one of Mp4, Mov, Mkv, Webm, Flv, Mpeg, Mpg, Wmv, ThreeGp.

Video is Gemini-only at the gateway: use gemini-2.5-flash, gemini-2.5-pro, or gemini-3-flash-preview — other models reject video input with an HTTP 400. Keep inline video ≤ ~20 MB (base64 encoding inflates the request body against the gateway's cap); upload larger files first and reference them by URI.

Media resolution (cheaper video tokens)

Set media_resolution to control how densely video (and image) input is sampled on Gemini models — Low cuts video token cost roughly 4x. Values are Low, Medium, High, UltraHigh; non-Gemini models ignore the field:

use audacity_sdk::MediaResolution;

let response = client.converse()
    .model_id("gemini-2.5-flash")
    .media_resolution(MediaResolution::Low)
    .messages(/* … video message … */)
    .send()
    .await?;

Large videos: upload + URI

For videos up to 1 GB, upload once via the file-upload helper and reference the returned audacity://files/… URI with VideoSource::Uri (mirrors Bedrock Converse's bytes | s3Location pattern):

use audacity_sdk::{ContentBlock, VideoBlock, VideoFormat, VideoSource};

let video_bytes = std::fs::read("large-demo.mp4")?;

// Under the hood: POST /v1/files issues a presigned upload URL (~15 min
// validity), then the bytes go up over a resumable upload session in 8 MiB
// chunks. If the connection drops mid-upload, the SDK automatically resumes
// from the last byte the server confirmed (bounded retries with jittered
// backoff) — no need to restart a large upload from zero.
let upload = client.upload_file()
    .data(video_bytes)
    .content_type("video/mp4")
    .send()
    .await?;

let response = client.converse()
    .model_id("gemini-2.5-flash")
    .messages(
        Message::builder()
            .role(ConversationRole::User)
            .content(ContentBlock::Text("Summarise this recording.".into()))
            .content(ContentBlock::Video(VideoBlock {
                format: VideoFormat::Mp4,
                source: VideoSource::Uri(upload.uri),
            }))
            .build()?
    )
    .send()
    .await?;

Uploaded files are transient inference inputs: they auto-delete after ~24 h, so upload shortly before use and re-upload for later sessions. Files are namespaced per API key's client — a URI from one client is not visible to another.

Image generation

Generate images from a text prompt with generate_image. With response_format("b64_json") the image bytes come back inline:

use base64::Engine as _;

let result = client.generate_image()
    .model("gpt-image-1")
    .prompt("A watercolor painting of a fox in a snowy forest")
    .size("1024x1024")
    .response_format("b64_json")
    .send()
    .await?;

let b64 = result.data[0].b64_json.as_deref().unwrap();
let bytes = base64::engine::general_purpose::STANDARD.decode(b64)?;
std::fs::write("fox.png", bytes)?;

With response_format "url" (the default) the gateway stores the image and returns a signed download URL that expires after ~24 hours:

let result = client.generate_image()
    .model("imagen-4")
    .prompt("A watercolor painting of a fox in a snowy forest")
    .send()
    .await?;

println!("{}", result.data[0].url.as_deref().unwrap()); // signed URL, valid ~24 h

Optional builder methods: n (1–10 images), size ("WxH", model-dependent), quality (e.g. "standard", "hd"), and user. The output carries created, data (each entry has url or b64_json, plus revised_prompt when the provider rewrites your prompt) and optional usage token counts. Errors map to the same Error variants as converse (401 → Error::AccessDenied, 429 → Error::Throttling, spend cap → Error::ServiceQuotaExceeded).

Image models

Per-image models bill a flat rate per generated image; token-based models report token counts in the output's usage. Each request's cost is recorded against your key like any other API call.

Reliability note. Upstream image backends occasionally stall with a 503 for a few minutes. There is deliberately no automatic fallback to a different image model (silently swapping models would change output style and quality) — the SDK already retries 503s with backoff up to max_retries, and callers should retry beyond that rather than switch models.

Prompt caching

Place a Bedrock-style cache-point block after the stable prefix you want the provider to cache (system prompt, large documents). Everything up to the cache point is cached provider-side on Claude models; OpenAI/Gemini models cache automatically and ignore the marker. At most 4 cache points per request.

use audacity_sdk::{CachePointBlock, ContentBlock, SystemContentBlock};

let response = client.converse()
    .model_id("claude-sonnet-4-5")
    .system(SystemContentBlock::text(long_system_prompt))
    .system(SystemContentBlock::cache_point())
    .messages(
        Message::builder()
            .role(ConversationRole::User)
            .content(ContentBlock::Text(big_reference_document))
            .content(ContentBlock::CachePoint(CachePointBlock::new()))
            .content(ContentBlock::Text("Summarise the key risks.".into()))
            .build()?
    )
    .send()
    .await?;

// Cache activity is reported in usage (Bedrock names):
println!("{}", response.usage().cache_read_input_tokens);  // tokens served from cache
println!("{}", response.usage().cache_write_input_tokens); // tokens written to cache

A cache point with nothing before it in the same message is silently ignored.

Migrating from aws-sdk-bedrockruntime

The SDK is designed to be a drop-in. Here is a side-by-side diff:

-use aws_sdk_bedrockruntime::Client;
-use aws_sdk_bedrockruntime::types::{ContentBlock, ConversationRole, Message};
+use audacity_sdk::{Client, ContentBlock, ConversationRole, Message};

-let config = aws_config::load_from_env().await;
-let client = Client::new(&config);
+let client = audacity_sdk::Client::from_env()?;   // reads AUDACITY_API_KEY

 let response = client.converse()
-    .model_id("anthropic.claude-3-5-sonnet-20241022-v2:0")
+    .model_id("gpt-5.4-mini")
     .messages(
         Message::builder()
             .role(ConversationRole::User)
-            .content(ContentBlock::Text("Hello".into()))
+            .content(ContentBlock::Text("Hello".into()))
             .build()?
     )
     .send()
     .await?;

Error handling

use audacity_sdk::{Client, Error};

match client.converse().model_id("m").messages(msg).send().await {
    Ok(resp) => { /* use resp */ }
    Err(Error::Throttling(d)) => {
        eprintln!("Rate limited (retry after {:?}s): {}", d.retry_after_seconds, d.message);
    }
    Err(Error::AccessDenied(d)) => {
        eprintln!("Access denied [{}]: {}", d.error_code.unwrap_or_default(), d.message);
    }
    Err(Error::MissingApiKey) => {
        eprintln!("Set AUDACITY_API_KEY");
    }
    Err(e) => eprintln!("Other error: {e}"),
}

Error variants

Every server error carries ErrorDetails { message, status_code, error_code, request_id, retry_after_seconds, raw_body }.

Configuration

Other options:

use std::time::Duration;

let config = audacity_sdk::Config::builder()
    .api_key("aireserve_api_…")
    .base_url("https://api.audacityinvestments.com")
    .timeout(Duration::from_secs(120))
    .max_retries(2)   // 3 total attempts
    .build()?;

let client = audacity_sdk::Client::new(&config)?;

The timeout bounds each converse attempt end to end (including the body read). For converse_stream it only bounds the wait for response headers — the SSE body read is unbounded, so long generations are never cut off mid-stream.

License

Copyright Audacity Investments. All rights reserved.
See LICENSE.