openai-oxide 0.9.3

openai-oxide implements the full Responses API, Chat Completions, and 20+ other endpoints. It introduces performance primitives like persistent WebSockets, hedged requests, and early-parsing for function calls — features previously unavailable in the Rust ecosystem.

Why openai-oxide?

We built openai-oxide to squeeze every millisecond out of the OpenAI API.

• Zero-Overhead Streaming: Uses a custom zero-copy SSE parser. By enforcing strict Accept: text/event-stream and Cache-Control: no-cache headers, it prevents reverse-proxy buffering, achieving Time-To-First-Token (TTFT) in ~670ms.
• WebSocket Mode: Maintains a persistent wss:// connection for the Responses API. By bypassing per-request TLS handshakes, it reduces multi-turn agent loop latency by up to 37%.
• Hedged Requests: Send redundant requests and cancel the slower ones. Costs 2-7% extra tokens but reliably reduces P99 tail latency by 50-96% (inspired by Google's "The Tail at Scale").
• Stream FC Early Parse: Yields function calls the exact moment arguments.done is emitted, allowing you to start executing local tools ~400ms before the overall response finishes.
• WASM First-Class: Compiles to wasm32-unknown-unknown without dropping features. Unlike other clients, streaming, retries, and early-parsing work flawlessly in Cloudflare Workers and browsers.

Quick Start

Add the crate to your Cargo.toml:

[dependencies]
openai-oxide = "0.9"
tokio = { version = "1", features = ["full"] }

use openai_oxide::{OpenAI, types::responses::*};

#[tokio::main]
async fn main() -> Result<(), openai_oxide::OpenAIError> {
    let client = OpenAI::from_env()?; // Uses OPENAI_API_KEY

    let response = client.responses().create(
        ResponseCreateRequest::new("gpt-5.4")
            .input("Explain quantum computing in one sentence.")
            .max_output_tokens(100)
    ).await?;

    println!("{}", response.output_text());
    Ok(())
}

Benchmarks

All benchmarks were run to ensure a fair, real-world comparison of the clients:

• Environment: macOS (M-series), native compilation.
• Model: gpt-5.4 via the official OpenAI API.
• Protocol: TLS + HTTP/2 multiplexing with connection pooling (warm connections).
• Execution: 5 iterations per test. The reported value is the Median time.
• Rust APIs: openai-oxide provides first-class support for both the traditional Chat Completions API (/v1/chat/completions) and the newer Responses API (/v1/responses). The Responses API has slightly higher backend orchestration latency on OpenAI's side for non-streamed requests, so we separate them for fairness.

Rust Ecosystem (openai-oxide vs async-openai vs genai)

Reproduce: cargo run --example benchmark --features responses --release

Python Ecosystem (openai-oxide-python vs openai)

openai-oxide comes with native Python bindings via PyO3, exposing a drop-in async interface that outperforms the official Python SDK (openai + httpx).

Run uv run python examples/bench_python.py from the openai-oxide-python directory to test locally (Python 3.13).

Python Usage

Install via uv or pip:

cd openai-oxide-python
uv sync
uv run maturin develop --release

import asyncio
from openai_oxide import Client

async def main():
    client = Client()
    
    # 1. Standard request
    res = await client.create("gpt-5.4", "Hello!")
    print(res["text"])
    
    # 2. Streaming (Async Iterator)
    stream = await client.create_stream("gpt-5.4", "Explain quantum computing...", max_output_tokens=200)
    async for event in stream:
        print(event)

asyncio.run(main())

Advanced Features Guide

WebSocket Mode

Persistent connections bypass the TLS handshake penalty for every request. Ideal for high-speed agent loops.

let client = OpenAI::from_env()?;
let mut session = client.ws_session().await?;

// All calls route through the same wss:// connection
let r1 = session.send(
    ResponseCreateRequest::new("gpt-5.4").input("My name is Rustam.").store(true)
).await?;

let r2 = session.send(
    ResponseCreateRequest::new("gpt-5.4").input("What's my name?").previous_response_id(&r1.id)
).await?;

session.close().await?;

Streaming FC Early Parse

Start executing your local functions instantly when the model finishes generating the arguments, rather than waiting for the entire stream to close.

let mut handle = client.responses().create_stream_fc(request).await?;

while let Some(fc) = handle.recv().await {
    // Fires immediately on `arguments.done`
    let result = execute_tool(&fc.name, &fc.arguments).await;
}

Hedged Requests

Protect your application against random network latency spikes.

use openai_oxide::hedged_request;
use std::time::Duration;

// Sends 2 identical requests with a 1.5s delay. Returns whichever finishes first.
let response = hedged_request(&client, request, Some(Duration::from_secs(2))).await?;

Parallel Fan-Out

Leverage HTTP/2 multiplexing natively. Send 3 concurrent requests over a single connection; the total wall time is equal to the slowest single request.

let (c1, c2, c3) = (client.clone(), client.clone(), client.clone());
let (r1, r2, r3) = tokio::join!(
    async { c1.responses().create(req1).await },
    async { c2.responses().create(req2).await },
    async { c3.responses().create(req3).await },
);

Implemented APIs

Cargo Features & WASM Optimization

Every endpoint is gated behind a Cargo feature. If you are building for WebAssembly (WASM) (e.g., Cloudflare Workers, Dioxus, Leptos), you can significantly reduce your .wasm binary size and compilation time by disabling default features and only compiling what you need.

[dependencies]
# Example: Compile ONLY the Responses API (removes Audio, Images, Assistants, etc.)
openai-oxide = { version = "0.9", default-features = false, features = ["responses"] }

Available API Features:

• chat — Chat Completions
• responses — Responses API (Supports WebSocket)
• embeddings — Text Embeddings
• images — Image Generation (DALL-E)
• audio — TTS and Transcription
• files — File management
• fine-tuning — Model Fine-tuning
• models — Model listing
• moderations — Moderation API
• batches — Batch API
• uploads — Upload API
• beta — Assistants, Threads, Vector Stores, Realtime API

Ecosystem Features:

• websocket — Enables Realtime API over WebSockets (Native: tokio-tungstenite)
• websocket-wasm — Enables Realtime API over WebSockets (WASM: gloo-net / web-sys)
• simd — Enables simd-json for ultra-fast JSON deserialization (requires nightly Rust)

Check out our Cloudflare Worker Examples showcasing a Full-Stack Rust app with a Dioxus frontend and a Cloudflare Worker Durable Object backend holding a WebSocket connection to OpenAI.

Configuration

use openai_oxide::{OpenAI, config::ClientConfig};
use openai_oxide::azure::AzureConfig;

let client = OpenAI::new("sk-...");                             // Explicit key
let client = OpenAI::with_config(                               // Custom config
    ClientConfig::new("sk-...").base_url("https://...").timeout_secs(30).max_retries(3)
);
let client = OpenAI::azure(AzureConfig::new()                   // Azure OpenAI
    .azure_endpoint("https://my.openai.azure.com").azure_deployment("gpt-4").api_key("...")
)?;

Keeping up with OpenAI

OpenAI moves fast. To ensure openai-oxide never falls behind, we built an automated architecture synchronization pipeline.

Types are strictly validated against the official OpenAPI spec and cross-checked directly with the official Python SDK's AST.

make sync       # downloads latest spec, diffs against local schema, runs coverage

make sync automatically:

1. Downloads the latest OpenAPI schema from OpenAI.
2. Displays a precise git diff of newly added endpoints, struct fields, and enums.
3. Runs the openapi_coverage test suite to statically verify our Rust types against the spec.

Coverage is enforced on every commit via pre-commit hooks. Current field coverage for all implemented typed schemas is 100%. This guarantees 1:1 feature parity with the Python SDK, ensuring you can adopt new OpenAI models and features on day one.

Used In

• sgr-agent — LLM agent framework with structured output, function calling, and agent loops. openai-oxide is the default backend.
• rust-code — AI-powered TUI coding agent.

See Also

• openai-python — Official Python SDK (our benchmark baseline)
• async-openai — Alternative Rust client (mature, 1800+ stars)
• genai — Multi-provider Rust client (Gemini, Anthropic, OpenAI)

License

MIT