openai_interface/

lib.rs

//! A low-level Rust interface for interacting with OpenAI's API.
//!
//! This crate provides a simple, efficient, and low-level way to interact with OpenAI's API,
//! supporting both streaming and non-streaming responses. It leverages Rust's powerful type
//! system for safety and performance, while exposing the full flexibility of the API.
//!
//! # Features
//!
//! - **Chat Completions**: Full support for OpenAI's chat completion and completion API,
//!   including both streaming and non-streaming responses, and multimodal user
//!   messages (text / image / audio / file content parts).
//! - **Responses**: Create model responses with the Responses API (string or
//!   item-based input, function tools, built-in web search, streaming events),
//!   retrieve and delete stored responses. Tested against DeepSeek and Qwen.
//! - **Models**: List, retrieve and delete models.
//! - **Embeddings**: Create embedding vectors from text input.
//! - **Moderations**: Classify whether text and/or image input is potentially
//!   harmful (untested).
//! - **Images**: Generate, edit, and create variations of images (untested).
//! - **Audio**: Text-to-speech, transcription, and translation endpoints (untested).
//! - **Files**: Support for the OpenAI file API (upload, list, retrieve, delete,
//!   download content).
//! - **Streaming and Non-streaming**: Support for both streaming and non-streaming responses.
//! - **Strong Typing**: Complete type definitions for all API requests and responses,
//!   utilizing Rust's powerful type system.
//! - **Configurable HTTP Client**: Every request method takes a [`reqwest::Client`], so
//!   proxies, timeouts and connection pooling are under your control. See
//!   [`rest::default_client`] for a sensible default, and
//!   [`rest::install_crypto_provider`] to pick the TLS backend.
//! - **Error Handling**: Comprehensive error handling with detailed error types defined in
//!   the [`errors`] module. Failed requests carry the API's error message, type and code.
//! - **Async/Await**: Built with async/await support.
//! - **Musl Support**: Designed to work with musl libc out-of-the-box; TLS is
//!   pure Rust, so no OpenSSL or C toolchain is needed.
//! - **Multiple Provider Support**: Expected to work with OpenAI, DeepSeek, Qwen, vLLM
//!   and other compatible API providers. Provider-specific fields are opt-in via cargo
//!   features (see below).
//!
//! ## Cargo Features
//!
//! Fields that are proprietary to a single provider are opt-in via cargo
//! features. Cross-vendor de-facto standards — such as `reasoning_content`
//! (streamed by DeepSeek, Qwen3, ollama, vLLM and OpenRouter alike) and
//! `reasoning_effort` — are always available:
//!
//! - **`reasoning`** (default): cross-vendor reasoning fields —
//!   `reasoning_content` on assistant messages (request and response),
//!   streamed deltas, and logprobs, plus its accumulation in
//!   [`chat::create::accumulator::ChatCompletionAccumulator`].
//!
//! - **`deepseek`**: enables DeepSeek's proprietary fields — the Beta chat
//!   prefix completion fields (`prefix`, and `reasoning_content` as the
//!   prefix-completion CoT input), the `thinking` and `user_id` request
//!   parameters, and the `prompt_cache_hit_tokens` /
//!   `prompt_cache_miss_tokens` usage statistics. Implies `reasoning`. See
//!   [api-docs.deepseek.com](https://api-docs.deepseek.com/).
//!
//! - **`qwen`**: Enables Qwen's proprietary fields — the chat request
//!   parameters `enable_thinking`, `thinking_budget` and `top_k`, the
//!   Responses API input part `input_file`, the built-in tools
//!   (`web_extractor`, `code_interpreter`, `web_search_image`,
//!   `image_search`, `file_search`, `mcp`), the corresponding output items
//!   and streaming events, and the `x_details` / `x_tools` usage
//!   statistics. Implies `reasoning`. See
//!   [the Qwen OpenAI-compatible Chat API docs](https://www.alibabacloud.com/help/zh/model-studio/qwen-api-via-openai-chat-completions)
//!   and
//!   [the Qwen Responses API docs](https://www.alibabacloud.com/help/zh/model-studio/qwen-api-via-openai-responses).
//!
//! - **`vllm`**: enables vLLM's proprietary fields, collected in the
//!   [`vllm`] module — the extra sampling parameters (`min_p`,
//!   `repetition_penalty`, `stop_token_ids`, `prompt_logprobs`,
//!   `bad_words`, ...), the chat-template controls (`chat_template`,
//!   `chat_template_kwargs`, `add_generation_prompt`, ...),
//!   `structured_outputs` (vLLM's successor to the deprecated `guided_*`
//!   keys), the KV-transfer and scheduling parameters
//!   (`kv_transfer_params`, `priority`, `cache_salt`, ...), and the extra
//!   response fields vLLM returns (`stop_reason`, `token_ids`,
//!   `prompt_logprobs`, `prompt_token_ids`, `prompt_text`,
//!   `kv_transfer_params`) plus `root` / `parent` / `max_model_len` on
//!   model objects. These are flattened into the request bodies through
//!   `RequestBody::vllm_sampling` and `RequestBody::vllm_chat`, matching
//!   the wire format the official client's `extra_body` produces. Implies
//!   `reasoning`. See
//!   [vLLM's OpenAI-compatible server docs](https://docs.vllm.ai/en/latest/serving/online_serving/openai_compatible_server/).
//!
//! - **`azure`**: deprecated no-op. Streaming `delta.annotations` and
//!   `delta.audio` are now always available (the non-streaming message
//!   fields were never gated). The empty feature remains defined so
//!   existing manifests keep compiling.
//!
//! There is one feature unrelated to request fields:
//!
//! - **`ferritls`**: Adds the pure-Rust `ferritls-rustls` TLS crypto backend
//!   and the [`rest::install_crypto_provider`] helper that installs it. Off by
//!   default, so the crate never dictates your crypto backend; when you leave
//!   it off, install a [`rustls::crypto::CryptoProvider`] yourself before
//!   building any client. See ["TLS Crypto Provider"](#tls-crypto-provider).
//!
//! ## Implemented APIs
//!
//! - Chat Completions (create / retrieve / update / delete)
//! - Responses (create / retrieve / delete)
//! - Completions
//! - Models (list / retrieve / delete)
//! - Embeddings
//! - Moderations (untested)
//! - Images (generate / edit / variation, untested)
//! - Audio (speech / transcriptions / translations, untested)
//! - Files (create / list / retrieve / delete / download content)
//!
//! # TLS Crypto Provider
//!
//! HTTP is done by `reqwest`, depended on with its `rustls-no-provider`
//! feature: the rustls stack is compiled **without** a crypto backend, which
//! keeps the pure-Rust build (no C or asm toolchain needed) and leaves the
//! backend choice to the application. Consequently, exactly one
//! [`rustls::crypto::CryptoProvider`] must be installed as the process default
//! before any [`reqwest::Client`] is built — including the one returned by
//! [`rest::default_client`]. If none is installed, reqwest panics at client
//! construction time.
//!
//! This crate never installs a provider on your behalf. The optional
//! **`ferritls`** cargo feature adds the pure-Rust `ferritls-rustls` backend
//! together with [`rest::install_crypto_provider`], so you can delegate that
//! one decision to the crate:
//!
//! ```toml
//! [dependencies]
//! openai-interface = { version = "0.12", features = ["ferritls"] }
//! ```
//!
//! ```rust,no_run
//! # #[cfg(feature = "ferritls")] {
//! // Choose the backend once, before building any client:
//! openai_interface::rest::install_crypto_provider()
//!     .expect("a rustls crypto provider was already installed");
//! # }
//! ```
//!
//! To use a different backend (`ring`, `aws-lc-rs`, or a hand-picked
//! [`rustls::crypto::CryptoProvider`]), leave the feature off and install it
//! yourself — first install wins, so whichever provider is in place when the
//! first client is built is the one everything in the process uses:
//!
//! ```rust,ignore
//! // In the application crate, with `rustls = "0.23"` (feature `ring` or
//! // `aws-lc-rs`) as one of its own dependencies:
//! rustls::crypto::ring::default_provider()
//!     .install_default()
//!     .expect("a rustls crypto provider was already installed");
//! ```
//!
//! ## When nothing needs to be installed
//!
//! Cargo features are additive across the dependency tree, so if your project
//! depends on `reqwest` itself with a crypto backend compiled in — its default
//! `default-tls`, or `rustls` explicitly — reqwest falls back to the
//! `aws-lc-rs` provider it ships with, and no install step is needed at all.
//! Enabling `native-tls` instead routes TLS through the system stack, so the
//! rustls path is never taken.
//!
//! ```toml
//! [dependencies]
//! reqwest = "0.13"          # default features: `default-tls` -> `rustls`
//! openai-interface = "0.12" # no provider of its own
//! ```
//!
//! The catch is that the backend is then decided by feature unification rather
//! than by you, and an unrelated dependency change can move it. To pin the
//! choice, enable the `ferritls` feature or install a provider yourself.
//!
//! # Examples
//!
//! ## Non-streaming Chat Completion
//!
//! This example demonstrates how to make a non-streaming request to the chat completion API.
//!
//! ```rust,no_run
//! use openai_interface::chat::create::request::{Message, RequestBody};
//! use openai_interface::chat::create::response::no_streaming::ChatCompletion;
//! use openai_interface::rest::{RequestOptions, default_client, post::PostNoStream};
//!
//! const DEEPSEEK_CHAT_URL: &'static str = "https://api.deepseek.com";
//! const DEEPSEEK_MODEL: &'static str = "deepseek-v4-flash";
//!
//! #[tokio::main]
//! async fn main() -> Result<(), Box<dyn std::error::Error>> {
//!     // Needs the `ferritls` cargo feature; leave it out if you install
//!     // your own rustls crypto provider. See the "TLS Crypto Provider"
//!     // section above.
//!     # #[cfg(feature = "ferritls")]
//!     openai_interface::rest::install_crypto_provider().ok();
//!
//!     let request = RequestBody {
//!         messages: vec![
//!             Message::system("You are a helpful assistant."),
//!             Message::user("Hello, how are you?"),
//!         ],
//!         model: DEEPSEEK_MODEL.to_string(),
//!         stream: Some(false),
//!         ..Default::default()
//!     };
//!
//!     // Send the request
//!     let chat_completion: ChatCompletion = request
//!         .get_response(&default_client(), DEEPSEEK_CHAT_URL, &RequestOptions::bearer("YOUR_API_KEY"))
//!         .await?;
//!     let text = chat_completion.choices[0]
//!         .message
//!         .content
//!         .as_deref()
//!         .unwrap();
//!     println!("{:?}", text);
//!     Ok(())
//! }
//! ```
//!
//! ## Streaming Chat Completion
//!
//! This example demonstrates how to handle streaming responses from the API. As with the non-streaming
//! example, all API parameters can be adjusted directly through the request struct.
//!
//! ```rust,no_run
//! use openai_interface::chat::create::request::{Message, RequestBody};
//! use openai_interface::chat::create::response::streaming::ChatCompletionChunk;
//! use openai_interface::rest::{RequestOptions, default_client, post::PostStream};
//! use futures_util::StreamExt;
//!
//! const DEEPSEEK_CHAT_URL: &'static str = "https://api.deepseek.com";
//! const DEEPSEEK_MODEL: &'static str = "deepseek-v4-flash";
//!
//! #[tokio::main]
//! async fn main() -> Result<(), Box<dyn std::error::Error>> {
//!     let request = RequestBody {
//!         messages: vec![
//!             Message::system("You are a helpful assistant."),
//!             Message::user("Who are you?"),
//!         ],
//!         model: DEEPSEEK_MODEL.to_string(),
//!         stream: Some(true),
//!         ..Default::default()
//!     };
//!
//!     // Send the request
//!     let mut response_stream = request
//!         .get_stream_response(&default_client(), DEEPSEEK_CHAT_URL, &RequestOptions::bearer("YOUR_API_KEY"))
//!         .await?;
//!
//!     let mut message = String::new();
//!
//!     while let Some(chunk_result) = response_stream.next().await {
//!         let chunk: ChatCompletionChunk = chunk_result?;
//!         if let Some(content) = chunk.choices[0].delta.content.as_deref() {
//!             println!("content chunk: {}", content);
//!             message.push_str(content);
//!         }
//!     }
//!
//!     println!("complete message: {}", message);
//!     Ok(())
//! }
//! ```
//!
//! # Musl Build
//!
//! This crate is designed to work with musl libc, making it suitable for
//! lightweight deployments in containerized environments. TLS is provided by
//! rustls with a pure-Rust crypto backend, so OpenSSL does not need to be
//! built from source. See [`rest::install_crypto_provider`] for how the
//! backend is selected at runtime.
//!
//! To build for musl:
//! ```bash
//! rustup target add x86_64-unknown-linux-musl
//! cargo build --target x86_64-unknown-linux-musl
//! ```

/// Implements `FromStr` for JSON response types by deserializing them with
/// `serde_json`, mapping any parse failure to
/// [`OapiError::DeserializationError`](crate::errors::OapiError::DeserializationError).
macro_rules! impl_from_str {
    ($($target:ty),* $(,)?) => {
        $(
            impl std::str::FromStr for $target {
                type Err = crate::errors::OapiError;

                fn from_str(content: &str) -> Result<Self, Self::Err> {
                    serde_json::from_str(content).map_err(|e| {
                        crate::errors::OapiError::DeserializationError(e.to_string())
                    })
                }
            }
        )*
    };
}

pub(crate) use impl_from_str;

/// Defines a wire-fidelity string enum: a closed set of unit variants with
/// explicit wire names, plus an `Unknown(String)` catch-all.
///
/// Upstream gateways routinely invent values the official API never
/// documented (new finish reasons, roles, service tiers). A closed enum
/// turns any of them into a hard deserialization error that kills the whole
/// chunk; the `Unknown(String)` variant keeps the chunk alive and preserves
/// the original string, so proxies can serialize it back out unchanged.
///
/// Also derives `Clone`, `PartialEq`, `Eq`, `Hash`, and implements
/// `AsRef<str>` / `Display` (the wire representation, original string for
/// `Unknown`).
macro_rules! wire_string_enum {
    (
        $(#[$meta:meta])*
        $vis:vis enum $name:ident {
            $($(#[$vmeta:meta])* $variant:ident => $wire:literal),* $(,)?
        }
    ) => {
        $(#[$meta])*
        #[derive(Debug, Clone, PartialEq, Eq, Hash, serde::Serialize, serde::Deserialize)]
        $vis enum $name {
            $(
                $(#[$vmeta])*
                #[serde(rename = $wire)]
                $variant,
            )*
            /// Any other value emitted by the backend, preserved verbatim.
            #[serde(untagged)]
            Unknown(String),
        }

        impl $name {
            /// The wire representation of this value.
            #[must_use]
            pub fn as_str(&self) -> &str {
                match self {
                    $(Self::$variant => $wire,)*
                    Self::Unknown(raw) => raw,
                }
            }
        }

        impl AsRef<str> for $name {
            fn as_ref(&self) -> &str {
                self.as_str()
            }
        }

        impl std::fmt::Display for $name {
            fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
                f.write_str(self.as_str())
            }
        }
    };
}

pub(crate) use wire_string_enum;

/// Defines a top-level JSON request-body struct with the standard
/// [`extra_body_map`](struct@Self::extra_body_map) catch-all field appended.
///
/// The flattened `Option<serde_json::Map<String, serde_json::Value>>` field
/// merges unknown keys into the serialized JSON on the way out and captures
/// unknown keys on the way in (so proxies can forward fields this crate
/// does not model losslessly). Two forms are supported, with and without a
/// lifetime parameter.
macro_rules! request_body {
    (
        $(#[$meta:meta])*
        $vis:vis struct $name:ident {
            $($(#[$fmeta:meta])* $fvis:vis $field:ident : $ftype:ty),* $(,)?
        }
    ) => {
        $(#[$meta])*
        $vis struct $name {
            $($(#[$fmeta])* $fvis $field: $ftype,)*
            /// Additional JSON properties flattened into the request body,
            /// for fields not covered by the typed struct.
            #[serde(flatten, default, skip_serializing_if = "Option::is_none")]
            pub extra_body_map: Option<serde_json::Map<String, serde_json::Value>>,
        }
    };
    (
        $(#[$meta:meta])*
        $vis:vis struct $name:ident<$lt:lifetime> {
            $($(#[$fmeta:meta])* $fvis:vis $field:ident : $ftype:ty),* $(,)?
        }
    ) => {
        $(#[$meta])*
        $vis struct $name<$lt> {
            $($(#[$fmeta])* $fvis $field: $ftype,)*
            /// Additional JSON properties flattened into the request body,
            /// for fields not covered by the typed struct.
            #[serde(flatten, default, skip_serializing_if = "Option::is_none")]
            pub extra_body_map: Option<serde_json::Map<String, serde_json::Value>>,
        }
    };
}

pub(crate) use request_body;

// The two most-referenced types, available without the module path
// (`openai_interface::errors::OapiError` also works).
pub use errors::{ApiError, OapiError};

pub mod audio;
pub mod batches;
pub mod chat;
pub mod completions;
pub mod containers;
pub mod conversations;
pub mod embeddings;
pub mod errors;
pub mod evals;
pub mod files;
pub mod fine_tuning;
pub mod images;
pub mod models;
pub mod moderations;
pub mod pagination;
pub mod realtime;
pub mod responses;
pub mod rest;
pub mod uploads;
pub mod vector_stores;
#[cfg(feature = "vllm")]
pub mod vllm;

#[cfg(test)]
mod tests {
    use crate::chat::create::request::{Message, RequestBody};
    use crate::chat::create::response::streaming::ChatCompletionChunk;
    use crate::rest::{
        RequestOptions, default_client,
        post::{PostNoStream, PostStream},
    };
    use futures_util::StreamExt;

    const DEEPSEEK_CHAT_URL: &str = "https://api.deepseek.com";
    const DEEPSEEK_MODEL: &str = "deepseek-v4-flash";

    fn deepseek_api_key() -> Option<String> {
        std::env::var("DEEPSEEK_API_KEY")
            .ok()
            .map(|key| key.trim().to_string())
            .filter(|key| !key.is_empty())
    }

    #[tokio::test]
    async fn test_no_streaming() -> Result<(), Box<dyn std::error::Error>> {
        let Some(api_key) = deepseek_api_key() else {
            println!("Skipping: set DEEPSEEK_API_KEY to run this test");
            return Ok(());
        };

        let request = RequestBody {
            messages: vec![
                Message::system("You are a helpful assistant."),
                Message::user("Hello, how are you?"),
            ],
            model: DEEPSEEK_MODEL.to_string(),
            stream: Some(false),
            ..Default::default()
        };

        // Send the request
        let chat_completion: crate::chat::create::response::no_streaming::ChatCompletion = request
            .get_response(
                &default_client(),
                DEEPSEEK_CHAT_URL,
                &RequestOptions::bearer(&api_key),
            )
            .await?;
        let text = chat_completion.choices[0]
            .message
            .content
            .as_deref()
            .unwrap();
        println!("lib::test_no_streaming message: {}", text);
        Ok(())
    }

    #[tokio::test]
    async fn test_streaming() -> Result<(), Box<dyn std::error::Error>> {
        let Some(api_key) = deepseek_api_key() else {
            println!("Skipping: set DEEPSEEK_API_KEY to run this test");
            return Ok(());
        };

        let request = RequestBody {
            messages: vec![
                Message::system("You are a helpful assistant."),
                Message::user("Who are you?"),
            ],
            model: DEEPSEEK_MODEL.to_string(),
            stream: Some(true),
            ..Default::default()
        };

        // Send the request
        let mut response_stream = request
            .get_stream_response(
                &default_client(),
                DEEPSEEK_CHAT_URL,
                &RequestOptions::bearer(&api_key),
            )
            .await?;

        let mut message = String::new();

        while let Some(chunk_result) = response_stream.next().await {
            let chunk: ChatCompletionChunk = chunk_result?;
            if let Some(content) = chunk.choices[0].delta.content.as_deref() {
                println!("lib::test_streaming message: {}", content);
                message.push_str(content);
            }
        }

        println!("lib::test_streaming message: {}", message);
        Ok(())
    }
}