litellm-rs 0.9.0

A high-performance AI Gateway written in Rust, providing OpenAI-compatible APIs with intelligent routing, load balancing, and enterprise features
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
//! Public completion streams use the canonical provider response types.
//!
//! Tool argument fragments, usage, thinking, audio, and finish reasons are
//! preserved without a second response schema or a lossy conversion.

use futures::stream::BoxStream;

pub use crate::core::types::responses::{
    ChatChunk as CompletionChunk, ChatDelta as StreamDelta, ChatStreamChoice as StreamChoice,
};

pub type CompletionStream =
    BoxStream<'static, Result<CompletionChunk, crate::utils::error::gateway_error::GatewayError>>;