llm-connector 0.7.0

High-performance Rust library for unifying LLM providers behind one type-safe API.

Installation • Usage • Documentation • Contributing • Providers

Philosophy

• One API, many providers.
• Unified response types for chat + streaming.
• Minimal configuration: explicit base_url, timeout, proxy.

Key Features

• Provider-agnostic client API (LlmClient)
• Universal streaming with a unified StreamingResponse
• Function calling / tools (OpenAI-compatible)
• Multi-modal messages (text + images)
• Reasoning model normalization
• Per-request overrides: API key, base URL, and headers for multi-tenant / gateway use

Installation

MSRV: Rust 1.85+ (Rust 2024 edition)

[dependencies]
llm-connector = "0.6.1"
tokio = { version = "1", features = ["full"] }

Usage

Chat

use llm_connector::{LlmClient, types::{ChatRequest, Message, Role}};

let client = LlmClient::openai("sk-...")?;
let request = ChatRequest {
    model: "gpt-4".to_string(),
    messages: vec![Message::text(Role::User, "Hello!")],
    ..Default::default()
};

let response = client.chat(&request).await?;
println!("{}", response.content);

Streaming

use llm_connector::{LlmClient, types::{ChatRequest, Message, Role}};
use futures_util::StreamExt;

let client = LlmClient::openai("sk-...")?;
let request = ChatRequest {
    model: "gpt-4".to_string(),
    messages: vec![Message::text(Role::User, "Tell me a story")],
    stream: Some(true),
    ..Default::default()
};

let mut stream = client.chat_stream(&request).await?;
while let Some(chunk) = stream.next().await {
    if let Some(content) = chunk?.get_content() {
        print!("{}", content);
    }
}

Per-Request Overrides (Multi-Tenant / Gateway)

Override API key, base URL, or headers per request without creating a new client:

let request = ChatRequest::new("gpt-4")
    .add_message(Message::user("Hello"))
    .with_api_key("tenant-key")
    .with_base_url("https://proxy.example.com/v1")
    .with_header("X-Trace-Id", "trace-123");

let response = client.chat(&request).await?;

Documentation

• Providers
• Streaming
• Tools / Function Calling
• Multi-modal
• Architecture
• Changelog

Contributing

Contributions welcome! See https://llmconn.com/guide/contributing for guidelines.

License

MIT