Crate llmrust 

llmrust

Call multiple LLM APIs with one unified Rust interface.

Installation

# Cargo.toml

# LLM client only (recommended for most users)
[dependencies]
llmrust = "0.1"

# With the built-in HTTP proxy server
[dependencies]
llmrust = { version = "0.1", features = ["proxy"] }

Quick Start

use llmrust::{LmrsClient, Message};

#[tokio::main]
async fn main() {
    let llm = LmrsClient::new();

    // OpenAI
    llm.set_openai("sk-...").await;
    let resp = llm.chat("openai/gpt-4o", "Hello!").await.unwrap();
    println!("{}", resp.content);

    // Anthropic
    llm.set_anthropic("sk-ant-...").await;
    let resp = llm.chat("anthropic/claude-sonnet-4-20250514", "Hello!").await.unwrap();
    println!("{}", resp.content);

    // DeepSeek
    llm.set_deepseek("sk-...").await;
    let resp = llm.chat("deepseek/deepseek-chat", "Hello!").await.unwrap();
    println!("{}", resp.content);
}

Logging

llmrust emits structured tracing events for provider registration, request lifecycle, proxy requests, retries, router failover, and upstream API errors. It does not install a global subscriber; applications remain in control of how logs are collected.

The built-in events include operational fields such as provider, model, HTTP status, retry attempt, and router group. They intentionally avoid logging API keys, request bodies, prompts, message content, and response text.

// In your application, not inside llmrust:
tracing_subscriber::fmt()
    .with_env_filter("llmrust=debug")
    .init();

Re-exports

pub use pricing::ModelPricing;

pub use providers::retry::RetryProvider;

pub use providers::LlmError;

pub use providers::Provider;

pub use providers::ProviderConfig;

pub use providers::Result;

pub use router::Router;

pub use router::RoutingStrategy;

pub use types::ChatRequest;

pub use types::ChatResponse;

pub use types::Content;

pub use types::ContentPart;

pub use types::Embedding;

pub use types::EmbeddingRequest;

pub use types::EmbeddingResponse;

pub use types::EmbeddingUsage;

pub use types::FinishReason;

pub use types::FunctionCall;

pub use types::FunctionDef;

pub use types::ImageUrl;

pub use types::LogProbs;

pub use types::Message;

pub use types::ResponseFormat;

pub use types::Role;

pub use types::StreamChunk;

pub use types::TokenLogProb;

pub use types::Tool;

pub use types::ToolCall;

pub use types::ToolChoice;

pub use types::TopLogProb;

pub use types::Usage;

Modules

prelude

Convenient re-exports for common llmrust usage.

pricing

Cost estimation for token Usage.

providers

Provider trait and unified LLM client.

router

Multi-deployment routing with automatic fallback and optional passive cooldown.

types

Core types for llmrust — unified LLM API interface.

Structs

LmrsClient

The unified LLM client. Routes provider/model strings to the right backend.