Turbine LLM

One interface, all LLMs - A unified Rust library for calling multiple LLM providers with growing model support.

🚀 Switch between OpenAI, Anthropic, Gemini, and Groq with minimal code changes. Perfect for building AI applications that need provider flexibility.

Features

Unified API: Single interface for multiple LLM providers
Simple & Clean: Minimal, straightforward code - no complexity
Text & JSON Output: Support for both text and structured JSON responses
Async/Await: Built with Tokio for high-performance async operations
Type-Safe: Full Rust type safety with proper error handling
Growing Support: New providers and models added regularly

Why Turbine?

Provider Independence: Easily switch providers or use multiple simultaneously
Consistent Interface: Same code works across all providers
Production Ready: Proper error handling, async support, comprehensive docs
Actively Maintained: Regular updates with new models and providers

Supported Providers

Currently integrated:

✅ OpenAI (GPT-4, GPT-3.5, etc.)
✅ Anthropic (Claude 3.5 Sonnet, Haiku, etc.)
✅ Google Gemini (Gemini 2.0, 1.5, etc.)
✅ Groq (Llama, Mixtral, etc.)

Coming soon:

🔜 Cohere
🔜 Mistral AI
🔜 Perplexity

New providers and models added regularly. Check CHANGELOG.md for updates.

Installation

Add this to your Cargo.toml:

[dependencies]
turbine-llm = "0.2"
tokio = { version = "1", features = ["full"] }

Quick Start

1. Simplified API (Recommended) 🚀

The easiest way to get started - just pass a model string:

use turbine_llm::TurbineClient;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Create client with model string - provider is automatically detected
    let client = TurbineClient::from_model("openai/gpt-4o-mini")?;

    // Simple one-liner to send a message
    let response = client.send("What is Rust?").await?;
    println!("{}", response.content);

    Ok(())
}

Supported model string formats:

Explicit provider: "openai/gpt-4o-mini", "google/gemini-flash", "anthropic/claude-3-5-sonnet"
Inferred from name: "gpt-4o", "claude-3-5-sonnet", "gemini-flash", "llama-3.3-70b"

If the API key isn't in your environment, you'll be prompted to enter it interactively.

With system prompt:

let response = client.send_with_system(
    "You are a Rust expert",
    "Explain ownership in one sentence"
).await?;

2. Traditional API (Full Control)

For advanced use cases, use the traditional builder pattern:

use turbine_llm::{TurbineClient, LLMRequest, Message, Provider};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Set API key in environment first
    // export OPENAI_API_KEY="your-key"

    let client = TurbineClient::new(Provider::OpenAI)?;

    let request = LLMRequest::new("gpt-4o-mini")
        .with_system_prompt("You are a helpful assistant.")
        .with_message(Message::user("What is Rust?"))
        .with_max_tokens(100);

    let response = client.send_request(&request).await?;
    println!("{}", response.content);

    Ok(())
}

3. JSON Output

Request structured JSON from any provider:

use turbine_llm::{TurbineClient, LLMRequest, Message, OutputFormat, Provider};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = TurbineClient::new(Provider::Anthropic)?;

    let request = LLMRequest::new("claude-3-5-sonnet-20241022")
        .with_system_prompt("Return data as JSON.")
        .with_message(Message::user("Info about Paris with keys: name, country, population"))
        .with_output_format(OutputFormat::Json);

    let response = client.send_request(&request).await?;
    println!("{}", response.content);

    Ok(())
}

4. Multi-turn Conversations

let request = LLMRequest::new("gemini-2.0-flash-exp")
    .with_messages(vec![
        Message::user("Hello! My name is Alice."),
        Message::assistant("Hello Alice! Nice to meet you."),
        Message::user("What's my name?"),
    ]);

API Reference

TurbineClient

Simplified Constructor (Recommended)

// Automatic provider detection from model string
let client = TurbineClient::from_model("openai/gpt-4o-mini")?;

// Simple message sending
let response = client.send("Your message").await?;

// With system prompt
let response = client.send_with_system("System prompt", "User message").await?;

With Explicit API Key

Pass the API key directly instead of using environment variables:

// With provider enum
let client = TurbineClient::new_with_key(Provider::OpenAI, "sk-xxx");

// With model string
let client = TurbineClient::from_model_with_key("openai/gpt-4o", "sk-xxx")?;
let response = client.send("Hello").await?;

Traditional Constructor

let client = TurbineClient::new(Provider::OpenAI)?;
let response = client.send_request(&request).await?;

Provider

Select which LLM provider to use (for traditional API):

pub enum Provider {
    OpenAI,      // Requires OPENAI_API_KEY
    Anthropic,   // Requires ANTHROPIC_API_KEY
    Gemini,      // Requires GEMINI_API_KEY
    Groq,        // Requires GROQ_API_KEY
}

Note: When using from_model(), the provider is automatically detected from the model string.

LLMRequest Builder

Construct requests with optional parameters:

LLMRequest::new("model-name")
    .with_system_prompt("System prompt")        // Optional
    .with_message(Message::user("Query"))       // Add single message
    .with_messages(vec![...])                   // Add multiple messages
    .with_max_tokens(1000)                      // Optional, default: 1024
    .with_temperature(0.7)                      // Optional, 0.0-2.0
    .with_top_p(0.9)                            // Optional
    .with_output_format(OutputFormat::Json)     // Text (default) or Json

Message Helpers

Message::user("User message")
Message::assistant("Assistant message")
Message::system("System message")

Model Examples

OpenAI

gpt-4o - Latest GPT-4 Omni
gpt-4o-mini - Faster, cost-effective
gpt-3.5-turbo - Fast and efficient

Anthropic

claude-3-5-sonnet-20241022 - Most capable
claude-3-5-haiku-20241022 - Fast and affordable

Gemini

gemini-2.0-flash-exp - Latest experimental
gemini-1.5-pro - Production ready

Groq

llama-3.3-70b-versatile - Powerful Llama model
mixtral-8x7b-32768 - Mixtral with large context

Error Handling

match client.send_request(&request).await {
    Ok(response) => println!("{}", response.content),
    Err(e) => eprintln!("Error: {}", e),
}

Examples

Run the included examples:

# Simplified API (recommended for beginners)
cargo run --example simple_usage

# Basic text generation
cargo run --example text_generation

# JSON output
cargo run --example json_output

# Multi-turn conversation
cargo run --example conversation

Documentation

API Documentation
Context7 Docs - AI-powered documentation
Examples
Changelog
Contributing

Troubleshooting

API Key Not Found

Error: API key not found for provider: OpenAI

Solution: Make sure the environment variable is set:

export OPENAI_API_KEY="your-key-here"

Model Not Found

Different providers use different model names. Check the Model Examples section for correct model identifiers.

Rate Limiting

If you hit rate limits, implement exponential backoff or switch providers temporarily.

Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

License

Licensed under either of:

Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
MIT license (LICENSE-MIT or http://opensource.org/licenses/MIT)

at your option.

Acknowledgments

Developed with ❤️ by the Rust community and sponsored by Renaiss AI.

turbine-llm 0.2.2