LLMKit

Unified LLM API Client for Rust, Python, and Node.js

One interface for 100+ LLM providers. Rust core with bindings for every language.

                         ┌──────────────┐
                         │  Rust Core   │
                         └──────┬───────┘
          ┌──────────┬─────────┼─────────┬──────────┐
          ▼          ▼         ▼         ▼          ▼
      ┌───────┐  ┌───────┐ ┌───────┐ ┌───────┐ ┌───────┐
      │Python │  │ Node  │ │ WASM  │ │  Go   │ │  ...  │
      │  ✅   │  │  ✅   │ │ Soon  │ │ Soon  │ │       │
      └───────┘  └───────┘ └───────┘ └───────┘ └───────┘

📖 Documentation | 📝 Changelog | 🤝 Contributing | 🔒 Security

Quick Start

Rust

use llmkit::{LLMKitClient, Message, CompletionRequest};

let client = LLMKitClient::from_env()?;
let response = client.complete(
    CompletionRequest::new("anthropic/claude-sonnet-4-20250514", vec![Message::user("Hello!")])
).await?;
println!("{}", response.text_content());

Python

from llmkit import LLMKitClient, Message, CompletionRequest

client = LLMKitClient.from_env()
response = client.complete(CompletionRequest(
    model="openai/gpt-4o",
    messages=[Message.user("Hello!")]
))
print(response.text_content())

Node.js

import { JsLLMKitClient as LLMKitClient, JsMessage as Message, JsCompletionRequest as CompletionRequest } from 'llmkit'

const client = LLMKitClient.fromEnv()
const response = await client.complete(CompletionRequest.create('groq/llama-3.3-70b-versatile', [Message.user('Hello!')]))
console.log(response.textContent())

Why LLMKit?

• 🌍 100+ Providers - OpenAI, Anthropic, Google, AWS Bedrock, Azure, Groq, Mistral, and more
• 🔄 Unified API - Same interface for all providers with provider/model format
• ⚡ Streaming - First-class async streaming support
• 🛠️ Tool Calling - Abstract tool definitions with builder pattern
• 🧠 Extended Thinking - Reasoning mode across 4 providers (OpenAI, Anthropic, Google, DeepSeek)
• 🦀 Pure Rust - Memory-safe, high performance core

Features

Installation

Rust

[dependencies]
llmkit = { version = "0.1", features = ["anthropic", "openai"] }

Python

pip install llmkit-python

Node.js

npm install llmkit-node

Providers

See PROVIDERS.md for full list with environment variables.

Examples

Streaming

let mut stream = client.complete_stream(request).await?;
while let Some(chunk) = stream.next().await {
    if let Some(text) = chunk?.text() { print!("{}", text); }
}

Tool Calling

from llmkit import ToolBuilder

tool = ToolBuilder("get_weather").description("Get weather").string_param("city", "City name").build()
request = CompletionRequest(...).with_tools([tool])

Extended Thinking

const request = CompletionRequest.create('deepseek/deepseek-reasoner', messages).withThinking(5000)
const response = await client.complete(request)
console.log(response.thinkingContent()) // Reasoning process

For more examples, see examples/.

Building from Source

git clone https://github.com/yfedoseev/llmkit
cd llmkit
cargo build --release
cargo test

# Python bindings
cd llmkit-python && maturin develop

# Node.js bindings
cd llmkit-node && pnpm install && pnpm build

Documentation

• Getting Started (Rust)
• Getting Started (Python)
• Getting Started (Node.js)
• Model Registry - 11,000+ models with pricing

cargo doc --open

Contributing

We welcome contributions! See CONTRIBUTING.md for guidelines.

cargo build && cargo test && cargo fmt && cargo clippy -- -D warnings

License

Dual-licensed under MIT or Apache-2.0 at your option.

Built with 🦀 Rust | Status: Production Ready | v0.1.0 | 100+ Providers