llm-connector 1.1.7

A simple protocol adapter layer for LLM services.

Installation • Usage

Overview

llm-connector is a lightweight protocol adapter for connecting various LLM services. It handles protocol conversion and standardization without caring about specific API endpoints.

• Zero Hardcoded URLs: No default endpoints. You provide the base_url
• Protocol Agnostic: Support multiple LLM providers with unified interface
• Lightweight & Standalone: No external configuration management or database dependencies

Protocol Architecture

The src/protocols/ module uses adapter pattern to convert different vendor APIs into unified internal interface:

• formats/: Protocol-agnostic data structures
• adapters/: Vendor-specific protocol adapters
• common/: Shared utility functions

Installation

Minimum Rust Version: 1.85+

[dependencies]
llm-connector = "1.0.2"
tokio = { version = "1", features = ["full"] }

Usage

Basic Chat

use llm_connector::{LlmClient, types::{ChatRequest, Message, Role}};

let client = LlmClient::openai("sk-...", "https://api.openai.com/v1")?;

let request = ChatRequest::new("gpt-4o")
    .add_message(Message::user("What is the speed of light?"));

let response = client.chat(&request).await?;
println!("{}", response.content);

Streaming Response

use llm_connector::{LlmClient, types::{ChatRequest, Message, Role}};
use futures_util::StreamExt;

let client = LlmClient::anthropic("sk-ant-...", "https://api.anthropic.com")?;
let request = ChatRequest::new("claude-3-5-sonnet-20240620").with_stream(true);

let mut stream = client.chat_stream(&request).await?;
while let Some(chunk) = stream.next().await {
    if let Some(content) = chunk?.get_content() {
        print!("{}", content);
    }
}

Builder Pattern

let client = LlmClient::builder()
    .openai("sk-...")
    .base_url("https://api.deepseek.com") // Required
    .timeout(60)
    .build()?;

Advanced Features

Reasoning Models

Support for reasoning models like OpenAI o1/o3 and Claude 3.7 Sonnet.

use llm_connector::types::ReasoningEffort;

let request = ChatRequest::new("claude-3-7-sonnet-20250219")
    .add_message(Message::user("Solve this logic puzzle..."))
    .with_thinking_budget(16000) // Enable thinking with 16k token budget
    .with_max_tokens(20000);     // Ensure max_tokens > thinking_budget

let response = client.chat(&request).await?;

File Upload

Support for uploading local files (images, PDFs, etc.) with automatic Base64 encoding and MIME type detection.

use llm_connector::types::MessageBlock;

let request = ChatRequest::new("claude-3-5-sonnet")
    .add_message(Message::user("Analyze this document"))
    .add_message_block(MessageBlock::from_file_path("report.pdf")?);

let response = client.chat(&request).await?;

License

MIT