Crate minillmlib 

MiniLLMLib - A minimalist Rust library for LLM interactions

This library provides a clean, async-first interface for interacting with Large Language Models via HTTP APIs (OpenRouter, OpenAI, etc.).

Features

• Conversation Trees: ChatNode provides a tree-based conversation structure supporting branching dialogues and conversation history
• Streaming Support: First-class support for streaming completions via SSE
• Multimodal: Support for images and audio in messages
• Multiple wires: One Provider trait owns the full dialect (endpoint, auth, request body, response/stream envelope). Ships OpenAI/OpenRouter, native Anthropic (/v1/messages, content[]), and a generic compatible provider; a custom enterprise API is a small impl Provider.
• Subscription auth: a Claude Pro/Max OAuth token works via Auth::BearerToken (see GeneratorInfo::claude_subscription); cost is a token-count ESTIMATE through TokenPrice.
• Cost Tracking: Per-provider usage & cost accounting behind the Provider trait; enforced tracking via CompletionContext, with honest CostResolution (Resolved/Unpriced/Unknown, never a fake $0)
• JSON Repair: Robust handling of malformed JSON from LLM outputs
• Async/Parallel: Built on Tokio for high-performance async operations

Quick Start

use minillmlib::{ChatNode, GeneratorInfo};

#[tokio::main]
async fn main() -> minillmlib::error::Result<()> {
    // Create a generator for OpenRouter
    let generator = GeneratorInfo::openrouter("anthropic/claude-3.5-sonnet");

    // Start a conversation
    let root = ChatNode::root("You are a helpful assistant.");
    let response = root.chat("Hello!", &generator).await?;

    println!("Assistant: {}", response.text().unwrap_or_default());
    Ok(())
}

Re-exports

pub use chat_node::format_conversation;

pub use chat_node::pretty_messages;

pub use chat_node::ChatNode;

pub use chat_node::ConversationBuilder;

pub use chat_node::PrettyPrintConfig;

pub use chat_node::ThreadData;

pub use error::MiniLLMError;

pub use error::Result;

pub use generator::CompletionParameters;

pub use generator::GeneratorInfo;

pub use generator::NodeCompletionParameters;

pub use generator::ProviderSettings;

pub use generator::ReasoningConfig;

pub use json_repair::loads;

pub use json_repair::repair_json;

pub use json_repair::JsonValue;

pub use json_repair::RepairOptions;

pub use message::AudioData;

pub use message::AudioInput;

pub use message::ContentPart;

pub use message::ImageData;

pub use message::ImageUrl;

pub use message::Media;

pub use message::MediaData;

pub use message::Message;

pub use message::MessageContent;

pub use message::Role;

pub use message::VideoData;

pub use message::VideoUrl;

pub use provider::resolve_claude_subscription_auth;

pub use provider::AnthropicProvider;

pub use provider::AppIdentity;

pub use provider::Auth;

pub use provider::CompletionResponse;

pub use provider::CostCallback;

pub use provider::CostInfo;

pub use provider::CostOutcome;

pub use provider::CostResolution;

pub use provider::GenericProvider;

pub use provider::LLMClient;

pub use provider::OpenAiProvider;

pub use provider::OpenRouterProvider;

pub use provider::PostStreamCtx;

pub use provider::Provider;

pub use provider::StreamChunk;

pub use provider::StreamingCompletion;

pub use provider::TokenPrice;

pub use provider::Usage;

pub use tracking::AsyncCostCallback;

pub use tracking::CompletionContext;

pub use tracking::CompletionMeta;

pub use tracking::TrackedStream;

pub use utils::configure_logging;

pub use utils::extract_json;

pub use utils::extract_json_value;

pub use utils::pretty_json;

pub use utils::to_dict;

pub use utils::LogLevel;

Modules

chat_node

ChatNode - Core conversation tree structure

error

Error types for MiniLLMLib

generator

Generator configuration types for LLM interactions

json_repair

JSON Repair module - fixes malformed JSON from LLM outputs

message

Message types for LLM conversations

provider

LLM Provider implementations

tracking

CompletionContext - Enforced cost tracking wrapper for LLM completions

utils

Utility functions for MiniLLMLib

Functions

init

Initialize the library with default settings

init_with_logging

Initialize with a specific log level