Crate herolib_ai 

herolib-ai: AI client with multi-provider support.

This crate provides a unified AI client that supports multiple providers (Groq, OpenRouter, SambaNova) with automatic failover and verification support.

Features

• Multi-provider support: Automatically tries providers in order of preference
• OpenAI-compatible API: Works with any OpenAI-compatible endpoint
• Automatic failover: Falls back to alternative providers on failure
• Verification support: Retry with feedback until response passes validation
• Model abstraction: Use our model names, mapped to provider-specific IDs

Example

use herolib_ai::{AiClient, Model, PromptBuilderExt};

// Create client from environment variables
let client = AiClient::from_env();

// Simple chat
let response = client
    .prompt()
    .model(Model::Llama3_3_70B)
    .system("You are a helpful coding assistant")
    .user("Write a hello world in Rust")
    .execute_content()
    .unwrap();

println!("{}", response);

Verification Example

use herolib_ai::{AiClient, Model, PromptBuilderExt};

let client = AiClient::from_env();

let response = client
    .prompt()
    .model(Model::Qwen2_5Coder32B)
    .system("You are a JSON generator. Only output valid JSON.")
    .user("Generate a JSON object with name and age fields")
    .verify(|content| {
        // Verify the response is valid JSON
        match serde_json::from_str::<serde_json::Value>(content) {
            Ok(_) => Ok(()),
            Err(e) => Err(format!("Invalid JSON: {}. Please output only valid JSON.", e)),
        }
    })
    .max_retries(3)
    .execute_verified()
    .unwrap();

Environment Variables

Set API keys using environment variables:

• GROQ_API_KEY - Groq API key
• OPENROUTER_API_KEY - OpenRouter API key
• SAMBANOVA_API_KEY - SambaNova API key

Re-exports

pub use client::AiClient;

pub use client::chat_simple;

pub use embedding::EmbeddingModel;

pub use embedding::EmbeddingRequest;

pub use embedding::EmbeddingResponse;

pub use error::AiError;

pub use error::AiResult;

pub use model::Model;

pub use prompt::PromptBuilder;

pub use prompt::PromptBuilderExt;

pub use prompt::VerifyFn;

pub use provider::Provider;

pub use provider::ProviderConfig;

pub use transcription::TranscriptionModel;

pub use transcription::TranscriptionOptions;

pub use transcription::TranscriptionResponse;

pub use transcription::VerboseTranscriptionResponse;

pub use types::ChatCompletionRequest;

pub use types::ChatCompletionResponse;

pub use types::Message;

pub use types::Role;

pub use types::Usage;

Modules

client

AI client with multi-provider support.

embedding

Embedding model definitions and types.

error

Error types for the AI client.

model

AI model definitions.

prompt

Prompt builder with verification support.

provider

AI provider definitions.

transcription

Voice transcription model definitions and types.

types

OpenAI-compatible types for chat completions.