Module models 

AI Model Types and Structures

This module defines the core data structures for interacting with AI models across different providers. It includes request and response types for chat completions, embeddings, image generation, audio processing, and more.

Overview

The models module provides:

• Chat Completions: Conversational AI model interactions
• Embeddings: Text embedding generation and processing
• Image Generation: AI-powered image creation
• Audio Processing: Speech-to-text and text-to-speech
• Streaming Support: Real-time response streaming
• Tool Integration: Function calling and tool usage

Chat Completions

The primary interface for conversational AI models:

use ultrafast_models_sdk::{ChatRequest, ChatResponse, Message, Role};

let request = ChatRequest {
    model: "gpt-4".to_string(),
    messages: vec![
        Message::system("You are a helpful assistant."),
        Message::user("Hello, how are you?"),
    ],
    temperature: Some(0.7),
    max_tokens: Some(100),
    stream: Some(false),
    ..Default::default()
};

// Response includes choices, usage statistics, and metadata
let response: ChatResponse = client.chat_completion(request).await?;

Message Types

Different message roles for conversation context:

• System: Instructions and context for the AI
• User: User input and questions
• Assistant: AI responses and completions
• Tool: Function call results and tool responses

let system_msg = Message::system("You are a helpful assistant.");
let user_msg = Message::user("What's the weather like?");
let assistant_msg = Message::assistant("I don't have access to real-time weather data.");

Embeddings

Text embedding generation for semantic analysis:

use ultrafast_models_sdk::{EmbeddingRequest, EmbeddingResponse};

let request = EmbeddingRequest {
    model: "text-embedding-ada-002".to_string(),
    input: EmbeddingInput::StringArray(vec![
        "Hello, world!".to_string(),
        "How are you?".to_string(),
    ]),
    ..Default::default()
};

let response: EmbeddingResponse = client.embedding(request).await?;

Image Generation

AI-powered image creation from text prompts:

use ultrafast_models_sdk::{ImageRequest, ImageResponse};

let request = ImageRequest {
    prompt: "A beautiful sunset over mountains".to_string(),
    model: Some("dall-e-3".to_string()),
    n: Some(1),
    size: Some("1024x1024".to_string()),
    quality: Some("standard".to_string()),
    response_format: Some("url".to_string()),
    ..Default::default()
};

let response: ImageResponse = client.image_generation(request).await?;

Audio Processing

Speech-to-text and text-to-speech capabilities:

Audio Transcription

use ultrafast_models_sdk::{AudioRequest, AudioResponse};

let request = AudioRequest {
    file: audio_data, // Vec<u8> containing audio file
    model: "whisper-1".to_string(),
    language: Some("en".to_string()),
    prompt: Some("This is a conversation about technology.".to_string()),
    response_format: Some("text".to_string()),
    temperature: Some(0.0),
};

let response: AudioResponse = client.audio_transcription(request).await?;

Text-to-Speech

use ultrafast_models_sdk::{SpeechRequest, SpeechResponse};

let request = SpeechRequest {
    model: "tts-1".to_string(),
    input: "Hello, this is a test of text-to-speech.".to_string(),
    voice: "alloy".to_string(),
    response_format: Some("mp3".to_string()),
    speed: Some(1.0),
};

let response: SpeechResponse = client.text_to_speech(request).await?;

Streaming Support

Real-time response streaming for chat completions:

use ultrafast_models_sdk::{StreamChunk, StreamChoice};

let mut request = ChatRequest {
    model: "gpt-4".to_string(),
    messages: vec![Message::user("Tell me a story.")],
    stream: Some(true),
    ..Default::default()
};

let mut stream = client.stream_chat_completion(request).await?;

while let Some(chunk) = stream.next().await {
    match chunk {
        Ok(StreamChunk { choices, .. }) => {
            for choice in choices {
                if let Some(content) = choice.delta.content {
                    print!("{}", content);
                }
            }
        }
        Err(e) => eprintln!("Error: {}", e),
    }
}

Tool Integration

Function calling and tool usage capabilities:

use ultrafast_models_sdk::{Tool, Function, ToolChoice};

let request = ChatRequest {
    model: "gpt-4".to_string(),
    messages: vec![Message::user("What's the weather in New York?")],
    tools: Some(vec![Tool {
        tool_type: "function".to_string(),
        function: Function {
            name: "get_weather".to_string(),
            description: Some("Get current weather for a location".to_string()),
            parameters: serde_json::json!({
                "type": "object",
                "properties": {
                    "location": {"type": "string"}
                },
                "required": ["location"]
            }),
        },
    }]),
    tool_choice: Some(ToolChoice::Auto),
    ..Default::default()
};

Usage Statistics

Token usage tracking for cost and performance monitoring:

use ultrafast_models_sdk::Usage;

let usage = Usage {
    prompt_tokens: 100,
    completion_tokens: 50,
    total_tokens: 150,
};

println!("Cost: ${:.4}", (usage.total_tokens as f64 / 1000.0) * 0.03);

Provider Compatibility

All models are designed to be compatible with multiple AI providers:

• OpenAI: GPT-4, GPT-3.5, DALL-E, Whisper, TTS
• Anthropic: Claude-3, Claude-2, Claude Instant
• Google: Gemini Pro, Gemini Pro Vision, PaLM
• Azure OpenAI: Azure-hosted OpenAI models
• Ollama: Local and remote Ollama models
• Custom Providers: Extensible for any provider

Serialization

All models support JSON serialization with provider-specific optimizations:

• Optional Fields: Skip serialization of None values
• Custom Serialization: Provider-specific field mapping
• Validation: Automatic request validation
• Error Handling: Comprehensive error responses

Structs

AudioRequest

AudioResponse

ChatRequest

Chat completion request.

ChatResponse

Chat completion response.

Choice

A generated completion choice.

Delta

DeltaFunction

DeltaToolCall

Embedding

EmbeddingRequest

EmbeddingResponse

Function

FunctionCall

FunctionChoice

ImageData

ImageRequest

ImageResponse

Message

A message in a conversation.

Segment

SpeechRequest

SpeechResponse

StreamChoice

StreamChunk

Tool

ToolCall

Usage

Word

Enums

EmbeddingInput

Role

Role of a message in a conversation.

ToolChoice