llm_models_spider 0.1.11

Auto-updated registry of LLM model capabilities (vision, audio, etc.)
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140

# llm_models_spider

[![Crates.io](https://img.shields.io/crates/v/llm_models_spider.svg)](https://crates.io/crates/llm_models_spider)
[![Documentation](https://docs.rs/llm_models_spider/badge.svg)](https://docs.rs/llm_models_spider)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)

Auto-updated registry of LLM model capabilities, arena rankings, and pricing.

## Features

- **Zero runtime dependencies** - Pure compile-time lookups
- **Multi-source data** - Aggregated from [OpenRouter](https://openrouter.ai), [LiteLLM](https://github.com/BerriAI/litellm), and [Chatbot Arena](https://huggingface.co/datasets/mathewhe/chatbot-arena-elo)
- **Rich model profiles** - Capabilities, pricing, context windows, and arena rankings
- **Auto-updated** - Model lists fetched daily via GitHub Actions
- **Fallback patterns** - Unknown models matched against common naming patterns

## Installation

```toml
[dependencies]
llm_models_spider = "0.1"
```

## Usage

### Capability Checks

```rust
use llm_models_spider::{supports_vision, supports_audio, supports_video, supports_pdf, is_text_only};

// Vision support
assert!(supports_vision("gpt-4o"));
assert!(supports_vision("claude-3-sonnet-20240229"));
assert!(supports_vision("google/gemini-2.0-flash"));
assert!(!supports_vision("gpt-3.5-turbo"));

// Audio, video, PDF support
assert!(supports_audio("gemini-2.0-flash"));
assert!(supports_video("gemini-2.0-flash"));
assert!(supports_pdf("claude-3-sonnet"));

// Text-only check
assert!(is_text_only("gpt-3.5-turbo"));
assert!(is_text_only("llama-3-70b"));
```

### Model Profiles

Get full model intelligence — capabilities, pricing, context window, and arena ranking:

```rust
use llm_models_spider::{model_profile, arena_rank};

if let Some(profile) = model_profile("gpt-4o") {
    println!("Vision: {}", profile.capabilities.vision);
    println!("Max input: {} tokens", profile.max_input_tokens);
    println!("Max output: {} tokens", profile.max_output_tokens);

    if let Some(cost) = profile.pricing.input_cost_per_m_tokens {
        println!("Input cost: ${:.2}/M tokens", cost);
    }
    if let Some(cost) = profile.pricing.output_cost_per_m_tokens {
        println!("Output cost: ${:.2}/M tokens", cost);
    }
}

// Arena ranking (0.0-100.0, higher is better)
if let Some(rank) = arena_rank("gpt-4o") {
    println!("Arena rank: {:.1}/100", rank);
}
```

### Full Capabilities Lookup

```rust
use llm_models_spider::ModelCapabilities;

if let Some(caps) = ModelCapabilities::lookup("gemini-2.0-flash") {
    println!("Vision: {}", caps.vision);
    println!("Audio: {}", caps.audio);
    println!("Video: {}", caps.video);
    println!("File/PDF: {}", caps.file);
}
```

### Direct Model Info Access

For advanced use cases, access the raw `MODEL_INFO` array directly:

```rust
use llm_models_spider::MODEL_INFO;

// MODEL_INFO is sorted by name — use binary search for fast lookups
let idx = MODEL_INFO.binary_search_by(|e| e.name.cmp("gpt-4o"));
if let Ok(idx) = idx {
    let info = &MODEL_INFO[idx];
    println!("{}: vision={}, audio={}, arena={}",
        info.name, info.supports_vision, info.supports_audio, info.arena_overall);
}
```

## Data Sources

Model data is aggregated from three sources:

| Source | Data Provided |
|--------|--------------|
| [OpenRouter API](https://openrouter.ai/api/v1/models) | Modalities (vision, audio), pricing, context length |
| [LiteLLM](https://github.com/BerriAI/litellm) | Vision, audio, video, PDF support, pricing, context window |
| [Chatbot Arena](https://huggingface.co/datasets/mathewhe/chatbot-arena-elo) | Arena Elo scores (normalized to 0-100) |

**Merge strategy:**
- Capabilities: OR across sources (if any source says vision=true, it's vision)
- Pricing: Prefer LiteLLM (more granular), fall back to OpenRouter
- Context window: Take max across sources
- Arena scores: From Chatbot Arena only

## Fallback Patterns

For models not yet in any data source, the library falls back to pattern matching:

| Pattern | Examples |
|---------|----------|
| `gpt-4o`, `gpt-4-turbo`, `gpt-4-vision` | OpenAI vision models |
| `claude-3`, `claude-4` | Anthropic multimodal |
| `gemini-1.5`, `gemini-2` | Google Gemini |
| `qwen2-vl`, `qwen-vl` | Alibaba Qwen VL |
| `-vision`, `-vl` suffix | Generic vision indicators |

## Manual Update

To manually update the model lists:

```bash
cargo run --bin update-models --features updater
```

## License

MIT License - see [LICENSE](LICENSE) for details.