pondus 0.2.0

Opinionated AI model benchmark aggregator
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124

# pondus

Opinionated AI model benchmark aggregator.

[![crates.io](https://img.shields.io/crates/v/pondus.svg)](https://crates.io/crates/pondus)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

## What it does

Aggregates AI model benchmark data from 8 trusted sources into a unified JSON schema. Designed for AI agents (Claude Code, etc.) to consume programmatically. Caches results for 24h to avoid rate limiting.

## Sources

| Source | Type | Data |
|--------|------|------|
| Artificial Analysis | REST API | Speed, quality, pricing metrics |
| LM Arena (LMSYS) | Community JSON | ELO ratings from human preferences |
| SWE-bench | GitHub JSON | Code generation resolve rates |
| SWE-rebench | agent-browser scrape | Code generation resolve rates (rebench variant) |
| Aider | GitHub YAML | Polyglot coding benchmark pass rates |
| LiveBench | HuggingFace | Multi-domain benchmark scores |
| Terminal-Bench | HuggingFace | Terminal/CLI task completion |
| SEAL | agent-browser scrape | Scale AI evaluation scores |

> **Note:** Sources marked "agent-browser scrape" require the `agent-browser` CLI installed separately. All other sources work out of the box.

## Installation

```
cargo install pondus
```

## Usage

```bash
pondus rank                     # rank all models (default command)
pondus                          # same as `pondus rank`
pondus rank --top 10            # top 10 only
pondus check claude-opus-4.6    # check one model across all sources
pondus compare gpt-5.2 claude-opus-4.6  # head-to-head comparison
pondus sources                  # show source status
pondus refresh                  # clear cache and re-fetch
```

### Global Flags

| Flag | Description |
|------|-------------|
| `--format json|table|markdown` | Output format (default: json) |
| `--refresh` | Bypass cache for this run |

## Configuration

Config location: `~/.config/pondus/config.toml`

```toml
[cache]
ttl_hours = 24

[alias]
path = "models.toml"  # relative to config dir, or absolute path

[sources.artificial_analysis]
api_key = "your-key"  # optional, for AA source

[sources.agent_browser]
path = "agent-browser"  # path to agent-browser CLI
```

## Model Aliases

Different benchmarks use different naming conventions. `models.toml` maps canonical model names to source-specific variants:

```toml
[claude-opus-4_6]
canonical = "claude-opus-4.6"
aliases = [
  "Claude Opus 4.6",
  "claude-opus-4-6",
  "anthropic/claude-opus-4.6",
  "Opus 4.6",
]
```

When you run `pondus check opus-4.6`, pondus resolves the alias to the canonical name and matches it across all sources regardless of how each source names the model. PRs welcome to add new models.

## Output Format

Default JSON output:

```json
{
  "timestamp": "2026-02-27T10:30:00Z",
  "query": { "query_type": "rank" },
  "sources": [
    {
      "source": "arena",
      "status": "ok",
      "scores": [
        { "model": "gpt-5.2", "rank": 1, "metrics": { "elo": 1350 } }
      ]
    }
  ]
}
```

## Contributing

- **Add a model:** Add an entry to `models.toml` with canonical name and known aliases
- **Add a source:** Implement the `Source` trait in `src/sources/`

PRs welcome.

## License

MIT

## Sister Tools

Part of a family of AI-augmented CLI tools:

- [lustro](https://github.com/terry-li-hm/lustro) — AI news aggregator
- [consilium](https://github.com/terry-li-hm/consilium) — Multi-model deliberation
- [hexis](https://github.com/hexis-framework/hexis) — Meta-cognitive framework