Skip to main content

evalhub_query/
lib.rs

1//! The evalhub query language: a small, typed filter DSL whose vocabulary is
2//! projected from the record schema.
3//!
4//! `POST /cards/query` and `POST /evals/query` accept a JSON query
5//! (envelope in `evalhub_schema::query`). This crate turns that JSON into
6//! something a backend can execute, in three stages, none of which knows
7//! SQL:
8//!
9//! ```text
10//! JSON ──▶ grammar::parse     structure: and / or / not / leaf { path, op, value | match }
11//!      ──▶ typecheck::check   every path exists, has a type, and the op fits it
12//!      ──▶ ir::Query          backend-independent tree the store compiles to SQL
13//! ```
14//!
15//! A query that exercises every construct:
16//!
17//! ```json
18//! {
19//!   "where": {"and": [
20//!     {"path": "model.id", "op": "eq", "value": "qwen3.6-32b"},
21//!     {"path": "generation.temperature", "op": "lte", "value": 0.2},
22//!     {"path": "results", "op": "any", "match": {"metric": "core/pass_rate", "value": {"op": "gte", "value": 0.5}}},
23//!     {"path": "relations", "op": "any", "match": {"type": "core/uses_eval", "to": "alice/single2-k4"}},
24//!     {"path": "fingerprint.generation", "op": "eq", "value": "…"},
25//!     {"path": "ext.alice/qwen-loop.rung", "op": "eq", "value": 4}
26//!   ]},
27//!   "sort": [{"path": "results[core/pass_rate].value", "dir": "desc"}],
28//!   "limit": 50, "cursor": null,
29//!   "expand": ["relations", "badges", "fingerprints", "changed"],
30//!   "version": "latest"
31//! }
32//! ```
33//!
34//! # The grammar is the specification
35//!
36//! The filter AST is a `dsl-kit` enum, and the conformance check, the typed
37//! builder and the published JSON Schema of the `where` value all come from
38//! it. The operator vocabulary in particular is declared once, as
39//! [`grammar::ScalarOp`]: adding an operator is a variant plus its row in
40//! the type table, and no parser arm changes.
41//!
42//! What *is* hand-written is the lowering from the wire shape
43//! (`{"and": [...]}`) into the tagged shape `dsl-kit`'s JSON front end reads
44//! (`{"type": "And", …}`), which names the five node shapes. See the
45//! [`grammar`] module doc for why that is a shape concern rather than a
46//! vocabulary one.
47//!
48//! # The path table is projected from the schema
49//!
50//! Which paths are legal (`model.id`, `generation.temperature`,
51//! `results[core/pass_rate].value`, `fingerprint.model`,
52//! `ext.alice/qwen-loop.rung`) and what type each has is not written by
53//! hand either. At start-up [`typecheck`] walks the committed JSON Schema
54//! from `evalhub_schema` and builds the table: every leaf of a facet becomes
55//! a path with its JSON type, `fingerprint.{facet}` is added for each facet,
56//! `results` / `relations` / `attachments` become array paths that accept
57//! `any`, and registered `ext_schemas` contribute typed `ext.` paths. A key
58//! added to the schema is queryable in the same commit.
59//!
60//! # Type rules
61//!
62//! | Operator                            | Accepts                          |
63//! | ----------------------------------- | -------------------------------- |
64//! | `eq`, `ne`, `in`                    | any scalar                       |
65//! | `gt`, `gte`, `lt`, `lte`            | number, string (lexical)         |
66//! | `prefix`, `contains`                | string                           |
67//! | `exists`                            | any path                         |
68//! | `any` with `match`                  | array of objects                 |
69//!
70//! A mismatch is `422 type_mismatch`. A path not in the table is
71//! `422 unknown_path` — never an empty result, because an empty result for
72//! a typo looks like data.
73//!
74//! # Index awareness
75//!
76//! The type check also knows which paths are indexed: the sixteen facet
77//! keys the migration materialises as generated columns, the fingerprints,
78//! and the `results` / `relations` / `attachments` side tables, plus `ext`
79//! paths whose `ext_schema` is `applied`. Every other path — an
80//! unregistered `ext` key, and equally a facet key with no column of its
81//! own — accepts `eq` and `exists` only, which is what the GIN index over
82//! the JSON body answers; any other operator is `422 not_indexed` with a
83//! hint naming the registry. This keeps a query from silently becoming a
84//! sequential scan.
85//!
86//! # Sorting and cursors
87//!
88//! `sort` accepts the same paths. `results[{metric}].value` sorts in the
89//! direction the metric's registry entry declares (`lower_is_better`); an
90//! unregistered metric cannot be sorted on, because the hub does not know
91//! which way is up. Pagination is keyset (`sort key + version_id`), and the
92//! IR carries the decoded cursor so the store can emit the `WHERE (k, id) >
93//! (?, ?)` clause; the cursor's opaque, signed encoding is the server's
94//! business.
95//!
96//! # Modules
97//!
98//! - [`grammar`] — the `dsl-kit` definition and the parser it yields.
99//! - [`typecheck`] — the schema-derived path table and the type rules.
100//! - [`ir`] — the tree handed to the store.
101
102pub mod grammar;
103pub mod ir;
104pub mod typecheck;
105
106pub use grammar::{Filter, MatchTerm, ScalarOp, parse, request_schema};
107pub use ir::Query;
108pub use typecheck::{ExtSchema, Indexed, PathInfo, PathTable, check, compile};