Expand description
The two Eval schema arms, and the one 1.0 → 2.0 conversion.
Release 0.2.0 accepts two majors of the Eval record schema (see
“Versioning of the schema itself” in evalhub_schema):
evalhub.eval/1.0 header + runs[] in one body (accepted by 0.2.0 only)
evalhub.eval/2.0 header only; runs are rows of the record, written apartdeclared_schema says which arm a body declares, so that the
validator, the store and the server agree on it without each parsing
schema for themselves. split_v1 turns a 1.0 body into what a 2.0
producer would have sent: a header and one run per runs[] element. It
is the only implementation of that conversion; the ingest of a 1.0 body
and the one-shot data migration of stored 1.0 versions both call it, so
a run converted at ingest and a run converted by the migration are the
same run, byte for byte, and have the same content hash.
§The conversion
1.0 body ──▶ header: body − runs, schema := "evalhub.eval/2.0"
└─▶ runs[i] ──▶ run: run_id, status (+ error / skip_reason),
started_at, ended_at, calls, artifacts (verbatim)
attachments the header entries calls/artifacts name
meta { "v0.2.0_migration": { "outcome": … } }
facets materialised from the headerNothing else in the header changes: its attachments[] keep the
entries the runs point at (they are copied, not moved), so the header
is exactly the 1.0 body minus runs under the 2.0 identifier, and its
content hash is the hash of that header posted as 2.0.
outcome mixed the execution state with a verdict; 2.0 keeps only the
state:
1.0 outcome | 2.0 status | detail |
|---|---|---|
pass | ok | |
fail | ok | |
error | error | error: { "kind": "unrecorded" } |
skipped | skipped | skip_reason: "unrecorded" |
A 1.0 run says that it errored or was skipped but never why, so the
conversion writes unrecorded rather than a guess. The dropped
outcome is kept verbatim in the run’s meta, under
V1_MIGRATION_META_KEY (v0.2.0_migration), so the verdict a 0.1.x
producer wrote is not lost; it is simply no longer the hub’s to
interpret. A later release that has to drop a meaning again keeps it the
same way, under its own v{version}_migration key.
The run’s facets are the header’s, copied onto the run
(crate::run::materialise); split_v1 owns materialisation on this
path, so the run write path’s own materialise, which is idempotent,
finds nothing left to copy.
A 1.0 run’s calls and artifacts[] named paths in the body’s
attachments[]. A 2.0 run points only into its own, so each header
entry a run names is copied into that run’s attachments[], in the
header’s order, once.
Structs§
Enums§
- Eval
Schema - Which major of the Eval schema a body declares.
Constants§
- UNRECORDED
- The value the conversion writes where 1.0 recorded no reason: the
error.kindof anerrorrun, theskip_reasonof askippedone. - V1_
MIGRATION_ META_ KEY - The
metakey under which the 1.0 → 2.0 conversion keeps what it dropped:{ "v0.2.0_migration": { "outcome": … } }.
Functions§
- declared_
schema - The Eval arm
bodydeclares in itsschemakey, orNonewhen the key is absent, not a string, or not an accepted Eval identifier. - split_
v1 - Split an
evalhub.eval/1.0body into a 2.0 header and its runs.