Expand description
The generic, RON-authored eval case schema + loader.
An EvalCase is one natural-language instruction, the world it runs against (the host’s Setup),
and the host’s Expect predicates its result must satisfy. The case container is domain-agnostic:
Setup and Expect are the host’s own types (e.g. AetherCore’s voxel EvalSetup / Expectation),
so eval-core stays free of any game/LLM specifics.
Cases are authored as .ron files and loaded by load_cases, which is fail-loud (a malformed
file is a hard error naming it) and deterministic (files are read in sorted order).
§One file, one OR many cases
A .ron case file may hold EITHER a single EvalCase or a list [EvalCase, EvalCase, …] of
related cases — see parse_cases_from_str, which load_cases uses per file. This lets an author
group a whole capability (say all the arithmetic checks) in one file without one-file-per-case
sprawl, while existing single-case files keep loading unchanged.
Structs§
- Eval
Case - One eval case: a named instruction, the world it runs in (
Setup), and the predicates (Expect) its result must satisfy.
Functions§
- load_
cases - Load every
*.roncase file fromdir, parsing each viaparse_cases_from_strand flattening the results — so a directory may freely mix single-case and multi-case files. - parse_
cases_ from_ str - Parse one
.roncase file’scontentsinto a list ofEvalCases, accepting EITHER authoring shape.