Skip to main content

Module case

Module case 

Source
Expand description

The generic, RON-authored eval case schema + loader.

An EvalCase is one natural-language instruction, the world it runs against (the host’s Setup), and the host’s Expect predicates its result must satisfy. The case container is domain-agnostic: Setup and Expect are the host’s own types (e.g. AetherCore’s voxel EvalSetup / Expectation), so eval-core stays free of any game/LLM specifics.

Cases are authored as .ron files and loaded by load_cases, which is fail-loud (a malformed file is a hard error naming it) and deterministic (files are read in sorted order).

§One file, one OR many cases

A .ron case file may hold EITHER a single EvalCase or a list [EvalCase, EvalCase, …] of related cases — see parse_cases_from_str, which load_cases uses per file. This lets an author group a whole capability (say all the arithmetic checks) in one file without one-file-per-case sprawl, while existing single-case files keep loading unchanged.

Structs§

EvalCase
One eval case: a named instruction, the world it runs in (Setup), and the predicates (Expect) its result must satisfy.

Functions§

load_cases
Load every *.ron case file from dir, parsing each via parse_cases_from_str and flattening the results — so a directory may freely mix single-case and multi-case files.
parse_cases_from_str
Parse one .ron case file’s contents into a list of EvalCases, accepting EITHER authoring shape.