Expand description
Test cases: the YAML a user writes to describe one test of a skill — the initial data to hand the skill, an optional simulated user for multi-turn runs, and the evals that decide pass/fail.
Structs§
- Simulated
User - The simulated-user block that turns a single-turn case into a multi-turn one.
When present, after each assistant turn the runner asks the provider to play
the user (guided by
persona) untildone_whenholds ormax_turnsis hit. - Test
Case - One test case.
Functions§
- discover_
cases - Discover test-case files: either a single
.yaml/.ymlfile or every such file directly inside a directory (sorted for deterministic ordering).