Structs§
- Agent
Config - Configuration for running an agent on a specific task
- Diff
Stats - Docker
Aether Agent - Docker
Image - Docker
Image Parse Error - Eval
Files Report - Outcome of running a collection of eval files.
- Eval
Outcome - Eval
Report - Eval
RunOptions - Request to discover and run a collection of declarative eval files.
- Fake
Agent - GitDiff
- GitRepo
Spec - Judge
Criterion Summary - Judge
Summary - Tool
Call - Workspace
Enums§
- Eval
File Error - Eval-file setup errors. Setup — loading, image resolution and builds, settings — happens
before any eval runs and aborts the collection. Faults inside an individual eval (workspace
setup, the agent run, the judge call) do not abort a collection run; they are recorded on
that eval’s
crate::spec::EvalOutcomeinstead. - Eval
RunError - GitRepo
Error - Image
Build Error - RunError
- Errors that can occur when running an agent
- Workspace
Error - Workspace
Source
Traits§
- Agent
- Trait for an agent under evaluation.
Functions§
- assert_
tool_ call_ count - assert_
tool_ call_ with_ args - assert_
tool_ called - run_
eval - run_
eval_ files - Discover, validate, and run declarative eval files. Agents and judge models are assembled before anything runs, so a broken case fails fast. Docker image builds are deduplicated and run concurrently before eval execution. Outcomes are returned in discovery order.