Crate aether_evals 

Structs

AgentConfig

Configuration for running an agent on a specific task

DiffStats

DockerAgent

DockerImage

DockerImageParseError

FakeAgent

GitDiff

GitRepoSpec

Judge

A built judge: the assembled prompt plus the normalized rubric it grades against. Run it with Judge::run against a model, or grade a parsed JudgeRubricResponse with Judge::summarize.

JudgeBuilder

JudgeContext

Evidence the judge grades against: the agent transcript, a workspace diff, and/or final files.

JudgeCriterionResponse

JudgeCriterionSpec

A single rubric criterion scored on a normalized 0.0..=1.0 scale.

JudgeCriterionSummary

JudgeRubricResponse

The raw rubric response the judge model is expected to return.

JudgeSummary

The graded result of running a judge: an overall pass/score plus per-criterion detail.

RetainedWorkspaceInfo

Task

TaskRun

TaskRunError

ToolCall

Transcript

Workspace

Enums

EvalRunError

GitRepoError

JudgeError

RunError

Errors that can occur when running an agent

WorkspaceError

WorkspaceSource

Constants

AETHER_EVAL_CWD_ENV

AETHER_EVAL_WORKSPACE_ROOT_ENV

AETHER_EVAL_WRAPPED_TASK_PROMPT_ENV

Environment variable through which DockerAgent hands the wrapped task prompt to the in-container command.

CONTAINER_AETHER_HOME

Traits

Agent

Trait for an agent under evaluation.

Functions

assert_tool_call_count

assert_tool_call_with_args

assert_tool_called

default_eval_env_vars

judge

Start building an LLM-as-judge from structured context and rubric criteria.