Crate agent_evaluate_contract 

HTTP API contract for the Evaluate service.

This service provides a programmatic interface for running agent evaluation cases, collecting metrics, and producing reports. The eval harness replays predefined conversation turns against an agent runtime instance and judges outcomes against oracles.

Layering

agent-evaluate-contract   paths + DTOs        (this crate)
agent-evaluate-runtime    eval runner + judge
agent-evaluate-server     router + auth + bin

Structs

AdminReviewPage

AdminRunPage

AdminRunSummary

AdminSuitePage

AdminSuiteSummary

ApiError

Error body shared by every non-2xx response.

AssertionVerdict

CancelRunRequest

CaseOutcomeDiff

CaseRunPage

CaseRunSummary

ClaimEvaluationCaseRequest

Runtime-facing pull protocol. evaluate_task_id is the durable EvalOperation::id returned when the SDK/CLI configures a run. Infra never needs a callback URL for the Runtime.

ClaimEvaluationCaseResponse

ClaimReviewRequest

CompareReportsRequest

CompleteReviewRequest

CreateEvalRunRequest

EnqueueReviewRequest

EnvironmentProfileVersion

EvalAccessScope

EvalBudget

EvalCase

EvalCaseFile

EvalCaseResult

EvalCaseSummary

EvalOperation

EvalOracle

EvalReport

EvalSuiteVersion

EvalTurn

EvaluateAdminOverview

EvaluationCaseLease

EvaluationSubject

Agent is an optimistic precondition; Runtime ID remains the execution selector and Runtime is the authoritative owner of their binding.

EvaluatorVerdict

EvaluatorVersion

EvidenceRef

Immutable evidence object owned by the scoped Evaluate namespace.

ExperimentVariant

GetCaseRequest

GetOperationRequest

GetReportRequest

HumanReview

ListAdminReviewsRequest

ListAdminRunsRequest

ListAdminSuitesRequest

ListCaseRunsRequest

ListCasesRequest

ListCasesResponse

LoadCasesRequest

OracleVerdict

OutcomeAssertionSpec

Strictly parsed compatibility schema for outcomeAssertions.

PluginEvaluationEvidence

RegisterEnvironmentProfileRequest

RegisterEvaluatorVersionRequest

RegisterSuiteRequest

RenewEvaluationCaseRequest

RenewEvaluationCaseResponse

ReportComparison

RunEvalRequest

SandboxExecutionEvidence

Bounded evidence returned by the isolated executor. Verdict calculation remains owned by Evaluate and is deterministic from this typed structure.

SandboxExecutionRequest

SubmitEvaluationEvidenceRequest

SubmitEvaluationEvidenceResponse

TurnAssertionSpec

VerifyCommand

VerifyCommandResult

VersionedEvalCase

VersionedResourceRef

Immutable reference to a catalog resource. Evaluate snapshots the resolved version into a CaseAttempt; mutable aliases are deliberately unsupported.

WorkspaceFileAssertionSpec

Enums

CaseFailureKind

CaseRunState

EvaluatorImplementation

OperationState

ReviewState

TypedAssertion

Versioned typed assertion schema used by durable suites. The legacy Value fields on EvalCase remain only for v1 wire compatibility.

Constants

ADMIN_CANCEL_RUN_PATH

ADMIN_CLAIM_REVIEW_PATH

ADMIN_COMPLETE_REVIEW_PATH

ADMIN_OVERVIEW_PATH

ADMIN_REPORT_PATH

ADMIN_REVIEWS_PATH

ADMIN_RUNS_PATH

ADMIN_RUN_CASES_PATH

ADMIN_SUITES_PATH

AUDIENCE

CANCEL_RUN_PATH

CLAIM_REVIEW_PATH

COMPARE_REPORTS_PATH

COMPLETE_REVIEW_PATH

DEFAULT_ADMIN_PAGE_SIZE

DEFAULT_PORT

DRIVER_CLAIM_PATH

DRIVER_RENEW_PATH

DRIVER_SUBMIT_PATH

DURABLE_REPORT_PATH

ENVIRONMENT_PROFILES_PATH

EVALUATOR_VERSIONS_PATH

GET_CASE_PATH

GET_REPORT_PATH

HEALTH_PATH

LIST_CASES_PATH

LOAD_CASES_PATH

MAX_ADMIN_PAGE_SIZE

MAX_RESOURCE_ID_CHARS

OPERATIONS_PATH

REVIEW_QUEUE_PATH

RUNS_PATH

RUN_CASES_PATH

RUN_EVAL_PATH

SERVICE_NAME

SERVICE_TOKEN_ENV

SUITES_PATH