Skip to main content

Module scorer

Module scorer 

Source
Expand description

Scoring one expectation against a run’s result, behind the Scorer trait — plus the batteries-included BuiltinScorer that needs NO host scoring code at all.

The host implements Scorer over the SAME World its Harness produces. For each of a case’s expect predicates the runner calls Scorer::score, collecting the returned (label, passed) pairs into the case’s CaseOutcome::predicates. A case passes iff the run succeeded AND every predicate’s passed is true.

score now also receives the run’s RunArtifacts, so a scorer can assert on what the agent DID (tool calls, params, final text) and not only on the post-run world. The common case — “score the built-in Expectations over the artifacts, ignoring the world” — is BuiltinScorer, which a host gets for free via run_suite.

Structs§

BuiltinScorer
The batteries-included scorer: evaluates the built-in Expectation assertions over a run’s RunArtifacts, ignoring the world entirely (World = ()).

Traits§

Scorer
Scores one expectation against the world AND artifacts a case produced. The host implements this for a custom predicate/world; for the built-in assertions use BuiltinScorer.