Expand description
Transcript-check grading.
tool_invocation_matches checks regex-match the "<name> <json-args>"
rendering of tool calls. assistant_message_matches checks ordered
assistant messages in a scripted conversation. Both honor optional
cross-event ordering constraints.
Functions§
- grade_
transcript_ check - Grade a
transcript_checkassertion against a run’s tool invocations, covering the empty-invocations, unsupported-kind, missing-pattern, invalid-regex, match, and no-match branches. - grade_
transcript_ check_ with_ context - Grade with the ordered conversation and harness write vocabulary available. The legacy wrapper above remains for one-shot callers and tests.