Skip to main content

Module transcript_check

Module transcript_check 

Source
Expand description

Transcript-check grading.

tool_invocation_matches checks regex-match the "<name> <json-args>" rendering of tool calls. assistant_message_matches checks ordered assistant messages in a scripted conversation. Both honor optional cross-event ordering constraints.

Functions§

grade_transcript_check
Grade a transcript_check assertion against a run’s tool invocations, covering the empty-invocations, unsupported-kind, missing-pattern, invalid-regex, match, and no-match branches.
grade_transcript_check_with_context
Grade with the ordered conversation and harness write vocabulary available. The legacy wrapper above remains for one-shot callers and tests.