Expand description
Native LLM-judge gate (SPEC §8.3): a separate model grades the candidate output against a rubric. Three anti-gaming properties are enforced structurally here, not left to the prompt:
- maker ≠ checker — a model never grades its own output; if the judge model equals the candidate’s model, the gate abstains rather than returning a compromised verdict.
- candidate is data, not instructions — the candidate is embedded in a fenced data block under a pinned system prompt that tells the judge to ignore any instructions inside it; the judge is called with no tools.
- the operator’s threshold governs — the verdict is
score ≥ threshold, not whatever the judge feels like passing.
Live judge quality (does a real model catch a bad output?) is verified only against a live provider; the wiring, injection-resistance, and verdict logic here are unit-tested offline with a mock judge.
Structs§
- Judge
Gate - A gate that asks a judge model to grade the candidate against a rubric.
Functions§
- build_
judge_ request - Build the judge request: pinned system prompt + the candidate fenced as data (never as instructions), no tools.
- parse_
judgment - Parse the judge’s reply into a verdict. The operator’s
thresholdon the judge’sscoreis authoritative; an explicitpassis a fallback when no score is given; anything unparseable abstains (never a fabricated pass/fail).