Expand description
Blindness: label assignment, attribution stripping, and leak detection.
A judge that knows which model wrote a candidate stops grading the patch and starts voting on the model’s reputation. Three things keep that from happening:
- Labels. Candidates are presented as
A/B/C, assigned by a seeded shuffle, and each judge sees them in its own order so position carries no signal either. - Stripping. Commit messages and candidate summaries lose their
attribution trailers — both at write time (a per-worktree
commit-msghook) and at presentation time (this module). Belt and braces: the hook can be bypassed with--no-verify, the presentation filter cannot. - Leak detection. The patch body is scanned for vendor-identifying text. Blanket redaction there would corrupt the artifact under judgement, so the policy is configurable and defaults to recording the leak.
magi itself is the facilitator, which is the structural reason this works: there is no moderator agent that could leak an author, because the moderator is code that never learns anything it does not print.
Structs§
- Leak
- A vendor token found in material shown to judges.
Functions§
- assign_
labels - Label for candidate index
iafter the seeded shuffle. - commit_
msg_ hook - The
commit-msghook installed into every candidate worktree. - presentation_
order - The order judge
jsees the candidates in, as indices into the candidate list. - redact
- Replace every vendor token with
[REDACTED], case-insensitively. - sanitize_
patch - Apply the configured leak policy to a patch body.
- sanitize_
prose - Sanitize prose written by a candidate (commit messages, summaries).
- scan
- Count occurrences of each vendor token in
text. - strip_
attribution - Drop every line containing one of
patterns(case-insensitive substring).