Skip to main content

Module evaluate

Module evaluate 

Source
Expand description

The evaluation: what an evaluate job’s report is made of.

Six checks over the files a claim carries. Five read the files and nothing else, and never call claude:

CheckWhat it reports
secretA private key; a cloud, GitHub, GitLab, Slack, Stripe, Google, OpenAI, Anthropic, npm or Hugging Face token; a Recall authkey or key; a JSON Web Token; a password in a URL; an AWS secret access key, a password, or a long hex string assigned to a name that says so
duplicateThe same normalized paragraph, or list item, in two files or two scopes
dead_linkA MEMORY.md link to a file that is not there
wrong_scopeA project file whose front matter says type: user, the case recall promote exists for
staleA file unchanged for RECALL_EVAL_STALE_DAYS that names a path or a command, for the owner to confirm

The sixth, contradiction, asks claude -p once per project, with the flags that keep a merge cheap and inert (Merger::ask), over that project’s files and the global scope. It runs only when the request asked for it: it spends the owner’s Claude usage.

A Report keeps note text out of its findings by construction: a finding is built from enums, the file it is in, line numbers and related files, and every excerpt, reason and suggested edit goes in Details. The server refuses a finding with anything else in it anyway. A secret is masked even in details, wherever a report quotes it and whichever check does (Redactor, in crate::redact): a report is not another place for a key to be kept. And details is held to a size (MAX_DETAILS_BYTES), so a report always fits in a result the server takes.

Structs§

Report
What the worker posts back.
Settings
What an evaluation needs besides its input.

Constants§

CONTRADICTION_PROMPT
Replaces Claude Code’s own system prompt for the contradiction check.
CONTRADICTION_TIMEOUT
How long one contradiction call may take.
MAX_DETAILS_BYTES
The most details a report carries, in bytes, well inside the 5 MiB a result may be. Past it, the rest of the findings keep no details.
MAX_EDIT_BYTES
The largest suggested edit kept. One larger is left out: an edit cannot be cut and still be the edit.
MAX_EXCERPT_BYTES
The most of one excerpt kept in details, in bytes.
MAX_PROMPT_BYTES
The most of a project, in bytes, one contradiction call is handed. A project larger than this is skipped, and the report says so.
MAX_REASON_BYTES
The most of one reasoning, or one reason a check was skipped, kept.

Functions§

counts
How many findings of each kind a report has, for the worker’s log.
evaluate
Runs the evaluation input asks for. The five checks that read files always run; the contradiction check runs, through claude, only when input.contradictions asks for it.