Module evaluations

Expand description

/v1/evaluations: reports on what memory holds, made by the worker.

A run is asked for with EvaluationRequest and becomes an KIND_EVALUATE job, which recall-worker claims like a merge. The claim carries the files to look at; the worker runs its checks (KINDS) and posts back an EvaluateResult: findings and details.

The split is the point. A Finding holds only enums, a file’s identity, line numbers and the files it relates to: never a word of a note. The server refuses a result whose finding carries any other key (check_finding), holds an id of any other shape, or names a file it does not store, so nothing a worker writes into a finding can be note text. Everything that quotes a note, the excerpt, the reasoning and a suggested edit, goes in details (Details), which only GET /v1/evaluations/{id} returns, and only to the operator or an admin device.

details is stored as plain JSON for now. The design seals it with the content key (docs/design/part5-plan.md, PR 6), and that is still the intended end state: encryption was parked on 2026-09-25, and when it comes back only how details is stored changes, never what a finding may hold.

Nothing here changes memory. A suggested edit is applied only when the owner runs recall eval apply, which writes it to the local file and pushes it like any other edit.

Structs§

Details: What details holds: everything about a finding that quotes a note. The server stores it as it came and never reads it; the worker writes it and recall eval reads it.
Evaluation: GET /v1/evaluations/{id}: one run, its findings and its details.
EvaluationCreated: POST /v1/evaluations’s answer.
EvaluationList: GET /v1/evaluations.
EvaluationRequest: Body of POST /v1/evaluations. Both members may be left out: {} asks for every project, without the contradiction check.
EvaluationSummary: One run, as GET /v1/evaluations lists it: counts, never findings or details.
FileRef: A file, by its project and path.
Finding: One finding: what kind, how bad, where. Nothing else, ever: see the module docs and check_finding.
FindingDetail: The note text behind one finding.
Skipped: Something a run did not check.
SuggestedEdit: An edit recall eval apply can make: lines lines of one file, replaced with replacement, only if the file is still the version the evaluation read.

Constants§

EVALUATIONS_PATH: POST asks for a run; GET lists runs, newest first. Admin only.
FILE_REF_KEYS: The keys a related file has, and the only ones it may.
FINDING_KEYS: The keys a finding has, in order, and the only ones it may.
GLOBAL_PREFIX: How a global scope’s key starts (global:eko). An evaluation always reads the global scopes beside the projects it was asked for.
KINDS: Every kind of finding, most urgent first: the order a report lists them in.
KIND_CONTRADICTION: Two notes that say opposite things. The only check that asks claude, and it runs only when the request asks for it.
KIND_DEAD_LINK: A MEMORY.md line linking to a file that is not there.
KIND_DUPLICATE: The same paragraph in two files, or two scopes.
KIND_SECRET: A file that is in memory, where it should not be: a key, a token.
KIND_STALE: A file unchanged for a long while that names a path or a command, for the owner to confirm is still true.
KIND_WRONG_SCOPE: A project file that says it is about the user (type: user), which belongs in the global scope: what recall promote is for.
MACHINE_PREFIX: How a machine scope’s key starts (machine:mbp).
MAX_FINDINGS: The most findings one result may carry. A worker that finds more keeps the first this many and says so in details.
MAX_PROJECTS: The most projects one request may name.
MAX_RELATED: The most files one finding may name as related.
SEVERITIES: Every severity, lowest first.
SEVERITY_HIGH: A finding to fix now: a secret.
SEVERITY_LOW: A finding that can wait.
SEVERITY_MEDIUM: A finding worth fixing soon.
STATE_DONE: Its report is in.
STATE_FAILED: Out of attempts: error says why.
STATE_QUEUED: Waiting for a worker to claim it.
STATE_RUNNING: A worker holds it.

Functions§

check_finding: Reads one finding as the server receives it, refusing anything but the shape Finding has: exactly FINDING_KEYS, an id is_finding_id accepts, a kind and a severity from their lists, two line numbers from 1 in order, a valid project key and path, and at most MAX_RELATED related files, each exactly FILE_REF_KEYS.
evaluation_path: GET: one run, with its findings and details. Admin only.
is_finding_id: Whether id is a finding id: f and a number from 1, at most six digits.