Module eval 

Retrieval eval — score ground against a golden set so graph changes are MEASURED, not asserted. Mirrors the discipline of the Python test_retrieval_eval, but runs over whatever graph it’s handed (a hermetic fixture in tests; the real vault from the CLI).

Metrics (over the CLEAN cases): p_at_1 (top hit is an expected id), hit_at_k (an expected id appears within the returned hits), mrr (mean reciprocal rank of the first expected hit). Over the GARBAGE cases: garbage_rejected (the engine did NOT answer with false confidence — top hit is weak/fallback, or there were no hits at all).

Structs

CaseResult

Per-case outcome, for a readable report.

ClassStat

Ok/total for one intent class (ok = p@1 for clean, rejected for garbage).

ComboValidationError

ContextCaseResult

Per-case context outcome.

ContextEdgeExpectation

ContextOmissionExpectation

ContextReport

Aggregate context-quality metrics over cases carrying context_must / context_must_not.

DogfoodCaseResult

Per-task dogfood trace across route, focus, brief decomposition, and readiness projection.

DogfoodContextJudgment

DogfoodDifficultyStat

Per-difficulty dogfood roll-up.

DogfoodProtocolComparison

Legacy flat-query output compared with an EKF compact evidence request.

DogfoodProtocolInput

Inputs needed to compare legacy flat protocol with compact EKF protocol.

DogfoodProtocolRun

Estimated protocol economics for one tool-call style.

DogfoodProtocolSummary

Aggregate protocol economics across dogfood cases.

DogfoodReport

End-to-end dogfood metrics for task usefulness.

EvalReport

Aggregate metrics. Fractions are in [0.0, 1.0]; nullable fractions were not measured.

GoldenCase

One golden judgment. expect is the set of acceptable node ids (any one counts as a correct top hit). garbage marks a nonsense query that MUST be rejected.

GoldenSet

A golden set file: { "cases": [ {query, expect|garbage}, ... ] }.

RelationMatchExpectation

WorkflowCaseResult

Per-case brief/decomposition outcome.

WorkflowInspectionJudgment

WorkflowReport

Aggregate brief/decomposition metrics over non-garbage cases.

Enums

ComboKind

ComboSpec

ComboSpecError

Functions

advance_dogfood_context_step

classify_combo_case

compare_protocol_economics

dogfood_case_difficulty

dogfood_operator_label

dogfood_task_context_is_useful

evaluate

Run every golden case through ground and aggregate the metrics, using the FROZEN ranker calibration.

evaluate_context

Score the CONTEXT payload (the activated subgraph’s context_order), not just the route. Only cases declaring context_must and/or context_must_not participate.

evaluate_dogfood

Run a task-shaped dogfood suite across the agent loop: route -> focus context -> brief decomposition -> readiness projection.

evaluate_with_config

Like evaluate, but with an explicit [RankerConfig] — the injection seam for the dev-only parameter sweep (eidos-eval-runner --k1/--b/--weights). The kernel never reads the environment.

evaluate_workflow

Score the internal decomposition artifact produced from a focused context pack.

has_required_operator_chain

is_dogfood_code_kind

is_dogfood_docs_kind

judge_dogfood_context

judge_required_checks

judge_risk_hints

judge_workflow_inspections

missing_relation_matches

score_dogfood_checks

summarize_dogfood

summarize_workflow

workflow_initial_ready_context_only