Module eval 

Attribution accuracy evaluation.

Scores vasari why against a labeled corpus and reports the v0.1 go/no-go gate: does why return the correct top-level Intent often enough to be the hero verb? See tests/corpus/attribution/ for the corpus + protocol.

What this measures, honestly: file→intent attribution accuracy. The current attributor is whole-file (every line of a file resolves to the same intent), so the :line in why <file>:<line> is decorative under v0.1. The evaluator reports single-intent-file vs multi-intent-file accuracy separately so a blended number can’t hide a multi-intent collapse, and reports recall (why_all contains the right intent) alongside precision (why’s single top pick is right).

Structs

EvalReport

Full evaluation result over a corpus.

Label

One ground-truth label. expected_intent is a list so multi-intent files can name every contributor; an empty list means “should return nothing.”

Row

One scored row in the report.

Enums

GateStatus

Whether the gate passed, failed, or could not be evaluated.

Outcome

Per-label scoring outcome.

Constants

ACCURACY_BAR

Accuracy bar (design doc): ≥80% correct top-level Intent.

MATCH_THRESHOLD

Default token-overlap threshold for matching a returned intent to a label.

N_MIN

Minimum corpus size before the gate is eligible to PASS/FAIL. Below this the result is reported but the gate is not evaluated — a 10-line pilot cannot clear the Wilson floor (8/10 ≈ 0.49), so gating on it would wedge why off permanently.

WILSON_BAR

Wilson 95% lower-bound bar (eng-review T4): ≥70%.

Z_95

z for a 95% confidence interval.

Functions

evaluate

Evaluate vasari why over a labeled corpus already ingested into store.

intent_matches

True if candidate covers at least threshold of expected’s tokens.

parse_labels

Parse a labels JSONL document (one Label per non-empty line).

wilson_lower_bound

Wilson score interval lower bound for a binomial proportion.