Skip to main content

Module eval

Module eval 

Source
Expand description

Evaluation metrics (DESIGN §14.2). Pure functions comparing extracted assertions against golden-corpus annotations.

Structs§

EvalMetrics
Quality metrics computed from an extraction run against golden annotations.
ExtractedTriple
A triple extracted from the brain, for comparison against golden annotations.

Functions§

compute_metrics
Compute metrics by comparing extracted triples against expected triples.
compute_metrics_with_fabrication
Like compute_metrics but with a measured fabrication rate (§17.3, 10.7). The fabricated rate is computed separately by measure_fabrication_rate because it requires the source text, which compute_metrics does not take.
measure_fabrication_rate
Measure fabricated entity rate (§17.3, F19, 10.7): of the entity surfaces extracted, what fraction do NOT appear verbatim in the source text? This detects hallucinated entities that slipped past validation.