Module validate
Expand description
Historical band scan, validation report, and constrained weight tuning — Units C and D of the own-repo defect-calibration design.
§Unit C — the historical band scan + validation report
band_history reuses health_trend’s at-rev machinery
(sampled_commits → live_paths_at → ingest_complexity_at_rev +
materialize_imports_at_rev → a history_cutoff-scoped
run_code_health_scoped) to recompute code-health bands at ≤12 evenly
spaced historical revisions — but, unlike health_trend::file_series,
with NO top-50 path cap: the validation report needs every scored file,
since any of them might be the one a mined defect touched.
validate matches each SzzLink’s defect-introducing commit to the
nearest band sample at-or-before its date, tallies the headline band
table, and scores HEAD’s structural_risk against the defect-implicated
file labels via stats::{auc, precision_at_k}.
§Unit D — constrained weight tuning
tune_weights runs a deterministic coordinate-descent search over the
eight SMELL_WEIGHTS and only adopts a tuned set when the evidence
clears an honesty floor: fewer than 30 linked defect-changes, fewer than 10
implicated files, a tuned validation AUC below random (0.5), or a
validation-AUC improvement short of the acceptance margin, each keep the
defaults instead of the tuned weights.
§Design decision: re-scoring without re-running the SQL pass
Evaluating a candidate weight set’s training/validation AUC needs a
structural_risk for every file under THAT candidate — but re-running
run_code_health_scoped (the full SQL pass, including the
coupling/god-class/clone sub-scans) once per candidate is far too
expensive across a coordinate-descent search. Instead:
capture_intensitiesreads thecode_health_biomarkers_v1session temp table’s rows ONCE, immediately after a single HEADrun_code_health_scopedcall, in the SAME connection/session (that table is deliberately left queryable after the scan returns — see its own doc comment incode_health.rs), into aHashMap<path, [f64; 8]>keyed inSMELL_WEIGHTSorder.- Each candidate’s risk is then computed Rust-side by
structural_risk_from_intensities:Σ wᵢ·intensityᵢclamped to 1.0 — mirroring the SQLLEAST(1.0, SUM(intensity * CASE …))formula exactly (the HEAD,include_clones = truepath, which carries no renormalization divisor).
The two are unit-tested for parity (tolerance 1e-9) against a real
run’s structural_risk on the biomarker fixture in
tests/defect_calibration_test.rs.
Functions§
- band_
history - Uncapped per-sample band maps: for each of ≤12 sampled revs (the same
evenly-spaced history sampler
health_trenduses, oldest-first, newest always included), the file→band map at that point in history. - capture_
intensities - Capture the per-file, per-smell biomarker intensities
tune_weightsneeds to re-score candidate weight sets, without re-running the whole code-health SQL pass for every candidate — see this module’s design- decision doc comment above. - default_
weights - The [
SMELL_WEIGHTS] defaults converted to owned(name, weight)tuples, inSMELL_WEIGHTSorder — the canonicaldefaultsargument fortune_weightsand the reference weight vectorstructural_risk_from_intensitiesis parity-tested against. - structural_
risk_ from_ intensities - Rust-side mirror of
code_health’s SQL formulaLEAST(1.0, SUM(intensity * weight))— specifically the HEAD (include_clones = true) path, which carries no renormalization divisor (STRUCTURAL_SCALE_NO_DRYonly applies when DRY is excluded, which never happens at HEAD).weightsmust be in the same fixed order asintensities’s 8 slots —default_weights’s order (SMELL_WEIGHTSorder) — for the parity to hold; see the parity test intests/defect_calibration_test.rs. - tune_
weights - Constrained deterministic coordinate search over the eight
SMELL_WEIGHTS(Unit D).intensitiesare per-file 8-smell intensity vectors inSMELL_WEIGHTSorder (seecapture_intensities);train/validationare(path, label)splits already partitioned by fix date (60/40, older/newer — a leakage guard against a random split, prepared by the caller);defaultsare the weights to fall back to and the search’s starting point (seedefault_weights). - validate
- Band table + AUC + precision@k against defect labels.