Expand description
§Quality Metrics Collection System
Comprehensive quality tracking across all ThinkTools modules. Provides dashboards, trends, and improvement recommendations.
§Metrics Collected
| Category | Metrics |
|---|---|
| Accuracy | GSM8K, MATH, ARC-C benchmark scores |
| Calibration | Brier score, ECE, overconfidence ratio |
| Reasoning | PRM scores, ToT success rate, step validity |
| Verification | Triangulation score, fact-check accuracy |
| Debate | Win rate, argument strength, verdict confidence |
§Usage
ⓘ
use reasonkit::thinktool::quality::{QualityDashboard, QualityMetric};
let mut dashboard = QualityDashboard::new();
dashboard.record_metric(QualityMetric::Accuracy { benchmark: "GSM8K", score: 0.859 });
dashboard.record_metric(QualityMetric::Calibration { brier: 0.15, ece: 0.08 });
let report = dashboard.generate_report();
println!("{}", report.format());Structs§
- Metric
Record - Timestamped metric record
- Quality
Dashboard - Quality dashboard for tracking and reporting
- Quality
Report - Quality
Score - Quality score aggregation
- Quality
Targets - Quality targets for comparison
- Target
Violation
Enums§
- Quality
Grade - Quality
Metric - Individual quality metric
- Trend