Expand description
100-point model quality scoring system (spec §7) 100-Point Model Quality Scoring System (spec §7)
Evaluates models across seven dimensions based on data science and ML best practices, aligned with Toyota Way principles:
| Dimension | Max Points | Toyota Way Principle |
|---|---|---|
| Accuracy & Performance | 25 | Kaizen (continuous improvement) |
| Generalization & Robustness | 20 | Jidoka (quality built-in) |
| Model Complexity | 15 | Muda elimination (waste reduction) |
| Documentation & Provenance | 15 | Genchi Genbutsu (go and see) |
| Reproducibility | 15 | Standardization |
| Security & Safety | 10 | Poka-yoke (error-proofing) |
§References
- [Raschka 2018] Model Evaluation, Model Selection, and Algorithm Selection in ML
- [Hastie et al. 2009] The Elements of Statistical Learning
- [Mitchell et al. 2019] Model Cards for Model Reporting
- [Gebru et al. 2021] Datasheets for Datasets
- [Pineau et al. 2021] ML Reproducibility Checklist
Structs§
- Critical
Issue - Critical issue that must be addressed
- Dimension
Score - Score for a single dimension
- Dimension
Scores - Individual dimension scores
- Model
Flags - Model feature flags
- Model
Metadata - Model metadata for scoring
- Quality
Score - 100-point model quality score
- Score
Breakdown - Breakdown item for dimension scoring
- Scoring
Config - Configuration for quality scoring
- Training
Info - Training information
Enums§
- Finding
- Finding from quality analysis
- Grade
- Letter grade for quality score
- Scored
Model Type - Model type for scoring context
- Severity
- Critical issue severity
Functions§
- compute_
quality_ score - Compute quality score for a model