# RunORExeriment
Takes as an input a (kinda) standardized OR experiment format.
Performs the experiments, stores the solutions and the performance statistics
for each solver, then generates various performance analysis.
## Features
- [X] Read experiment description
- [X] Read instance list
- [X] Run the experiments
- [X] Generate primal bounds table
- [X] Generate average relative percentage deviation tables
- [X] Generate Pareto fronts for each instance class
- [X] add time to best-known
- [X] add time to optimality proof
- [X] add which algorithm is statistically better than the others ARPD
- [X] latex table for ARPD with Wilcoxon signed-rank test
- [X] Handle single-point multiple primal algorithms
- [X] handle analysis generation only
- [X] Handle reference for ARPD != BK_ARPD
- [X] Handle external "primal/time" ARPD
- [X] custom Pareto diagrams (subset of solvers)
- [X] best-known stats (number improved, number reached, number proved to optimality)
- [X] add custom data in the best_known table
- [X] generate latex file for best known solutions
- [X] add spaces in the latex table outputs for easy copy & paste
- [X] custom ARPD tables with custom time_limit for curves + inclusion of external ARPD
- [X] re-run missed experiments with lower number of threads
- [X] add multiple_y types to the best known table
- [ ] call fallback between the runs and the analysis
- [ ] report for each instance class, the number of optimal proofs
- [ ] if Cartesian product over parameters, perform analysis similar to the ROADEF18 paper
# known issues
- [ ] some latex outputs have "_" not escaped