# gugen v0.1 benchmark report
Generated by `cargo run --example benchmark_report` (AGENTS.md §22). Every number below is measured against the five curated literature fixtures in `tests/validation.rs` plus dedicated adversarial cases -- not estimated. Re-run this example and replace this file's content after any change to `score.rs`, `planner.rs`, or the fixture set, rather than hand-editing numbers here.
- **Valid reaction generation rate:** 5/5 fixtures produced at least one plan.
- **Element-balance exactness:** 16/16 produced plans conserve every element exactly (re-verified against the plan's own reaction, not assumed from `balance()`'s design alone).
- **Known precursor-set top-k recovery / exact precursor match:** 5/5 cited literature routes recovered exactly ([("LaAlO3 (perovskite oxide)", true), ("MgAl2O4 (spinel oxide)", true), ("LiFePO4 (phosphate)", true), ("CaO (simple binary oxide)", true), ("BaTiO3 (carbonate precursor route)", true)]; k = anywhere in the ranked list; every fixture's catalog is small enough that nothing pushes the correct route off the ranking).
- **Partial precursor match (valid alternative beyond the cited route):** 3/5 fixtures also found at least one additional, chemically valid route beyond the exact cited one -- not errors, real alternatives the catalog happens to also support: ["LaAlO3 (perovskite oxide)", "MgAl2O4 (spinel oxide)", "BaTiO3 (carbonate precursor route)"].
- **Route-family coverage:** 2/2 route families exercised across the fixture set (Phase 12 added `Mechanochemical` alongside `ConventionalSolidState`; both are offered unconditionally for every accepted precursor set, AGENTS.md §13, so this is expected to be 100% for any fixture with at least one accepted precursor set, not a discriminator between fixtures).
- **Process-step coverage:** 7/7 step kinds exercised across the fixture set: {"Cool", "Form", "Grind", "Heat", "IntermediateCharacterization", "Mix", "Weigh"}.
- **Condition evidence coverage:** 0/16 plans have any process condition resolved (temperature/duration/atmosphere/ramp/pressure) -- these fixtures are planned via `Planner::offline_minimal` (no provider configured), matching `tests/validation.rs`'s own false-confidence audit fixtures, so this is honestly 0. Since Phase 10, `InMemoryLiteratureConditionProvider` can resolve real, cited conditions for some targets when configured -- see `tests/literature_conditions.rs`, not exercised by this offline-only benchmark.
- **Unresolved condition rate:** 109 total unresolved condition entries across 16 plans (6.8 per plan on average).
- **False confident plan rate:** every one of the 16 produced plans carries `confidence.overall == 0.75` (1 distinct value(s) observed). This is not a wrong number -- it is the honest average of four sub-scores, two of which (`process_conditions`, always 0.0) directly signal the gap -- but it does not yet discriminate between plans of genuinely different real uncertainty. See `tasks/todo.md`'s Phase 8 §28-format report for the full finding.
- **Rejected-candidate reason correctness:** 2 distinct `RejectionCode`(s) fired across the fixture set: {"DuplicatePlan", "MissingTargetElement"}. Each was spot-checked against its `explanation` string for this report (e.g. every `MissingTargetElement` rejection names an element genuinely absent from that combination).
- **Deterministic reproducibility:** yes -- re-running every fixture twice produced a byte-for-byte identical report both times (also pinned as its own test in `tests/validation.rs`).
- **Planning throughput:** well under 1 millisecond per `Planner::plan` call, averaged over 1000 calls against these small (2-4 candidate) catalogs on the machine that generated this report. The raw microsecond figure is printed to stderr, not into this checked-in report, since it varies run to run and would make this file diff against itself on every regeneration. Not a claim about larger catalogs or production hardware.
- **Search-budget exhaustion rate:** 0/5 default-budget fixtures exhaust `SearchBudget::default()` (4 plans for BaTiO3 with the generous default); a deliberately tight budget (`max_precursor_sets: 1`) on the same target does trigger it, confirming the code path fires correctly rather than never being reachable: true.
- **Out-of-domain abstention rate:** 1/1 dedicated self-contradictory-target case correctly abstains (`ApplicabilityLevel::OutOfDomain`, 0 plans). This is currently the *only* reachable path to `OutOfDomain` in v0.1 -- there is no real structural domain classifier yet (`assess_applicability`'s doc comment), so this rate does not generalize beyond self-contradictory constraints.
- **Arithmetic overflow handling:** an extreme (10^25) formula-unit scale surfaces `GugenError::ArithmeticOverflow` cleanly: true.
## Skipped, not silently
- **§23 differential validation** against another synthesis-planning implementation: not attempted. §23 says 可能なら ("if possible"); no runnable reference implementation exists in this workspace, and building one only to compare against would itself need the same literature verification this phase already did, without a clear second source of truth. Open in `tasks/todo.md`.
- **§22 temperature-specific metrics** (predicted-range-contains-reference rate, evidence-covered-condition coverage, unsupported-exact-value rate): undefined in v0.1, not zero -- `TemperatureRange` is always `None` (score.rs), so there is no predicted temperature to score against anything.