Crate rustev_eval

Expand description

Rustev’s offline evaluation (spec 004).

Reproduction is a checked statement about available retained inputs: a replay::reproduce of a bundle the host retained is reproduced, diverged or incomparable with a typed reason, never a pass by default. Candidate comparison reuses retained raw outputs only for requests with exactly equal identity. Nothing here performs I/O, reads a clock or calls a backend: the host resolves external bytes through a synchronous resolve::Resolver and supplies the time.

Re-exports§

pub use diff::EvalDiff;
pub use diff::diff;
pub use diff::render_markdown;
pub use comparison::Comparison;
pub use comparison::NamedCandidate;
pub use comparison::RepeatEvidence;
pub use comparison::compare_candidates;
pub use selection::Selection;
pub use selection::SelectionGate;
pub use selection::evaluate_selection;

Modules§

assemble: Assembling a replay bundle from what the host retained (spec 004, 5.5).
baseline: Named baselines, rustev.baseline/1, and the reproducibility check (spec 029, 3.8 and 3.9).
batch_qualification: The batching qualification report, rustev.batch-qualification/1 (spec 044, 3.8).
bundle: A case’s replay bundle as the host supplies it (spec 015, 3.1).
compare: Candidate comparison over a reproduced case (spec 004, 3.4.4 to 3.4.7).
comparison: Multi-candidate measurement evidence (spec 029, 3.11). No direction or choice.
config: Evaluator configurations, rustev.evaluator-config/1 and /2 (spec 004, 3.5.3; spec 014).
dataset: Dataset manifests, rustev.dataset/1 (spec 004, 3.5.1).
diff: Baseline differences and their deterministic Markdown rendering (spec 029, 3.10).
dynamic: Dynamic-question evaluation (spec 019, 3.7): offline reproduction of a retained outcome with zero backend calls, and a dedicated dataset, evaluator configuration and report for one dynamic task family.
eval_cases: Per-case measures, rustev.eval-cases/1 (spec 029, 3.2).
explain: Pure explanations of retained decision evidence (spec 040).
fit: Temperature fitting on the calibration split (spec 004, 3.6).
generator: The bootstrap generator xoshiro256starstar-splitmix64/1 (spec 029, 3.4.2 to 3.4.4 and 3.4.6).
gold: Gold records and gold values (spec 041, 3.3).
holdout: The held-out ledger, rustev.holdout-ledger/1 (spec 029, 3.7.3 to 3.7.5).
judge: Judge reproduction (spec 041, 3.8.1 to 3.8.3).
live: Comparison of a finished live re-execution with its historical case (spec 026, 3.7 and 3.8).
measured: Evaluation under rustev.evaluator-config/3 (spec 041, 3.2 and 3.4): ranking, short@k, backfill@p<p> and constraint metrics over the comparable proposal cases of a split, with their detail counts and the per-case measures spec 029 reconciles (3.11).
measures: Adapter measures (spec 041, 3.4): the ranking, constraint and field declarations of rustev.task-adapter/2.
metrics: Metric definitions (spec 004, 3.5.4 to 3.5.7). Every metric states its numerator, denominator and exclusions; a zero denominator, a non-finite value or an incompatible input is unknown, never a pass.
optimization: Offline optimization measurements for spec 017.
origin: Origin and evidence-quality metrics (spec 041, 3.5).
ranking_metrics: Ranking measures (spec 041, 3.4.2 to 3.4.4 and 3.11).
remote_evidence: Remote evidence availability (spec 020, 3.2).
repeat: Repeat sets and rustev.repeat-report/1 (spec 029, 3.1, 3.5, 3.6).
repeat_config: Repeat configurations, rustev.repeat-config/1 (spec 029, 3.3.1).
repeat_stats: Repeat statistics, case-clustered bootstrap intervals and noise bands (spec 029, 3.3.2 to 3.5).
replay: Historical reproduction from a replay bundle (spec 004, 3.3).
report: Evaluating a dataset split into a report (spec 004, 3.5).
resolve: Resolving retained items to typed documents (spec 004, 3.1.6 and 3.1.7).
scores: Scorers, rustev.score-views/1 and rustev.scores/1 (spec 041, 3.6 and 3.7).
scoring: Scorer, rubric, human scale, agreement and origin declarations of rustev.evaluator-config/3 (spec 041, 3.5 to 3.8).
selection: Declarative selection evidence (spec 029, 3.12), never an execution grant.

Structs§

MissingAcquisition: An uncaptured request, with its recomputed identity.