Expand description
ML benchmarking framework: evaluators, metrics, reports, visualization.
Migrated from sentinel/rs-services/ruskit to use rskit-errors (AppResult/AppError) instead of anyhow, and rskit-provider for the Provider/RequestResponse traits.
Benchmark orchestration accepts injected clocks and storage. Use FixedClock
and a test-owned RunStorage implementation for deterministic tests; use SystemClock
and FileRunStorage for normal CLI runs.
Re-exports§
pub use evaluator::Evaluator;pub use evaluator::EvaluatorFunc;pub use evaluator::FromProvider;pub use execution::BenchExecutionPlan;pub use result::BenchRunResult;pub use result::MetricResult;pub use run_storage::FileRunStorage;pub use run_storage::ListOptions;pub use run_storage::RunStorage;pub use runner::BenchRunner;pub use runner::RunOptions;pub use types::BenchSample;pub use types::Prediction;pub use types::ScoredSample;
Modules§
- cli
- CLI utilities for bench operations.
- compare
- Run comparison and regression detection.
- curves
- Curve data types for visualization.
- dataset
- Dataset loading for accuracy benchmarking.
- dataset_
loader - Generic dataset loader with pipeline integration.
- evaluator
- Evaluator trait and adapters for bench.
- execution
- Benchmark execution planning with worker and observability integration.
- metric
- Pluggable evaluation metrics for the bench framework.
- middleware
- Evaluator middleware: timing and caching wrappers.
- report_
gen - Report generation for bench results.
- result
- Result types for bench runs.
- run_
storage - Storage trait and file-based implementation for bench results.
- runner
- Bench runner — orchestrates the complete benchmark lifecycle.
- schema
- Schema version constants for bench result serialization.
- types
- Core data types for the bench framework.
- viz
- SVG chart rendering for benchmark results.
Structs§
- Fixed
Clock - Deterministic clock for tests and reproducible harnesses.
- System
Clock - System-backed clock.
Traits§
- Bench
Clock - Clock used for wall-clock timestamps and monotonic elapsed durations.
Functions§
- system_
clock - Return a shared system-backed clock.
Type Aliases§
- Shared
Clock - Shared clock handle.