Expand description
Bench command — single-process CLI benchmark (no HTTP).
Same forward path as ferrum run (batch=1 by default, no HTTP). Emits
the canonical BenchReport schema from ferrum-bench-core. This is
the “Level 1” command in PLAYBOOK § 1 — single-user feel of the engine.
Modes: ferrum bench qwen3:4b # default: sequential, 5 rounds ferrum bench qwen3:4b –concurrency 4 # concurrent (tests batch decode) ferrum bench qwen3:4b –max-tokens 1024 # long decode (tests flash decode) ferrum bench qwen3:4b –long-context # 2k prompt + 256 decode ferrum bench qwen3:4b –concurrency 8 –max-tokens 64 # throughput stress
Phase 0 additions: –n-repeats N independent runs (≥3 unlocks stddev + CI95) –goodput … SLO triple for goodput computation –output json emit canonical BenchReport JSON –out PATH write JSON to file (else stdout)