Skip to main content

bench_format

Function bench_format 

Source
pub fn bench_format(quant_format: &str) -> &str
Expand description

The bench format key an engine quant name is measured under.

The benchmark keys its numbers by expert format, not by model, because the CPU-MoE-vs-PCIe-gather ratio the choice rides on is dominated by (format, hardware) – so a profile taken on one workload transfers to any model with the same expert format on the same card. Most engine names are already the bench name; mxfp4 is benched under its kernel’s name, mxfp4_triton.

Anything not in the table is passed through unchanged, which is deliberate rather than lossy: an unmapped name finds no entry in the profile, the lookup yields None, and the caller keeps its safe offload default. Only the offload-family formats that have a CPU MoE weight path can ever resolve to hybrid, and those are exactly the ones listed here.