pub fn parse_cpu_layers_spec(
spec: &str,
num_layers: usize,
) -> Result<BTreeSet<u32>, CpuLayerSpecError>Expand description
Read a --moe-cpu-layers spec.
Three shapes, told apart by punctuation:
3,7,11– exactly these layers;0.5– this fraction of the model, evenly strided;8– this many layers, evenly strided.
Striding rather than taking a prefix keeps the CPU layers spread through the stack, so the CPU work interleaves with GPU work instead of arriving in one lump that nothing can overlap with.