Skip to main content

parse_cpu_layers_spec

Function parse_cpu_layers_spec 

Source
pub fn parse_cpu_layers_spec(
    spec: &str,
    num_layers: usize,
) -> Result<BTreeSet<u32>, CpuLayerSpecError>
Expand description

Read a --moe-cpu-layers spec.

Three shapes, told apart by punctuation:

  • 3,7,11 – exactly these layers;
  • 0.5 – this fraction of the model, evenly strided;
  • 8 – this many layers, evenly strided.

Striding rather than taking a prefix keeps the CPU layers spread through the stack, so the CPU work interleaves with GPU work instead of arriving in one lump that nothing can overlap with.