Skip to main content

Module parallel

Module parallel 

Source
Expand description

Model-specific parallel topology contracts.

The rank planner is reusable, but model support is never inferred from a loader or a few scalar dimensions. Each family must register the complete geometry that its TP/EP program shards. Step-3.7-Flash is the first registered contract because its query-head count varies by layer (64 full-attention / 96 sliding-attention), while KV heads stay at 8. Step-3.5 and other siblings do not inherit this contract merely because they share the step35 architecture tag.

Structs§

GroupedParallelPlan
GroupedTopologyRequest
ModelParallelContract
ParallelPlan
StageGroupRequest
One pipeline stage’s model-specific tensor/expert group.
StageRankGroup
TopologyError
TopologyRequest

Enums§

HardwareTarget

Constants§

PRODUCT_MAX_CARDS
The execution planner’s supported rank envelope. Hardware qualification and tuned defaults remain model x rig evidence, but the placement/runtime contract must not stop at earlier three-card qualification cells.
STEP37_TRUNK_LAYERS

Functions§

validate_step_fp8_checkpoint
Prove that the official Step checkpoint exposes every routed expert projection as a native stacked block-128 E4M3 bank. Converted and per-tensor artifacts do not inherit this contract.
validate_step_nvfp4_checkpoint
Prove that the official Step NVFP4 checkpoint exposes every routed expert projection as a native stacked modelopt NVFP4 bank: packed e2m1 codes [E, out, in/2], per-16 UE4M3 scales [E, out, in/16], and a finite positive per-expert weight_scale_2 macro. Converted and per-tensor artifacts do not inherit this contract. The macro census matters: those values run ~1e-5..1e-4 in the official artifact and dropping them silently produces garbage, so a bank whose macros fail the finite-positive check refuses here rather than at first decode.
validate_step_pp_request
Validate the live Step PP request before the loader allocates CUDA state. Checkpoint tensor census is deliberately a separate loader gate: topology legality must remain testable without opening model files, while serving requires both gates.