Skip to main content

Module parallel

Module parallel 

Source
Expand description

ModelPlan-driven parallel topology and artifact placement contracts.

The automatic loader does not select a family-specific TP/EP recipe. It compiles the canonical operation plan, binds the source tensor census, estimates the checkpoint residency of every legal program, and then selects a registered numeric backend. Family packs remain responsible for semantic tensor/config validation; they do not carry per-layer placement lists.

Structs§

GroupedParallelPlan
GroupedTopologyRequest
ModelParallelContract
ParallelPlan
StageGroupRequest
One pipeline stage’s model-specific tensor/expert group.
StageRankGroup
TopologyError
TopologyRequest

Enums§

HardwareTarget

Constants§

AUTO_PARALLEL_MAX_CARDS
PRODUCT_MAX_CARDS
The execution planner’s supported rank envelope. Hardware qualification and tuned defaults remain model x rig evidence, but the placement/runtime contract must not stop at earlier three-card qualification cells.
STEP37_TRUNK_LAYERS

Functions§

validate_fp8_expert_checkpoint
Prove that every routed layer in the structural contract exposes native stacked block-128 E4M3 expert banks. Converted and per-tensor artifacts do not inherit this backend.
validate_nvfp4_expert_checkpoint
Prove that every routed layer in the structural contract exposes native ModelOpt NVFP4 experts: either one stacked bank or the Hugging Face per-expert layout. Both carry packed e2m1 codes, per-16 UE4M3 scales, and finite-positive per-expert macros.
validate_step_fp8_checkpoint
Legacy API retained for existing focused gates.
validate_step_nvfp4_checkpoint
Legacy API retained for existing focused gates.
validate_step_pp_request
Validate the live Step PP request before the loader allocates CUDA state. Checkpoint tensor census is deliberately a separate loader gate: topology legality must remain testable without opening model files, while serving requires both gates.