Expand description
ModelPlan-driven parallel topology and artifact placement contracts.
The automatic loader does not select a family-specific TP/EP recipe. It compiles the canonical operation plan, binds the source tensor census, estimates the checkpoint residency of every legal program, and then selects a registered numeric backend. Family packs remain responsible for semantic tensor/config validation; they do not carry per-layer placement lists.
Structs§
- Grouped
Parallel Plan - Grouped
Topology Request - Model
Parallel Contract - Parallel
Plan - Stage
Group Request - One pipeline stage’s model-specific tensor/expert group.
- Stage
Rank Group - Topology
Error - Topology
Request
Enums§
Constants§
- AUTO_
PARALLEL_ MAX_ CARDS - PRODUCT_
MAX_ CARDS - The execution planner’s supported rank envelope. Hardware qualification and tuned defaults remain model x rig evidence, but the placement/runtime contract must not stop at earlier three-card qualification cells.
- STEP37_
TRUNK_ LAYERS
Functions§
- validate_
fp8_ expert_ checkpoint - Prove that every routed layer in the structural contract exposes native stacked block-128 E4M3 expert banks. Converted and per-tensor artifacts do not inherit this backend.
- validate_
nvfp4_ expert_ checkpoint - Prove that every routed layer in the structural contract exposes native ModelOpt NVFP4 experts: either one stacked bank or the Hugging Face per-expert layout. Both carry packed e2m1 codes, per-16 UE4M3 scales, and finite-positive per-expert macros.
- validate_
step_ fp8_ checkpoint - Legacy API retained for existing focused gates.
- validate_
step_ nvfp4_ checkpoint - Legacy API retained for existing focused gates.
- validate_
step_ pp_ request - Validate the live Step PP request before the loader allocates CUDA state. Checkpoint tensor census is deliberately a separate loader gate: topology legality must remain testable without opening model files, while serving requires both gates.