Module parallel 

Architecture-declared parallel parameter semantics and local layouts. Backend-neutral semantic parameter sharding and rank-local layouts.

Architectures describe physical checkpoint members in logical groups. An execution backend may then realize the resulting placement without knowing projection names, attention geometry, or other model-family semantics.

Structs

LocalModelLayout

Complete rank-local model geometry produced alongside checkpoint placement.

LocalTensorLayout

Rank-local shape and placement for one planned physical tensor.

ParallelModelInfo

Architecture-neutral information for one rank-local parallel model.

ParameterGroupSpec

Atomic logical parameter and all of its physical checkpoint companions.

ParameterMemberSpec

One physical tensor belonging to a logical parameter group.

Enums

MemberSharding

Rank-local selection rule for one physical checkpoint tensor.

ParallelPlanError

Invalid architecture-declared parallel semantics.

ParameterRole

Semantic role of a logical parameter group.

ProjectionSharding

Logical sharding behavior for a parameterized affine projection.

ShardingPolicy

Behavior when a requested shard is not legal for the current TP size.

TensorPlacement

Backend-neutral placement decision for one physical tensor.

Functions

aligned_partition_units

Returns the finest legal logical-unit count for an aligned partition.

expand_linear_format_parameter_groups

Rewrites semantic dense matrix declarations into their authoritative physical checkpoint representation and publishes every required companion in the same atomic parameter group.

module_parameter_group

Describes every parameter in a neutral module as one logical group.

partitioned_module_parameter_group

Describes every parameter in a neutral module as one shared logical partition.

partitioned_projection_group

Describes projections that consume one shared logical partition.

projection_parameter_group

Describes one affine projection and all encoding companions.

segmented_projection_group

Describes a component-major fused column projection and its row-parallel output as one shared logical partition.