Struct SequenceParallelConfig 

pub struct SequenceParallelConfig {
    pub sp_rank: usize,
    pub sp_size: usize,
    pub full_seq_len: usize,
    pub hidden_size: usize,
    pub num_heads: usize,
    pub head_dim: usize,
}

Sequence parallel configuration.

Fields

sp_rank: usize

This GPU’s rank in the SP group

sp_size: usize

Total GPUs in the SP group

full_seq_len: usize

Full sequence length (before sharding)

hidden_size: usize

Hidden size (not sharded in SP)

num_heads: usize

Number of attention heads

head_dim: usize

Head dimension

Implementations

impl SequenceParallelConfig

pub fn new( sp_rank: usize, sp_size: usize, full_seq_len: usize, hidden_size: usize, num_heads: usize, ) -> Self

Create a new SP config.

Panics

Panics if sequence length is not divisible by sp_size.

pub fn local_seq_len(&self) -> usize

Local sequence length on this GPU.

pub fn seq_start(&self) -> usize

Start token index for this GPU’s chunk.

pub fn seq_end(&self) -> usize

End token index (exclusive) for this GPU’s chunk.

pub fn attention_memory_savings(&self) -> f64

Memory savings for attention scores.

Attention score matrix: [num_heads, local_seq, full_seq] per GPU. Without SP: [num_heads, full_seq, full_seq]. Savings: 1 - 1/sp_size (e.g., 50% with 2 GPUs).

pub fn ring_steps(&self) -> usize

Number of ring communication steps needed.

Each GPU must see all other GPUs’ K,V → sp_size - 1 steps.

Trait Implementations

impl Clone for SequenceParallelConfig

fn clone(&self) -> SequenceParallelConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SequenceParallelConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.