Struct SlidingWindowMaskCache 

pub struct SlidingWindowMaskCache { /* private fields */ }

Precomputed sliding window attention mask

Each position can only attend to the previous window_size positions. Used by Mistral, Mixtral, and other sliding window attention models.

Implementations

impl SlidingWindowMaskCache

pub fn new( max_seq_len: usize, window_size: usize, device: &Device, ) -> Result<Self>

Create a new sliding window mask cache

Arguments

• max_seq_len - Maximum sequence length to precompute
• window_size - Number of positions to attend to
• device - Device to store the mask on

pub fn get(&self, seq_len: usize) -> Result<Tensor>

Get sliding window mask for a specific sequence length

pub fn get_broadcast(&self, seq_len: usize) -> Result<Tensor>

Get sliding window mask reshaped for attention broadcasting

pub fn window_size(&self) -> usize

Get the window size

pub fn max_seq_len(&self) -> usize

Get the maximum sequence length