Struct RoPECache 

pub struct RoPECache { /* private fields */ }

Precomputed RoPE (Rotary Position Embedding) frequencies

RoPE is used by most modern LLMs for position encoding. Computing cos/sin values for each position is expensive, so we precompute them once for max_seq_len positions.

Example

let rope = RoPECache::new(128, 4096, 10000.0, &Device::CPU)?;
let (cos, sin) = rope.get(seq_len)?;

Implementations

impl RoPECache

pub fn new( head_dim: usize, max_seq_len: usize, base: f32, device: &Device, ) -> Result<Self>

Create a new RoPE cache with precomputed frequencies

Arguments

• head_dim - Dimension of each attention head
• max_seq_len - Maximum sequence length to precompute
• base - RoPE base frequency (typically 10000.0 for LLaMA)
• device - Device to store tensors on

pub fn get(&self, seq_len: usize) -> Result<(Tensor, Tensor)>

Get precomputed cos/sin for a specific sequence length

Returns slices of the precomputed tensors for positions [0, seq_len)

pub fn get_range(&self, start: usize, end: usize) -> Result<(Tensor, Tensor)>

Get precomputed cos/sin for a range of positions

Useful for KV cache scenarios where we only need positions [start, end)

pub fn head_dim(&self) -> usize

Get the head dimension

pub fn max_seq_len(&self) -> usize

Get the maximum sequence length

pub fn base(&self) -> f32

Get the RoPE base frequency