Struct RotaryPositionEmbedding 

pub struct RotaryPositionEmbedding {
    pub head_dim: usize,
    pub base: f64,
    pub max_seq_len: usize,
    /* private fields */
}

Rotary Position Embedding (RoPE) as introduced in Su et al. 2021.

Pre-computes cosine and sine caches up to max_seq_len positions and applies the rotation in-place to the last dimension of the input tensor.

Fields

head_dim: usize

Dimension of each attention head.

base: f64

Base for the geometric frequency sequence (default 10000.0).

max_seq_len: usize

Maximum sequence length for which the cache was pre-computed.

Implementations

impl RotaryPositionEmbedding

pub fn new( head_dim: usize, max_seq_len: usize, base: f64, ) -> Result<Self, PositionError>

Create a new RoPE module, pre-computing the cos/sin cache.

Returns an error if head_dim is not even.

pub fn apply( &self, x: &ArrayD<f64>, seq_offset: usize, ) -> Result<ArrayD<f64>, PositionError>

Apply RoPE to the input tensor starting at seq_offset.

x is expected to have shape [seq_len, ..., head_dim] where the last axis is the head dimension (or any shape where the last axis == head_dim).

pub fn rotate_half(x: &ArrayD<f64>) -> ArrayD<f64>

Compute rotate_half: negate the first half and concatenate with the second half.

Given x shaped [..., head_dim], returns [-x[..., head_dim/2:], x[..., :head_dim/2]].

pub fn frequencies_at(&self, pos: usize) -> ArrayView1<'_, f64>

Return the pre-computed frequencies (cos values) at a specific position.

Trait Implementations

impl Clone for RotaryPositionEmbedding

fn clone(&self) -> RotaryPositionEmbedding

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for RotaryPositionEmbedding

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.