Struct RKLLMCrossAttnParam 

#[repr(C)]
pub struct RKLLMCrossAttnParam {
    pub encoder_k_cache: *mut f32,
    pub encoder_v_cache: *mut f32,
    pub encoder_mask: *mut f32,
    pub encoder_pos: *mut i32,
    pub num_tokens: c_int,
}

@struct RKLLMCrossAttnParam @brief Structure holding parameters for cross-attention inference.

This structure is used when performing cross-attention in the decoder. It provides the encoder output (key/value caches), position indices, and attention mask.

• encoder_k_cache must be stored in contiguous memory with layout: [num_layers][num_tokens][num_kv_heads][head_dim]
• encoder_v_cache must be stored in contiguous memory with layout: [num_layers][num_kv_heads][head_dim][num_tokens]

Fields

encoder_k_cache: *mut f32

< Pointer to encoder key cache (size: num_layers * num_tokens * num_kv_heads * head_dim).

encoder_v_cache: *mut f32

< Pointer to encoder value cache (size: num_layers * num_kv_heads * head_dim * num_tokens).

encoder_mask: *mut f32

< Pointer to encoder attention mask (array of size num_tokens).

encoder_pos: *mut i32

< Pointer to encoder token positions (array of size num_tokens).

num_tokens: c_int

< Number of tokens in the encoder sequence.

Trait Implementations

impl Clone for RKLLMCrossAttnParam

fn clone(&self) -> RKLLMCrossAttnParam

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for RKLLMCrossAttnParam

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Copy for RKLLMCrossAttnParam