Skip to main content

attention_causal_with_scale

Function attention_causal_with_scale 

Source
pub fn attention_causal_with_scale(
    q: &[f32],
    k_cache: &[f32],
    v_cache: &[f32],
    n_heads: usize,
    n_kv_heads: usize,
    head_dim: usize,
    scale: f32,
    window: Option<usize>,
) -> Result<Vec<f32>, EngineError>
Expand description

Prefill causal attention with an explicit softmax scale. window is the sliding-window length (None = full causal prefix).