Skip to main content

batch_attention_compute

Function batch_attention_compute 

Source
pub async fn batch_attention_compute(
    queries: Vec<Float32Array>,
    keys: Vec<Vec<Float32Array>>,
    values: Vec<Vec<Float32Array>>,
    dim: u32,
) -> Result<BatchResult>
Expand description

Process a batch of attention computations