burn-msa 0.2.4

MiniMax Sparse Attention (MSA) - block-sparse attention with per-GQA-group top-k block selection for Burn
Documentation

burn-msa - MiniMax Sparse Attention

Crates.io License: AGPL-3.0 Burn MS-DSA

Block-sparse attention for the Burn deep learning framework. Reduces attention from O(S_q × S_kv) to O(S_q × topk × block_size) by selecting the top-K KV blocks per query position via a lightweight index branch.

Algorithm reference: MiniMax Sparse Attention.

Per-GQA-group top-k block selection with GPU-sync-free indexer. 5D batched matmul kernel launches one kernel per KV head instead of per-token per-head. Works on any Burn backend: CUDA, WGPU, NdArray.

Requirements

  • Rust: ≥ 1.75
  • Burn: 0.21
  • GPU (optional): CUDA or Vulkan/Metal via WGPU

Install

cargo add burn-msa

Or in Cargo.toml:

[dependencies]
burn-msa = "0.1"

Enable CUDA:

burn-msa = { version = "0.1", features = ["cuda"] }

Quick start

use burn_ndarray::NdArray;
use burn_msa::{MsaConfig, MsaModule};

let device = Default::default();
let cfg = MsaConfig::new(768, 12, 4, 64, 64);
let module = MsaModule::<NdArray>::new(&cfg, &device);

// Self-attention: Q, K, V from the same input
let x = Tensor::random([1, 128, 768], Distribution::Normal(0.0, 1.0), &device);
let result = module.forward(x);
let output = result.output;  // [batch, seq, d_model]

// Cross-attention: separate Q and KV inputs
let q = Tensor::random([1, 64, 768], Distribution::Normal(0.0, 1.0), &device);
let kv = Tensor::random([1, 256, 768], Distribution::Normal(0.0, 1.0), &device);
let result = module.forward_cross(q, kv);

// Dense GQA fallback (no sparsity)
let output = module.forward_dense(hidden_states, kv_states);

Architecture

Input [B, T, D] ──→ IndexBranch ──→ TopKSelector ──→ SparseAttention ──→ Output
                      │                   │                  │
                      │ Q·K^T / √d_idx   │ topk blocks      │ gather K/V
                      │ per-block max    │ per GQA group    │ 5D matmul
                      │                  │                  │ softmax
                      │                  │                  │ output proj
  1. Index branch projects Q/K into low-dim space (d_idx), scores each KV block via max-pooling.
  2. TopK selector picks topk highest-scoring blocks per GQA group - GPU-sync-free.
  3. Sparse attention gathers K/V from selected blocks via single batched gather, then 5D batched matmul for scores and output.

Configuration

Field Default Description
d_model 1152 Hidden size
n_heads_q 18 Query heads
n_heads_kv 1 KV heads (GQA factor)
d_head 64 Head dimension
d_idx 32 Index projection dimension
block_size 128 Tokens per KV block
topk 16 Blocks selected per query
causal true Causal masking
use_kl_loss true KL alignment loss for index branch
kl_coeff 0.1 KL loss weight

Performance

Benchmarked on RTX 5060 Ti, Burn 0.21 vs PyTorch 2.13.

Attention kernel (gather + matmul + softmax + output) is 1.2-8.2× faster across all 21 tested configs. Dense attention: 1.1-1.5× faster.

Full sparse forward (index branch + topk + attention) is faster for small configs and large-scale prefill, but on par or slightly slower for medium configs where the index branch overhead dominates. This gap comes from Burn's cubecl matmul vs PyTorch's cuBLAS for small matrix sizes. For long contexts (>32K) the attention kernel dominates and burn-msa wins decisively.

Config Burn CUDA PyTorch CUDA Attention speedup Full sparse
small (d=128, seq=256) 361 µs 431 µs 1.26× 1.19×
large (d=512, seq=256, kv=4096) 1,508 µs 1,897 µs 1.26× 0.70×
xl (d=768, seq=512, kv=8192) 26,028 µs 29,772 µs 1.14× 0.92×
decode (d=512, seq=1, kv=64K) 159 µs 1,303 µs 8.2× 0.57×

Training with KL alignment

let mut cfg = MsaConfig::new(768, 12, 4, 64, 64);
cfg.use_kl_loss = true;
cfg.kl_coeff = 0.1;

let module = MsaModule::<NdArray>::new(&cfg, &device);
let result = module.forward(x);
if let Some(kl_loss) = result.kl_loss {
    let total_loss = ce_loss + kl_loss;
}

KV-cache decoding

use burn_msa::MsaCache;

let cache = MsaCache::new(k, v, k_idx);
let cache = cache.update(new_k, new_v, new_k_idx);

Layout

src/
  lib.rs              Crate root, re-exports
  attention.rs        SparseAttention, 5D batched GQA kernel
  index_branch.rs     Low-dim Q/K projections, block scoring
  topk.rs             GPU-sync-free top-K block selector
  module.rs           MsaModule: unified forward API
  config.rs           MsaConfig with validation
  loss.rs             KL alignment loss
  cache.rs            KV-cache for incremental decoding
  kernel/             Experimental cubecl kernels
tests/
  basic.rs            22 unit + integration tests
  bench.rs            10-section performance benchmarks
  bench_vs_pt.rs      Head-to-head PyTorch comparison
examples/
  simple_msa.rs       Minimal usage example

License

AGPL-3.0. See LICENSE.