Module precompute 

Precomputed static tensors for performance (RoPE, causal masks) Precomputed Static Tensors for Performance

This module provides caching utilities for static tensors that don’t change during inference. By precomputing these once at model initialization, we avoid redundant computation on every forward pass.

Supported caches:

• RoPECache: Precomputed rotary position embeddings (cos/sin frequencies)
• CausalMaskCache: Precomputed causal attention masks

These caches work with ALL model architectures:

• RoPE: LLaMA, Qwen, Gemma, Mistral, Phi, DeepSeek, etc.
• Causal mask: All autoregressive decoder models

Structs

CausalMaskCache

Precomputed causal attention mask

RoPECache

Precomputed RoPE (Rotary Position Embedding) frequencies

SlidingWindowMaskCache

Precomputed sliding window attention mask