Enum DecodeOutput 

pub enum DecodeOutput {
    Fused(Tensor),
    Dequantized(DequantResult),
}

Result of a decode step.

The cache implementation decides internally whether to compute attention via a fused kernel or return decompressed data for the caller’s SDPA.

Variants

Fused(Tensor)

The implementation computed attention internally (e.g. fused CUDA kernel). Shape: [batch, num_attention_heads, 1, head_dim]

Dequantized(DequantResult)

The implementation decompressed the cache — caller runs SDPA. Used on CPU, Metal, or when fused attention is not available.