Enum LayerState 

pub enum LayerState {
    Attention {
        key_cache: Vec<f32>,
        value_cache: Vec<f32>,
        key_cache_f16: Vec<u16>,
        value_cache_f16: Vec<u16>,
        compressed_keys: Option<CompressedKeyCache>,
        compressed_values: Option<CompressedValueCache>,
    },
    Conv {
        buffer: Vec<f32>,
        history: ConvHistory,
    },
}

Variants

Attention

KV cache for attention layers.

Fields

key_cache: Vec<f32>

value_cache: Vec<f32>

key_cache_f16: Vec<u16>

f16 key cache (IEEE-754 half bits). Populated when KvCompression::F16 is active; key_cache stays empty in that mode. Time-major [seq_len × kv_dim], same layout as key_cache.

value_cache_f16: Vec<u16>

f16 value cache (IEEE-754 half bits). Populated when KvCompression::F16 is active; value_cache stays empty in that mode.

compressed_keys: Option<CompressedKeyCache>

Compressed key cache (populated when TurboQuant is active; key_cache stays empty).

compressed_values: Option<CompressedValueCache>

Compressed value cache (populated when TurboQuant is active; value_cache stays empty).

Conv

Rolling buffer for convolution layers. Stores previous d_conv pre-conv activations (bx values), time-major.

Fields

buffer: Vec<f32>

history: ConvHistory

Pre-allocated ring buffer history snapshots for speculative decoding rollback.