Struct KvLayer 

pub struct KvLayer {
    pub k: CudaSlice<u8>,
    pub v: CudaSlice<u8>,
    pub kv_dim_k: usize,
    pub kv_dim_v: usize,
    pub k_tok_bytes: usize,
    pub v_tok_bytes: usize,
    pub len: usize,
    pub ring: Option<KvRing>,
    pub len_d: CudaSlice<i32>,
}

Per-full-attn-layer growing KV cache, resident on GPU. QUANTIZED (KVQUANT-PLAN §B): K stored q8_0 (34 B/32 elem), V stored q5_1 (24 B/32 elem). Per-token byte layout keeps the [token, kv_head, dim] element order so a 32-block never straddles a head (assert head_dim%32==0). Element-within-token index = kv_head*head_dim + d; block = idx/32; lane = idx%32.

Fields

k: CudaSlice<u8>

v: CudaSlice<u8>

kv_dim_k: usize

kv_dim_v: usize

k_tok_bytes: usize

v_tok_bytes: usize

len: usize

ring: Option<KvRing>

Step35 SWA physical-row state. len remains absolute; None keeps the original flat [0, max_ctx) addressing contract.

len_d: CudaSlice<i32>

Device-resident mirror of len (CUDA-GRAPH-PLAN Phase 2). Holds the KV write SLOT for the append-dc kernel (old len, before this step’s append); after inc_seqlen it holds the new len == t_kv for fa_decode_dc. Kept in lock-step with the host len. i32[1].

Implementations

impl KvLayer

pub fn physical_rows( &self, start: usize, end: usize, ) -> Result<Range<usize>, String>