Struct KvLayer 

pub struct KvLayer {
    pub k: CudaSlice<u8>,
    pub v: CudaSlice<u8>,
    pub kv_dim_k: usize,
    pub kv_dim_v: usize,
    pub k_tok_bytes: usize,
    pub v_tok_bytes: usize,
    pub len: usize,
    pub ring: Option<KvRing>,
    pub len_d: CudaSlice<i32>,
    pub base_d: Option<CudaSlice<i32>>,
}

Per-full-attn-layer growing KV cache, resident on GPU. QUANTIZED (KVQUANT-PLAN §B): K stored q8_0 (34 B/32 elem), V stored q5_1 (24 B/32 elem). Per-token byte layout keeps the [token, kv_head, dim] element order so a 32-block never straddles a head (assert head_dim%32==0). Element-within-token index = kv_head*head_dim + d; block = idx/32; lane = idx%32.

Fields

k: CudaSlice<u8>

v: CudaSlice<u8>

kv_dim_k: usize

kv_dim_v: usize

k_tok_bytes: usize

v_tok_bytes: usize

len: usize

ring: Option<KvRing>

Step35 SWA physical-row state. len remains absolute; None keeps the original flat [0, max_ctx) addressing contract.

len_d: CudaSlice<i32>

Device-resident mirror of len (CUDA-GRAPH-PLAN Phase 2). Holds the KV write SLOT for the append-dc kernel (old len, before this step’s append); after inc_seqlen it holds the new len == t_kv for fa_decode_dc. Kept in lock-step with the host len. i32[1].

base_d: Option<CudaSlice<i32>>

Device-resident mirror of ring.base() (physical row of logical row 0) for the WINDOWED device-counter draft arm (append_kv_quantized_dcw / fa_decode_dcw): the kernels derive the SWA view as {lstart = max(0, len - window); physical = row - base} entirely from device state, so a captured draft chain replays with zero per-token node updates. Armed only on ring-backed draft-scratch planes (step35); None keeps the plain _dc contract (base 0) and costs nothing. The ONE writer is the rebase arm of prepare_kv_append (rebases are host-side, outside any captured region); rewinds move len/len_d only, never base, so no other site touches it. i32[1].

Implementations

impl KvLayer

pub fn physical_rows( &self, start: usize, end: usize, ) -> Result<Range<usize>, String>