Skip to main content

kv_len_ptr_table

Function kv_len_ptr_table 

Source
pub fn kv_len_ptr_table(
    e: &Engine,
    cache: &Cache,
    pos_ctr: Option<&CudaSlice<i32>>,
) -> Result<CudaSlice<u64>, Box<dyn Error>>
Expand description

Per-layer kvl.len_d device-pointer table (+ the position counter appended when pos_ctr is given) for spec_rollback_stream. Pointers are stable for the cache’s lifetime; 0 marks layers without KV (linear-attention / KV-shared).