Skip to main content

kv_append

Function kv_append 

Source
pub fn kv_append(cache: &mut Tensor, src: &Tensor, len: usize) -> Result<()>
Expand description

Append src [h, t, hd] into the preallocated cache [h, cap, hd] at row offset len within each head, in place. Used by KV-cache decode.