Struct KvCacheSlotInfo 

pub struct KvCacheSlotInfo {
    pub position: u32,
    pub token_id: u32,
    pub layer: u16,
    pub head: u16,
    pub valid: bool,
    pub last_access: u64,
}

Metadata for a KV cache slot in transformer inference.

Tracks position, token info, and usage for cache management.

Fields

position: u32

Sequence position this slot represents

token_id: u32

Token ID stored in this slot

layer: u16

Layer index

head: u16

Head index

valid: bool

Whether this slot is valid/filled

last_access: u64

Last access time (in steps)

Implementations

impl KvCacheSlotInfo

pub fn new(position: u32, token_id: u32, layer: u16, head: u16) -> Self

Create a new slot info.

pub fn touch(&mut self, step: u64)

Mark slot as accessed.

pub fn invalidate(&mut self)

Invalidate the slot.

pub fn eviction_priority(&self, current_step: u64) -> u64

Check if slot can be evicted (LRU policy).

Trait Implementations

impl Clone for KvCacheSlotInfo

fn clone(&self) -> KvCacheSlotInfo

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for KvCacheSlotInfo

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for KvCacheSlotInfo

fn default() -> KvCacheSlotInfo

Returns the “default value” for a type.