Struct CacheConfig 

pub struct CacheConfig {
    pub max_seq_len: usize,
    pub page_size: usize,
    pub kind: CacheKind,
}

Configuration for a KV cache instance.

Fields

max_seq_len: usize

Maximum number of cached positions (arena capacity).

page_size: usize

Tokens per page (paged cache only).

kind: CacheKind

Implementation to use.

Implementations

impl CacheConfig

pub const DEFAULT_PAGE_SIZE: usize = 16

Default page size (MLC uses 16 as well).

pub fn paged(max_seq_len: usize) -> Self

Paged cache with the default page size.

pub fn contiguous(max_seq_len: usize) -> Self

Contiguous (baseline) cache.

pub fn num_pages(&self) -> usize

Number of pages in the arena.

Trait Implementations

impl Clone for CacheConfig

fn clone(&self) -> CacheConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for CacheConfig

impl Debug for CacheConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.