Struct Cache 

pub struct Cache {
    pub use_kv_cache: bool,
    /* private fields */
}

Cache for LFM2 model supporting both attention KV cache and convolution state cache.

Fields

use_kv_cache: bool

Implementations

impl Cache

pub fn new( use_kv_cache: bool, dtype: DType, config: &Config, device: &Device, ) -> Result<Self>

pub fn clear(&mut self)

Trait Implementations

impl Clone for Cache

fn clone(&self) -> Cache

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Cache

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.