Skip to main content

cache_ring_bytes_per_token_for_layers

memra_engine::cache

Function cache_ring_bytes_per_token_for_layers 

Source 
pub fn cache_ring_bytes_per_token_for_layers(
    cfg: &ModelConfig,
    lo: usize,
    hi: usize,
) -> usize
Expand description

Ring-capped portion of cache_bytes_per_token_for_layers for [lo, hi).