pub fn cache_ring_bytes_per_token(cfg: &ModelConfig) -> usizeExpand description
Portion of cache_bytes_per_token whose physical row count is capped by the Step35 SWA
ring. Zero with the flag off and for every non-Step35 architecture.
pub fn cache_ring_bytes_per_token(cfg: &ModelConfig) -> usizePortion of cache_bytes_per_token whose physical row count is capped by the Step35 SWA
ring. Zero with the flag off and for every non-Step35 architecture.