pub fn new_cache(
e: &Engine,
cfg: &ModelConfig,
max_ctx: usize,
) -> Result<Cache, Box<dyn Error>>Expand description
Stage-owned cache allocation door: when the ppN door is open AND MEMRA_PP_DEVICES
is set (placement plumbing), each layer’s cache is allocated by its OWNING stage’s
engine — on one device this is byte-for-byte today’s allocation (gated); cross-device
it puts each stage’s KV on that stage’s HBM. Door shut or devices unset: plain
Cache::new (zero behavior change). Trailing MTP/NextN layers (beyond the trunk)
map to the LAST stage.