Expand description
Tensor memory pool with LRU eviction.
The pool reduces allocation pressure for intermediate tensors that are created and freed on every forward pass.
Structsยง
- Pool
Allocator - LRU memory pool for CPU tensor buffers.
Tensor memory pool with LRU eviction.
The pool reduces allocation pressure for intermediate tensors that are created and freed on every forward pass.