Expand description
Memory pooling for efficient tensor allocation.
Reduces allocation overhead by reusing memory buffers across operations. Critical for low-latency, high-throughput inference.
Structs§
- Memory
Pool - Memory pool for efficient allocation and reuse of buffers.
- Pool
Config - Configuration for memory pool.
- Pool
Stats - Statistics about memory pool usage.
- Pooled
Buffer - A memory buffer that can be reused.
Functions§
- global_
pool - Get the global memory pool instance.