Struct BufferPool 

pub struct BufferPool { /* private fields */ }

Free lists of GPU buffers keyed by exact byte size.

Returned buffers are kept for reuse until the pool holds max_pooled_bytes; beyond that they are released. Without the cap a workload with many distinct tensor sizes (e.g. training on sequences of varying length) would keep every buffer it ever freed.

Implementations

impl BufferPool

pub fn new(usage: BufferUsages) -> Self

A pool whose buffers are created with usage.

pub fn set_max_pooled_bytes(&self, max: u64)

Change how many bytes of freed buffers the pool keeps (0 disables reuse). Buffers above the new cap are released immediately.

pub fn pooled_bytes(&self) -> u64

Bytes currently held for reuse.

pub fn get(&self, device: &Device, size: u64) -> Arc<Buffer>

A free buffer of exactly size bytes, or a new one.

pub fn return_buffer(&self, size: u64, buffer: Arc<Buffer>)

Returns buffer (of size bytes) to the pool for reuse.

pub fn clear(&self)

Drops every pooled buffer.

Trait Implementations

impl Debug for BufferPool

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.