Struct BufferPool 

pub struct BufferPool { /* private fields */ }

A pool of reusable GPU buffers, organized by size buckets

Design Rationale

GPU buffer allocation is expensive (involves driver calls and memory mapping). This pool enables buffer reuse to:

1. Reduce allocation overhead for repeated operations
2. Minimize memory fragmentation
3. Prevent memory exhaustion during benchmarking/training

Buffers are bucketed by size (rounded up to power-of-2) for efficient matching. A 5000-byte request gets a buffer from the 8192-byte bucket.

Thread Safety

All operations are protected by a Mutex, making the pool safe for concurrent access (though Volta is currently single-threaded, this future-proofs the design).

Implementations

impl BufferPool

pub fn new(config: BufferPoolConfig) -> Self

Create a new buffer pool with the given configuration

pub fn with_defaults() -> Self

Create a buffer pool with default configuration

pub fn allocation_size(min_size: usize) -> usize

Get the allocation size for a given minimum size

This rounds up to the next power of 2, ensuring all buffers in a bucket have the same size and can be safely reused.

pub fn acquire(&self, min_size: usize) -> Option<Buffer>

Try to acquire a buffer from the pool

Returns Some(buffer) if a suitable buffer is available, None if the pool is empty for that size or the size is too large.

Arguments

• min_size - Minimum required size in bytes

pub fn release(&self, buffer: Buffer, size: usize) -> bool

Return a buffer to the pool for reuse

Returns true if the buffer was pooled, false if it was rejected (e.g., pool full or buffer too large). Rejected buffers should be dropped normally.

Arguments

• buffer - The GPU buffer to return
• size - The buffer size in bytes

pub fn clear(&self)

Clear all buffers from the pool

This immediately frees all GPU memory held by the pool. Useful for reducing memory pressure or before shutdown.

pub fn stats(&self) -> BufferPoolStats

Get statistics about pool usage