Module bufpool 

A low-latency memory-budgeted buffer pool to manage fixed-sized buffer allocations

Memory Allocations

All the allocations are allocated using the global memory allocator as requested (on-the-fly).

While a single allocation retunred as BufPoolAllocation is a large contineous slice of memory w/ size as BufPoolCfg::buffer_size.bytes() * n_buffers.

Memory layout structure,

allocation = [[buf0][buf1][buf2]]
where,
  - every buffer is of size `buffer_size`
  - each buffer is pointed using `*mut u8`

Backpressure

Every allocation reserves a memory budget and is only allowed to allocate memory if enough budget (i.e. memory space) is available. Otherwise, the caller is blocked/polled till enough space is available.

When the BufPoolAllocation and all its references are dropped, the underlying memory is deallocated while relaxing the budget and dropping the backpressure (if any).

NOTE: There is no faireness guarantee for the caller’s who are polled when faced with backpressure, as the waiting callers are awaken opportunistically.

Benchmarks

Observed measurements of latency and throughput,

| Metric                       | Value              |
|:-----------------------------|:-------------------|
| Allocation Latency (avg)     | ~254 nanosecond    |
| Allocation Throughput (avg)  | ~3.94 million/sec  |

NOTE: All measurements includes the complete RAII lifecycle (i.e. allocation + deallocation).

Observed allocation latency for N buffers,

| Buffers  | Latency  |
|:---------|:---------|
| 0x01     | 246 ns   |
| 0x10     | 251 ns   |
| 0x400    | 300 ns   |

INFO: As seen, the allocation latency stays near constant irrespective to the size of buffers and the allocated bytes.

Environment used for benching,

• OS: NixOS (WSL2)
• Architecture: x86_64
• Memory: 8 GiB RAM (DDR4)
• Rust: rustc 1.86.0 w/ cargo 1.86.0
• Kernel: Linux 6.6.87.2-microsoft-standard-WSL2
• CPU: Intel® Core™ i5-10300H @ 2.50GHz (4C / 8T)

Example

use frozen_core::{
    bufpool::{BufPool, BufPoolCfg, BufferPointer},
    utils::BufferSize,
};

const BUF_SIZE: BufferSize = BufferSize::S16;

let pool = BufPool::new(BufPoolCfg {
    buffer_size: BUF_SIZE,
    max_memory: BUF_SIZE as usize * 0x40,
});

let alloc = pool.allocate(0x2A);

assert_eq!(alloc.length(), 0x2A);
assert!(!alloc.first().is_null());
assert_eq!(alloc.allocated_bytes(), BUF_SIZE as usize * 0x2A);

let ptrs: Vec<BufferPointer> = alloc.iter().collect();
assert_eq!(ptrs.len(), 0x2A);

Structs

BufPool

An implementation of a low-latency memory-budgeted buffer pool managing fixed-sized buffer allocations

BufPoolAllocation

A RAII safe allocation object containing allocated buffers

BufPoolAllocationIter

A custom Iterator object to iterate over the list of allocated buffers

BufPoolCfg

All the available configrations for BufPool

Type Aliases

BufferPointer

An unsafe pointer to an individual in memory buffer