Skip to main content

ChunkPool

Struct ChunkPool 

Source
pub struct ChunkPool<'a> { /* private fields */ }
Expand description

A pool of 64-byte chunks backing many independent append-only lists.

use plugmem_arena::{ChunkPool, ChunkPoolCfg, ListHandle};

let mut pool = ChunkPool::new(ChunkPoolCfg::new());
let mut evens = ListHandle::EMPTY;
let mut odds = ListHandle::EMPTY;
for n in 0u32..10 {
    let list = if n % 2 == 0 { &mut evens } else { &mut odds };
    pool.push(list, &n.to_be_bytes()).unwrap();
}
let bytes: Vec<u8> = pool.iter(&evens).flatten().copied().collect();
let evens_back: Vec<u32> = bytes
    .chunks_exact(4)
    .map(|b| u32::from_be_bytes(b.try_into().unwrap()))
    .collect();
assert_eq!(evens_back, [0, 2, 4, 6, 8]);
pool.free(&mut odds); // chunks recycle; `evens` is untouched

Implementations§

Source§

impl<'a> ChunkPool<'a>

Source

pub const fn new(cfg: ChunkPoolCfg) -> Self

Creates an empty pool. Allocates nothing until the first push.

Source

pub fn push(&mut self, list: &mut ListHandle, value: &[u8]) -> Result<(), Error>

Appends one value to a list. Zero-length values only bump the handle’s len.

§Errors
Source

pub fn free(&mut self, list: &mut ListHandle)

Returns the whole chain of a list to the free-list and resets the handle to ListHandle::EMPTY. O(1): the chain is spliced onto the free-list as-is, chunk by chunk bookkeeping happens on reuse.

Source

pub fn iter<'s>(&'s self, list: &ListHandle) -> ChunkIter<'s>

Iterates a list’s bytes as one &[u8] slice per chunk, in push order. Slices contain exactly the pushed bytes (skipped tail bytes of earlier chunks are excluded), and no pushed value ever spans two slices.

Source

pub fn pool_bytes(&self) -> usize

Total bytes held by the pool (allocated chunks, including free ones).

Source

pub fn chunks(&self) -> usize

Number of chunks the pool holds (used and free alike).

Source

pub fn validate_chain( &self, list: &ListHandle, visited: &mut [bool], ) -> Result<(), Error>

Walks one list’s chain, marking every chunk in visited (visited.len() must equal ChunkPool::chunks).

This is the owner’s load-time validation hook: the pool cannot see the owners’ handles, so cycle-freedom and exclusive ownership of chains are checked here — a shared bitmap across all of an owner’s handles catches a chunk claimed twice, a cycle (the chain would revisit), and a chain that leaks past its tail. Never panics on untrusted handle bytes.

§Errors

Error::Corrupt naming the violated invariant.

Source

pub fn orphan_count(&self, claimed: &[bool]) -> usize

Counts chunks that are neither claimed (per the owner’s visited map from ChunkPool::validate_chain walks) nor in the free-list — leaked chunks that no snapshot writer produces, so a loader treats any as corruption.

Source

pub fn dump_meta(&self, out: &mut Vec<u8>)

Appends the pool’s metadata section to out.

Layout (little-endian): [chunks u32][free_head u32] then one used u8 per chunk. Free chunks are written with used = 0 regardless of their stale in-memory value, making the dump canonical. Together with ChunkPool::dump_pool this is the complete pool-side state; the list handles live with their owners.

Source

pub fn dump_pool(&self, out: &mut Vec<u8>)

Appends the pool section to out.

Each chunk contributes its [LINK_BYTES] link, its used payload prefix and zero padding to CHUNK_BYTES; free chunks contribute link plus zeros. This canonicalizes the stale bytes recycling leaves behind (see ChunkPool::free) — identical logical state, identical bytes.

Source

pub fn load(cfg: ChunkPoolCfg, meta: &[u8], pool: &[u8]) -> Result<Self, Error>

Rebuilds a pool from its two dumped sections.

The input is untrusted — validation is O(chunks), never panics on arbitrary bytes: exact section lengths (within cfg.max_bytes), per-chunk used within CHUNK_PAYLOAD, every chain link in bounds (or the NONE sentinel — this is what keeps a later ChunkPool::iter from indexing out of bounds), and a free-list that is acyclic (visited bitmap) with used = 0 on every member.

What this method cannot check: cycles among chunks referenced by the owners’ ListHandles — the handles live in the owner’s records, so the owning engine walks its handles with a shared visited bitmap as part of its own load.

§Errors

Error::Corrupt for any inconsistency.

Source

pub fn load_borrowed( cfg: ChunkPoolCfg, meta: &[u8], pool: &'a [u8], ) -> Result<Self, Error>

Rebuilds a pool that borrows its chunk bytes from a longer-lived buffer (a memory-mapped snapshot) instead of copying them. Validation is identical to ChunkPool::load; the free-list walk and per-chunk link check touch the chunk bytes (needed for safety), but no byte is copied.

§Errors

Error::Corrupt for any inconsistency (same gates as load).

Source

pub fn load_overlay( cfg: ChunkPoolCfg, meta: &[u8], pool: &'a [u8], ) -> Result<Self, Error>

Opens a pool over a borrowed base for the overlay write path: the chunk bytes are mapped read-only, and the first write to any base chunk (a value push, or a chain/free-list link update) copies just that chunk into owned storage (per-page copy-on-write, see Paged), while chunks grown after open live in an owned tail. Unlike ChunkPool::load_borrowed the returned pool is fully mutable — pushes and frees work — yet the borrowed base is never cloned as a whole or mutated, so a memory-mapped pool can be written to while resident only in the chunks it actually touches. Validation is identical to ChunkPool::load.

§Errors

Error::Corrupt for any inconsistency (same gates as load).

Trait Implementations§

Source§

impl Debug for ChunkPool<'_>

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Summary only — chunk contents are the owners’ data, not ours to dump.

Auto Trait Implementations§

§

impl<'a> Freeze for ChunkPool<'a>

§

impl<'a> RefUnwindSafe for ChunkPool<'a>

§

impl<'a> Send for ChunkPool<'a>

§

impl<'a> Sync for ChunkPool<'a>

§

impl<'a> Unpin for ChunkPool<'a>

§

impl<'a> UnsafeUnpin for ChunkPool<'a>

§

impl<'a> UnwindSafe for ChunkPool<'a>

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.