Struct FrozenMMap 

pub struct FrozenMMap<T, const MODULE_ID: u8>
where
    T: Sized + Send + Sync,
{ /* private fields */ }

Custom implementation of mmap(2)

Constraints

FrozenMMap treats the mapped file as raw storage for values of T. Because of that, T must be a POD type which is safe to persist and later reinterpret from bytes.

Required properties for T,

• Must use #[repr(C)]
• Should be 8-bytes aligned
• Must not implement Drop
• size_of::<T>() should be multiple of 8

NOTE: T must not contain heap owning or process-local pointers like Vec, String, Box, references and function pointers, or other fields whose bit-pattern is not stable across reopen.

These constrains are enforced as FrozenMMap does not serialize or deserialize values. It directly reads and writes T inside a memory mapped file. That means T must have a stable layout and must remain valid when the file is reopened in a later process.

Example

use frozen_core::fmmap::{FrozenMMap, FMCfg};

const MODULE_ID: u8 = 0;

let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tmp_frozen_mmap");

let cfg = FMCfg {
    initial_count: 0x0A,
    flush_duration: std::time::Duration::from_micros(0x96),
};

let mmap = FrozenMMap::<u64, MODULE_ID>::new(&path, cfg.clone()).unwrap();
assert_eq!(mmap.total_slots(), 0x0A);

let epoch = unsafe { mmap.write(0, |v| *v = 0xDEADC0DE) }.unwrap();
mmap.wait_for_durability(epoch).unwrap();

let val = unsafe { mmap.read(0, |v| *v) }.unwrap();
assert_eq!(val, 0xDEADC0DE);

drop(mmap);

let reopened = FrozenMMap::<u64, MODULE_ID>::new_grown(&path, cfg, 0x05).unwrap();
assert_eq!(reopened.total_slots(), 0x0A + 0x05);

let val = unsafe { reopened.read(0, |v| *v) }.unwrap();
assert_eq!(val, 0xDEADC0DE);

Implementations

impl<T, const MODULE_ID: u8> FrozenMMap<T, MODULE_ID>

where T: Sized + Send + Sync,

pub const SLOT_SIZE: usize

Memory space required for each slot of [T] in FrozenMMap

pub fn new<P: AsRef<Path>>(path: P, cfg: FMCfg) -> FrozenResult<Self>

Create a new FrozenMMap instance w/ given FMCfg

Multiple Instances

For each FrozenMMap instance, we acquire an exclusive lock from the kernal for the underlying FrozenFile, when trying to create multiple instances of FrozenMMap, an error will be thrown ///

Capacity Growth

FrozenMMap does not support in-place growth of a live mapping, to increase capacity, drop the current instance and reopen w/ [FrozenMMap::open_grown] which provides memory mapping over grown capacity

FMCfg

All configs for FrozenMMap are stored in FMCfg

Working

We first create a new FrozenFile if note already, then map the entire file using mmap(2), the entire file must read/write T, which also should stay constant for the entire lifetime of file

Important

The cfg must not change any of its properties for the entire life of FrozenFile, which is used under the hood, one must use config stores like Rta to store config

Example

use frozen_core::fmmap::{FrozenMMap, FMCfg};

const MODULE_ID: u8 = 0;

let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tmp_frozen_mmap");

let cfg = FMCfg {
    initial_count: 0x0A,
    flush_duration: std::time::Duration::from_micros(0x96),
};

let mmap = FrozenMMap::<u64, MODULE_ID>::new(&path, cfg.clone()).unwrap();
assert_eq!(mmap.total_slots(), 0x0A);

let epoch = unsafe { mmap.write(0, |v| *v = 0xDEADC0DE) }.unwrap();
mmap.wait_for_durability(epoch).unwrap();

let val = unsafe { mmap.read(0, |v| *v) }.unwrap();
assert_eq!(val, 0xDEADC0DE);

pub fn new_grown<P: AsRef<Path>>( path: P, cfg: FMCfg, additional_slots: usize, ) -> FrozenResult<Self>

Create a new FrozenMMap instance w/ given FMCfg, while growing the underlying FrozenFile by additional_slots before creating memory mapping

Multiple Instances

For each FrozenMMap instance, we acquire an exclusive lock from the kernal for the underlying FrozenFile, when trying to create multiple instances of FrozenMMap, an error will be thrown

Why not create a [FrozenMMap::grow] call?

Previously when [FrozenMMap::grow] was attempted, it was observed that, resizing an active memory mapping in place is tricky in concurrent code, as some threads would still hold stale/unmapped pointers to mmap due to preemption from the OS schedular

So, instead of remmaping a live instance, the current API performs growth during open, making capacity expansion an explicit lifecycle operation, and not a side effect on a live instance

FMCfg

All configs for FrozenMMap are stored in FMCfg

Working

We first create a new FrozenFile if note already, then we grow the file using FrozenFile::grow then map the entire file using mmap(2), the entire file must read/write T, which also should stay constant for the entire lifetime of file

Important

The cfg must not change any of its properties for the entire life of FrozenFile, which is used under the hood, one must use config stores like Rta to store config

Example

use frozen_core::fmmap::{FrozenMMap, FMCfg};

const MODULE_ID: u8 = 0;

let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tmp_frozen_mmap");

let cfg = FMCfg {
    initial_count: 0x0A,
    flush_duration: std::time::Duration::from_micros(0x96),
};

let mmap = FrozenMMap::<u64, MODULE_ID>::new_grown(&path, cfg.clone(), 0x0A).unwrap();
assert_eq!(mmap.total_slots(), 0x0A * 2);

let epoch = unsafe { mmap.write(0, |v| *v = 0xDEADC0DE) }.unwrap();
mmap.wait_for_durability(epoch).unwrap();

let val = unsafe { mmap.read(0, |v| *v) }.unwrap();
assert_eq!(val, 0xDEADC0DE);

pub fn wait_for_durability(&self, epoch: u64) -> FrozenResult<()>

Blocks until given epoch becomes durable

Batching

With respect to flush_duration, all write ops are batched before sync, which is executed by flusher tx working in background, while each write is assigned w/ current durable epoch, and all writes which observe the exact same epoch, belong to the same durability window, and are all sync’ed together

When a background sync succeeds, the internal durable epoch is incremented, indicating that all writes that observed the previous epoch are now durable on disk

Example

use frozen_core::fmmap::{FrozenMMap, FMCfg};

const MODULE_ID: u8 = 0;

let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tmp_wait_epoch");

let cfg = FMCfg {
    initial_count: 0x04,
    flush_duration: std::time::Duration::from_micros(0x60),
};

let mmap = FrozenMMap::<u64, MODULE_ID>::new(&path, cfg).unwrap();

let epoch = unsafe { mmap.write(0, |v| *v = 0x8A) }.unwrap();
mmap.wait_for_durability(epoch).unwrap();

let val = unsafe { mmap.read(0, |v| *v) }.unwrap();
assert_eq!(val, 0x8A);

pub unsafe fn read<R>( &self, index: usize, f: impl FnOnce(*const T) -> R, ) -> FrozenResult<R>

Read a T at given index via callback (f)

Concurrency

Internally, FrozenMMap implements per-slot locking, so concurrent reads and writes for at same index is atomic and thread safe, while operations on different indices may proceed fully in parallel

Safety

The caller must ensure following:

• given index is within bounds
• underlying memory contains a valid instance of T
• provided callback f must not,
  • write through the pointer
  • store or leak pointer beyound there lifetime

Violating any of the above may result in undefined behavior

Example

use frozen_core::fmmap::{FrozenMMap, FMCfg};

const MODULE_ID: u8 = 0;

let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tmp_read_mmap");

let cfg = FMCfg {
    initial_count: 0x02,
    flush_duration: std::time::Duration::from_micros(0x60),
};

let mmap = FrozenMMap::<u64, MODULE_ID>::new(&path, cfg).unwrap();

let epoch = unsafe { mmap.write(0, |v| *v = 0x0A) }.unwrap();
mmap.wait_for_durability(epoch).unwrap();

let val = unsafe { mmap.read(0, |v| *v) }.unwrap();
assert_eq!(val, 0x0A);

pub unsafe fn write( &self, index: usize, f: impl FnOnce(*mut T), ) -> FrozenResult<TEpoch>

Write/update a T at given index via callback (f)

Concurrency

Internally, FrozenMMap implements per-slot locking, so concurrent reads and writes for at same index is atomic and thread safe, while operations on different indices may proceed fully in parallel

Safety

The caller must ensure following:

• given index is within bounds
• underlying memory contains a valid instance of T
• provided callback f must not store or leak pointer beyound there lifetime

Violating any of the above may result in undefined behavior

Example

use frozen_core::fmmap::{FrozenMMap, FMCfg};

const MODULE_ID: u8 = 0;

let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tmp_write_mmap");

let cfg = FMCfg {
    initial_count: 0x02,
    flush_duration: std::time::Duration::from_micros(0x96),
};

let mmap = FrozenMMap::<u64, MODULE_ID>::new(&path, cfg).unwrap();

let epoch = unsafe {mmap.write(1, |v| *v = 0x2B) }.unwrap();
mmap.wait_for_durability(epoch).unwrap();

let val = unsafe { mmap.read(1, |v| *v) }.unwrap();
assert_eq!(val, 0x2B);

pub unsafe fn write_sync( &self, index: usize, f: impl FnOnce(*mut T), ) -> FrozenResult<()>

Write/update a T at given index via callback (f) w/ instant durability

This function performs a blocking hard-sync, unlike FrozenMMap::write, the update is immediately persisted to the underlying storage device

Concurrency

Internally, FrozenMMap implements per-slot locking, so concurrent reads and writes for at same index is atomic and thread safe, while operations on different indices may proceed fully in parallel

Safety

The caller must ensure following:

• given index is within bounds
• underlying memory contains a valid instance of T
• provided callback f must not store or leak pointer beyound there lifetime

Violating any of the above may result in undefined behavior

Example

use frozen_core::fmmap::{FrozenMMap, FMCfg};

const MODULE_ID: u8 = 0;

let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tmp_write_sync");

let cfg = FMCfg {
    initial_count: 0x02,
    flush_duration: std::time::Duration::from_micros(0x96),
};

let mmap = FrozenMMap::<u64, MODULE_ID>::new(&path, cfg).unwrap();
unsafe { mmap.write_sync(0, |v| *v = 0xC0DE) }.unwrap();

let val = unsafe { mmap.read(0, |v| *v) }.unwrap();
assert_eq!(val, 0xC0DE);

pub fn total_slots(&self) -> usize

Read current available count of slots, where each slot has size of FrozenMMap::<T>::SLOT_SIZE

Working

This call performs a syscall to fetch current length of FrozenFile from fs, as the current length of the file is not cached anywhere in the pipeline to avoid TOCTAU race conditions

Example

use frozen_core::fmmap::{FrozenMMap, FMCfg};

const MODULE_ID: u8 = 0;

let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tmp_grow_mmap");

let cfg = FMCfg {
    initial_count: 0x02,
    flush_duration: std::time::Duration::from_micros(0x96),
};

let mmap = FrozenMMap::<u64, MODULE_ID>::new(&path, cfg.clone()).unwrap();
assert_eq!(mmap.total_slots(), 0x02);

drop(mmap);

let mmap = FrozenMMap::<u64, MODULE_ID>::new_grown(&path, cfg, 0x03).unwrap();
assert_eq!(mmap.total_slots(), 0x02 + 0x03);

pub fn memory_usage(&self) -> usize

Read the total memory footprint (in bytes) used by FrozenMMap

NOTE: This is an approzimation of memory used, actual RSS may differ depending on paging and OS

Example

use frozen_core::fmmap::{FrozenMMap, FMCfg};

const MODULE_ID: u8 = 0;

let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tmp_mem_usage");

let cfg = FMCfg {
    initial_count: 0x10,
    flush_duration: std::time::Duration::from_micros(0x60),
};

let mmap = FrozenMMap::<u64, MODULE_ID>::new(&path, cfg).unwrap();

let bytes = mmap.memory_usage();
assert!(bytes >= mmap.total_slots() * std::mem::size_of::<u64>());

pub fn new_tx(&self) -> FMTransaction<'_, T>

Create a new FMTransaction context for grouping multi write ops into a single atomic operation

Overview

The use of FMTransaction allows to group multiple write ops into a single atomic operation, hence creating a transactional write operation, which gives following guarantees,

• All write ops succeed together
• Single epoch to track durability of all writes ops
• Same durability guarantee for all the included write ops

Simply, this preserves atomic durability semantics for multi index updates

Example

use frozen_core::fmmap::{FrozenMMap, FMCfg};

const MID: u8 = 0;

let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tmp_tx");

let cfg = FMCfg {
    initial_count: 0x0A,
    flush_duration: std::time::Duration::from_micros(50),
};

let mmap = FrozenMMap::<u64, MID>::new(&path, cfg).unwrap();

let mut tx = mmap.new_tx();
unsafe { tx.write(0, |v| *v = 0x0A) }.unwrap();
unsafe { tx.write(1, |v| *v = 0x14) }.unwrap();

let epoch = tx.commit().unwrap();
mmap.wait_for_durability(epoch).unwrap();

let v0 = unsafe { mmap.read(0, |v| *v).unwrap() };
let v1 = unsafe { mmap.read(1, |v| *v).unwrap() };

assert_eq!((v0, v1), (0x0A, 0x14));

pub fn delete(&mut self) -> FrozenResult<()>

Delete the underlying FrozenFile used for FrozenMMap from fs

Working

When delete is called, all read, write, and (background) sync ops are paused (indefinitely), whule deletion is done with following steps:

• acquire an exclusive io_lock (all other ops are paused indefinitely)
• if any batch is pending for sync,
  • swap the flag
  • call sync manually
  • incr epoch and update cv
• brodcast closing so flusher tx could wrap up
• munmap(2) current mapping
• call delete on FrozenFile

Example

use frozen_core::fmmap::{FrozenMMap, FMCfg};

const MODULE_ID: u8 = 0;

let dir = tempfile::tempdir().unwrap();
let path = dir.path().join("tmp_delete_mmap");

let cfg = FMCfg {
    initial_count: 0x04,
    flush_duration: std::time::Duration::from_micros(0x96),
};

let mut mmap = FrozenMMap::<u64, MODULE_ID>::new(&path, cfg).unwrap();
mmap.delete().unwrap();
assert!(!path.exists());

Trait Implementations

impl<T, const MODULE_ID: u8> Debug for FrozenMMap<T, MODULE_ID>

where T: Sized + Send + Sync + Debug,

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<T, const MODULE_ID: u8> Display for FrozenMMap<T, MODULE_ID>

where T: Sized + Send + Sync,

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<T, const MODULE_ID: u8> Drop for FrozenMMap<T, MODULE_ID>

where T: Sized + Send + Sync,

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.

impl<T, const MODULE_ID: u8> Send for FrozenMMap<T, MODULE_ID>

where T: Sized + Send + Sync,

impl<T, const MODULE_ID: u8> Sync for FrozenMMap<T, MODULE_ID>

where T: Sized + Send + Sync,