Trait corundum::MemPoolTraits

source · [−]
pub unsafe trait MemPoolTraits where
    Self: 'static + Sized, {

Show 64 methods    unsafe fn pre_alloc(size: usize) -> (*mut u8, u64, usize, usize);
    unsafe fn pre_dealloc(ptr: *mut u8, size: usize) -> usize;

    fn name() -> &'static str { ... }
    fn open_no_root(_path: &str, _flags: u32) -> Result<PoolGuard<Self>> { ... }
    unsafe fn close() -> Result<()> { ... }
    fn zone(_off: u64) -> usize { ... }
    fn open<'a, U: 'a + PSafe + RootObj<Self>>(
        _path: &str, 
        _flags: u32
    ) -> Result<RootCell<'a, U, Self>>
    where
        Self: MemPool,
    { ... }
    fn is_open() -> bool { ... }
    unsafe fn format(_path: &str) -> Result<()> { ... }
    unsafe fn apply_flags(path: &str, flags: u32) -> Result<()> { ... }
    fn allocated(_off: u64, _len: usize) -> bool { ... }
    fn verify() -> bool { ... }
    unsafe fn off_unchecked<T: ?Sized>(x: *const T) -> u64 { ... }
    unsafe fn get_unchecked<'a, T: 'a + ?Sized>(off: u64) -> &'a T { ... }
    unsafe fn get_mut_unchecked<'a, T: 'a + ?Sized>(off: u64) -> &'a mut T { ... }
    unsafe fn deref_slice_unchecked<'a, T: 'a>(off: u64, len: usize) -> &'a [T]ⓘNotable traits for &'_ [u8]impl<'_> Read for &'_ [u8]impl<'_> Write for &'_ mut [u8] { ... }
    unsafe fn deref_slice_unchecked_mut<'a, T: 'a>(
        off: u64, 
        len: usize
    ) -> &'a mut [T]ⓘNotable traits for &'_ [u8]impl<'_> Read for &'_ [u8]impl<'_> Write for &'_ mut [u8] { ... }
    unsafe fn deref<'a, T: 'a>(off: u64) -> Result<&'a T> { ... }
    unsafe fn deref_mut<'a, T: 'a>(off: u64) -> Result<&'a mut T> { ... }
    fn off<T: ?Sized>(x: *const T) -> Result<u64> { ... }
    fn rng() -> Range<u64> { ... }
    fn start() -> u64 { ... }
    fn end() -> u64 { ... }
    fn size() -> usize { ... }
    fn available() -> usize { ... }
    fn used() -> usize { ... }
    fn valid<T: ?Sized>(p: *const T) -> bool { ... }
    fn contains(addr: u64) -> bool { ... }
    unsafe fn alloc(size: usize) -> (*mut u8, u64, usize) { ... }
    unsafe fn dealloc(ptr: *mut u8, size: usize) { ... }
    unsafe fn log64(_off: u64, _val: u64, _zone: usize) { ... }
    unsafe fn drop_on_failure(_off: u64, _len: usize, _zone: usize) { ... }
    unsafe fn prepare(_zone: usize) { ... }
    unsafe fn perform(_zone: usize) { ... }
    unsafe fn discard(_zone: usize) { ... }
    unsafe fn alloc_zeroed(size: usize) -> *mut u8 { ... }
    unsafe fn new<'a, T: PSafe + 'a>(x: T, j: &Journal<Self>) -> &'a mut T
    where
        Self: MemPool,
    { ... }
    unsafe fn new_slice<'a, T: PSafe + 'a>(
        x: &'a [T], 
        journal: &Journal<Self>
    ) -> &'a mut [T]ⓘNotable traits for &'_ [u8]impl<'_> Read for &'_ [u8]impl<'_> Write for &'_ mut [u8]
    where
        Self: MemPool,
    { ... }
    unsafe fn new_copy<'a, T: 'a>(x: &T, j: &Journal<Self>) -> &'a mut T
    where
        T: ?Sized,
        Self: MemPool,
    { ... }
    unsafe fn new_copy_slice<'a, T: 'a>(
        x: &[T], 
        j: &Journal<Self>
    ) -> &'a mut [T]ⓘNotable traits for &'_ [u8]impl<'_> Read for &'_ [u8]impl<'_> Write for &'_ mut [u8]
    where
        Self: MemPool,
    { ... }
    unsafe fn atomic_new<'a, T: 'a>(x: T) -> (&'a mut T, u64, usize, usize) { ... }
    unsafe fn atomic_new_slice<'a, T: 'a + PSafe>(
        x: &'a [T]
    ) -> (&'a mut [T], u64, usize, usize) { ... }
    unsafe fn new_uninit<'a, T: PSafe + 'a>(j: &Journal<Self>) -> &'a mut T
    where
        Self: MemPool,
    { ... }
    unsafe fn new_uninit_for_layout(
        size: usize, 
        journal: &Journal<Self>
    ) -> *mut u8
    where
        Self: MemPool,
    { ... }
    unsafe fn atomic_new_uninit<'a, T: 'a>() -> (&'a mut T, u64, usize, usize) { ... }
    unsafe fn alloc_for_value<'a, T: ?Sized>(x: &T) -> &'a mut T { ... }
    unsafe fn free<'a, T: PSafe + ?Sized>(x: &mut T)
    where
        Self: MemPool,
    { ... }
    unsafe fn free_slice<'a, T: PSafe>(x: &[T])
    where
        Self: MemPool,
    { ... }
    unsafe fn free_nolog<'a, T: ?Sized>(x: &T) { ... }
    unsafe fn drop_journal(_journal: &mut Journal<Self>)
    where
        Self: MemPool,
    { ... }
    unsafe fn journals_head() -> &'static u64 { ... }
    unsafe fn journals<T, F: Fn(&mut HashMap<ThreadId, (u64, i32)>) -> T>(
        _: F
    ) -> T { ... }
    unsafe fn recover() { ... }
    unsafe fn commit()
    where
        Self: MemPool,
    { ... }
    unsafe fn commit_no_clear()
    where
        Self: MemPool,
    { ... }
    unsafe fn clear()
    where
        Self: MemPool,
    { ... }
    unsafe fn rollback() -> bool
    where
        Self: MemPool,
    { ... }
    unsafe fn rollback_no_clear()
    where
        Self: MemPool,
    { ... }
    unsafe fn dealloc_history() -> *mut HashSet<u64> { ... }
    fn transaction<T, F: FnOnce(&'static Journal<Self>) -> T>(
        body: F
    ) -> Result<T>
    where
        F: TxInSafe + UnwindSafe,
        T: TxOutSafe,
        Self: MemPool,
    { ... }
    fn gen() -> u32 { ... }
    fn tx_gen() -> u32 { ... }
    fn print_info() { ... }
    fn stat_footprint() -> usize { ... }

}
Expand description

Persistent Memory Pool

This trait can be used to define a persistent memory pool type. The methods of MemPool trait do not have a reference to self in order to make sure that all information that it works with, including the virtual address boundaries, are static. Therefore, all objects with the same memory allocator will share a unique memory pool type. Having a strong set of type checking rules, Rust prevents referencing from one memory pool to another.

To implement a new memory pool, you should define a new type with static values, that implements MemPool. You may redefine the default allocator as a new pool using pool!() which creates a pool module and generates the necessary code segments of type Allocator.

Examples

The following example shows how to use MemPool to track allocations of a single numerical object of type i32.

use std::alloc::{alloc,dealloc,realloc,Layout};

struct TrackAlloc {}

unsafe impl MemPool for TrackAlloc {
    fn rng() -> Range<u64> { 0..u64::MAX }
    unsafe fn pre_alloc(size: usize) -> (*mut u8, u64, usize, usize) {
        let p = alloc(Layout::from_size_align_unchecked(size, 4));
        println!("A block of {} bytes is allocated at {}", size, p as u64);
        (p, p as u64, size, 0)
    }
    unsafe fn pre_dealloc(p: *mut u8, size: usize) -> usize {
        println!("A block of {} bytes at {} is deallocated", size, p as u64);
        dealloc(p, Layout::from_size_align_unchecked(size, 1));
        0
    }
}

unsafe {
    let (p, _, _) = TrackAlloc::alloc(1);
    *p = 10;
    println!("loc {} contains {}", p as u64, *p);
    TrackAlloc::dealloc(p, 1);
}

The following example shows how to use pool!() to define a multiple pools.

// Declare p1 module
pool!(p1);
 
// Declare p2 module
pool!(p2);
 
let _pool1 = p1::Allocator::open_no_root("p1.pool", O_CF).unwrap();
let _pool2 = p2::Allocator::open_no_root("p2.pool", O_CF).unwrap();
 
transaction(|j| {
    // Create a Pbox object in p1
    let b = p1::Pbox::new(10, j);
}).unwrap();
 
transaction(|j| {
    // Create a Prc object in p2
    let p = p2::Prc::new(10, j);
}).unwrap();

Safety

This is the developer’s responsibility to manually drop allocated objects. One way for memory management is to use pointer wrappers that implement Drop trait and deallocate the object on drop. Unsafe methods does not guarantee persistent memory safety.

pmem crate provides Pbox, Prc, and Parc for memory management using RAII. They internally use the unsafe methods.

Required Methods

Prepares allocation without performing it

This function is used internally for low-level atomicity in memory allocation. As an example, please see drop_on_failure.

It returns a 4-tuple: 1. Raw pointer 2. Offset 3. Size 4. Zone index

Examples
unsafe {
    let (ptr, _, _, z) = P::pre_alloc(8);
    *ptr = 10;
    P::perform(z);
}

Prepares deallocation without performing it

This function is used internally for low-level atomicity in memory allocation. As an example, please see drop_on_failure.

It returns the zone in which the deallocation happens.

Examples
unsafe {
    let (ptr, _, _) = P::alloc(8);
    *ptr = 10;
    let zone = P::pre_dealloc(ptr, 8);
    assert_eq!(*ptr, 10);
    P::perform(zone);
    assert_ne!(*ptr, 10);
}

Provided Methods

Returns the name of the pool type

Opens a new pool without any root object. This function is for testing and is not useful in real applications as none of the allocated objects in persistent region is durable. The reason is that they are not reachable from a root object as it doesn’t exists. All objects can live only in the scope of a transaction.

Flags
  • O_C: create a memory pool file if not exists
  • O_F: format the memory pool file
  • O_CNE: create a memory pool file if not exists
  • O_CF: create and format a new memory pool file
  • O_CFNE: create and format a memory pool file only if not exists

See open_flags for more options.

Commits all changes and clears the logs for all threads

This method should be called while dropping the MemPool object to make sure that all uncommitted changes outside transactions, such as reference counters, are persistent.

Returns the zone index corresponding to a given address

Opens a pool and retrieves the root object

The root type should implement RootObj trait in order to create a root object on its absence. This function [creates and] returns an immutable reference to the root object. The pool remains open as long as the root object is in the scope. Like other persistent objects, the root object is immutable and it is modifiable via interior mutability.

Flags
  • O_C: create a memory pool file if not exists
  • O_F: format the memory pool file
  • O_CNE: create a memory pool file if not exists
  • O_CF: create and format a new memory pool file
  • O_CFNE: create and format a memory pool file only if not exists

See open_flags for more options.

Examples
use corundum::default::*;

let root = Allocator::open::<i32>("foo.pool", O_CF).unwrap();

assert_eq!(*root, i32::default());
Single-thread Shared Root Object

Prc<PCell<T>> can be used in order to have a mutable shared root object, as follows.

use corundum::default::*;

type Root = Prc<PCell<i32>>;

let root = Allocator::open::<Root>("foo.pool", O_CF).unwrap();

let data = root.get();

if data == i32::default() {
    println!("Initializing data");
    // This block runs only once to initialize the root object
    transaction(|j| {
        root.set(10, j);
    }).unwrap();
}

assert_eq!(root.get(), 10);
Thread-safe Root Object

If you need a thread-safe root object, you may want to wrap the root object in Parc<PMutex<T>>, as shown in the example below:

use corundum::default::*;
use std::thread;

type Root = Parc<PMutex<i32>>;

let root = Allocator::open::<Root>("foo.pool", O_CF).unwrap();

let mut threads = vec!();

for _ in 0..10 {
    let root = Parc::demote(&root);
    threads.push(thread::spawn(move || {
        transaction(|j| {
            if let Some(root) = root.promote(j) {
                let mut root = root.lock(j);
                *root += 10;
            }
        }).unwrap();
    }));
}

for thread in threads {
    thread.join().unwrap();
}

transaction(|j| {
    let data = root.lock(j);
    assert_eq!(*data % 100, 0);
}).unwrap();
Errors
  • A volatile memory pool (e.g. Heap) doesn’t have a root object.
  • The pool should be open before accessing the root object.

Returns true if the pool is open

Formats the memory pool file

Applies open pool flags

Indicates if the given offset is allocated

Indicates if there the pool is in a good shape

Translates raw pointers to memory offsets

Safety

The raw pointer should be in the valid range

Acquires a reference pointer to the object

Safety

The offset should be in the valid address range

Acquires a mutable reference to the object

Safety

The offset should be in the valid address range

Acquires a reference to the slice

Safety

The offset should be in the valid address range

Acquires a mutable reference to the slice

Safety

The offset should be in the valid address range

Acquires a reference to the object

Acquires a mutable reference pointer to the object

Translates raw pointers to memory offsets

Valid Virtual Address Range

Start of virtual address range

End of virtual address range

Total size of the memory pool

Available space in the pool

Total occupied space

Checks if the reference p belongs to this pool

Checks if addr is in the valid address range if this allocator

addr contains the scalar of a virtual address. If you have a raw fat pointer of type T, you can obtain its virtual address by converting it into a thin pointer and then u64.

Examples
let p = Box::new(1);
println!("Address {:#x} contains value '{}'", p.as_ref() as *const _ as u64, *p);

Allocate memory as described by the given size.

Returns a pointer to newly-allocated memory.

Safety

This function is unsafe because undefined behavior can result if the caller does not ensure that size has non-zero. The allocated block of memory may or may not be initialized. Using alloc may lead to memory leak if the transaction fails after this function successfully returns. To allocate memory in a failure-atomic manner, use pre_alloc, Log::drop_on_failure, and perform functions respectively.

Deallocate the block of memory at the given ptr pointer with the given size.

Safety

This function is unsafe because undefined behavior can result if the caller does not ensure all of the following:

  • ptr must denote a block of memory currently allocated via this allocator,

  • size must be the same size that was used to allocate that block of memory.

Adds a low-level log to update as 64-bit obj to val when perform() is called. As an example, please see Log::set().

Adds a low-level DropOnFailure log to perform inside the allocator. This is internally used to atomically allocate a new objects. Calling perform() drops these logs.

Examples
unsafe {
    // Prepare an allocation. The allocation is not durable yet. In case
    // of a crash, the prepared allocated space is gone. It is fine
    // because it has not been used. The `pre_` and `perform` functions
    // form a low-level atomic section.
    let (obj, off, len, zone) = P::pre_alloc(1);
 
    // Create a low-level DropOnFailure log. This log is going to be used
    // when a crash happens while performing the changes made by the
    // preparation functions. If a crash happens before that, these logs
    // will be discarded.
    P::drop_on_failure(off, len, zone);
     
    // It is fine to work with the prepared raw pointer. All changes in
    // the low-level atomic section are considered as part of the
    // allocation and will be gone in case of a crash, as the allocation
    // will be dropped.
    *obj = 20;
 
    // Transaction ends here. The perform function sets the `operating`
    // flag to show that the prepared changes are being materialized.
    // This flag remains set until the end of materialization. In case
    // of a crash while operating, the recovery procedure first continues
    // the materialization, and then uses the `DropOnFailure` logs to
    // reclaim the allocation. `perform` function realizes the changes
    // made by the `pre_` function on the given memory zone.
    P::perform(zone);
}

In case of not using pre_alloc or pre_dealloc, starts a low-level atomic section on a given zone.

Performs the prepared operations

It materializes the changes made by pre_alloc, pre_dealloc, and pre_realloc. See drop_on_failure for more details.

Discards the prepared operations

Discards the changes made by pre_alloc, pre_dealloc, and pre_realloc. See drop_on_failure for more details.

Behaves like alloc, but also ensures that the contents are set to zero before being returned.

Safety

This function is unsafe for the same reasons that alloc is. However the allocated block of memory is guaranteed to be initialized.

Errors

Returning a null pointer indicates that either memory is exhausted or size does not meet allocator’s size constraints, just as in alloc.

Clients wishing to abort computation in response to an allocation error are encouraged to call the handle_alloc_error function, rather than directly invoking panic! or similar.

Allocates new memory and then places x into it with DropOnFailure log

Allocates a new slice and then places x into it with DropOnAbort log

Allocates new memory and then copies x into it with DropOnFailure log

Allocates new memory and then copies x into it with DropOnFailure log

Allocates new memory and then places x into it without realizing the allocation

Allocates new memory and then places x into it without realizing the allocation

Allocates new memory without copying data

Allocates new memory without copying data

Allocates new memory without copying data and realizing the allocation

Allocates new memory for value x

Creates a DropOnCommit log for the value x

Creates a DropOnCommit log for the value x

Frees the allocation for value x immediately

Drops a journal from memory

Returns a reference to the offset of the first journal

Runs a closure with a mutable reference to a thread->journal HashMap

Recovers from a crash

Commits all changes and clears the logs for one thread

If the transaction is nested, it postpones the commit to the top most transaction.

Safety

This function is for internal use and should not be called elsewhere.

Commits all changes without clearing the logs

If the transaction is nested, it postpones the commit to the top most transaction.

Safety

This function is for internal use and should not be called elsewhere.

Clears the logs

If the transaction is nested, it postpones the clear to the top most transaction.

Safety

This function is for internal use and should not be called elsewhere.

Discards all changes and clears the logs

If the transaction is nested, it propagates the panic up to the top most transaction to make all of them tainted. It returns true if it runs the rollback procedure; otherwise false.

Safety

This function is for internal use and should not be called elsewhere.

Discards all changes without clearing the logs

If the transaction is nested, it propagates the panic upto the top most transaction to make all of them tainted.

Safety

This function is for internal use and should not be called elsewhere.

Executes commands atomically with respect to system crashes

The transaction function takes a closure with one argument of type &Journal<Self>. Before running the closure, it atomically creates a Journal object, if required, and prepares an immutable reference to it. Since there is no other safe way to create a Journal object, it ensures that every function taking an argument of type &Journal<P> is enforced to be invoked from a transaction.

The captured types are bounded to be TxInSafe, unless explicitly asserted otherwise using AssertTxInSafe type wrapper. This guarantees the volatile state consistency, as well as the persistent state.

The returned type should be TxOutSafe. This prevents sending out unreachable persistent objects. The only way out of a transaction for a persistent object is to be reachable by the root object.

Examples
use corundum::default::*;
 
type P = Allocator;
 
let root = P::open::<PCell<i32>>("foo.pool", O_CF).unwrap();
 
let old = root.get();
let new = Allocator::transaction(|j| {
    root.set(root.get() + 1, j);
    root.get()
}).unwrap();
 
assert_eq!(new, old + 1);

Prints memory information

Implementors