Struct MmapArray 

pub struct MmapArray<T: Copy + Send + Sync + 'static> { /* private fields */ }

A generic memory-mapped array backed by a file.

MmapArray<T> provides zero-copy access to array data stored in a file, using the operating system’s virtual memory system to page data in and out as needed.

Type Requirements

T must be Copy + Send + Sync + 'static to ensure safe memory-mapped access. The type must also have a fixed, well-defined memory layout (no padding concerns for single elements).

Thread Safety

MmapArray is Send + Sync and can be shared across threads. However, concurrent mutation (with ReadWrite mode) requires external synchronization.

Implementations

impl<T: Copy + Send + Sync + 'static> MmapArray<T>

pub fn from_slice(data: &[T], path: &Path, mode: MmapMode) -> CoreResult<Self>

Create a new MmapArray from a slice of data, writing it to the specified path.

This creates a new file (or truncates an existing one) and writes the header and data to it.

pub fn open(path: &Path, mode: MmapMode) -> CoreResult<Self>

Open an existing MmapArray file.

The file must have been created by MmapArray::from_slice or MmapArray::from_ndarray. The type parameter T must match the type used when the file was created.

pub fn from_ndarray( array: &Array1<T>, path: &Path, mode: MmapMode, ) -> CoreResult<Self>

Create a new MmapArray from an ndarray Array1.

pub fn len(&self) -> usize

Get the number of elements in the array

pub fn is_empty(&self) -> bool

Check if the array is empty

pub fn data_size_bytes(&self) -> usize

Get the size in bytes of the data portion (excluding header)

pub fn file_size_bytes(&self) -> usize

Get the total file size in bytes (header + data)

pub fn mode(&self) -> MmapMode

Get the access mode

pub fn path(&self) -> &Path

Get the file path

pub fn as_slice(&self) -> CoreResult<&[T]>

Get a zero-copy immutable view of the data as a slice.

For COW mode, if the buffer has been materialized (i.e., a write occurred), this returns the COW buffer instead of the mmap data.

pub fn as_slice_mut(&mut self) -> CoreResult<&mut [T]>

Get a zero-copy mutable view of the data as a mutable slice.

Errors

Returns an error if the array is in ReadOnly mode.

For COW mode, the first call to this method materializes the COW buffer by copying the mapped data. Subsequent calls return the COW buffer directly.

pub fn get(&self, index: usize) -> CoreResult<T>

Get a single element by index.

pub fn set(&mut self, index: usize, value: T) -> CoreResult<()>

Set a single element by index.

Errors

Returns an error if the array is in ReadOnly mode or the index is out of bounds.

pub fn to_ndarray(&self) -> CoreResult<Array1<T>>

Convert to an ndarray Array1 by copying the data.

pub fn as_ndarray_view(&self) -> CoreResult<ArrayView1<'_, T>>

Convert to an ndarray ArrayView1 (zero-copy).

The returned view borrows from this MmapArray.

pub fn flush(&self) -> CoreResult<()>

Flush changes to disk (only meaningful for ReadWrite mode).

pub fn flush_async(&self) -> CoreResult<()>

Flush changes asynchronously (non-blocking).

pub fn view(&self, start: usize, len: usize) -> CoreResult<&[T]>

Create a read-only view of a subrange of the array (zero-copy).

pub fn iter(&self) -> CoreResult<Iter<'_, T>>

Iterate over the elements of the array.

pub fn as_ptr(&self) -> CoreResult<*const T>

Get raw pointer to the data (for advanced use).

Safety

The returned pointer is only valid while self is alive and the underlying memory map is not remapped.

pub fn map<U, F>(&self, f: F) -> CoreResult<Vec<U>>

where F: Fn(T) -> U,

Apply a function to each element and return a new Vec.

pub fn chunked_iter( &self, chunk_size: usize, ) -> CoreResult<MmapChunkIter<'_, T>>

Apply a function to chunks of elements. Useful for processing large arrays without loading everything at once.

Trait Implementations

impl<T: Copy + Send + Sync + 'static> Debug for MmapArray<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<T: Copy + Send + Sync + 'static> Send for MmapArray<T>

impl<T: Copy + Send + Sync + 'static> Sync for MmapArray<T>