Struct DistributedNdarray 

pub struct DistributedNdarray<T, D>
where
    T: Clone + 'static,
    D: Dimension + 'static,
{
    pub config: DistributedConfig,
    /* private fields */
}

A distributed array implementation

Fields

config: DistributedConfig

Configuration for this distributed array

Implementations

impl<T, D> DistributedNdarray<T, D>

where T: Clone + Send + Sync + 'static + Zero + Div<f64, Output = T> + Default, D: Dimension + Clone + Send + Sync + 'static + RemoveAxis,

pub fn new( chunks: Vec<ArrayChunk<T, D>>, shape: Vec<usize>, config: DistributedConfig, ) -> Self

Create a new distributed array from chunks.

pub fn from_array(array: &Array<T, D>, config: DistributedConfig) -> Self

where T: Clone,

Create a distributed array by splitting an existing array.

Every chunk gets a genuinely disjoint slice of the original data, split along axis 0 and recorded with its real starting offset in global_index, so Self::to_array can reconstruct the original array exactly. The communication backend (config.backend) is still simulated (nodeid is a simple round-robin label, not a real network destination) — only the actual data partitioning needed for correct round-tripping is implemented here.

pub fn num_chunks(&self) -> usize

Get the number of chunks in this distributed array.

pub fn shape(&self) -> &[usize]

Get the shape of this distributed array.

pub fn chunks(&self) -> &[ArrayChunk<T, D>]

Get a reference to the chunks in this distributed array.

pub fn to_array(&self) -> CoreResult<Array<T, IxDyn>>

Convert this distributed array back to a regular array.

Reconstructs the array by copying each chunk’s real data back to the axis-0 offset recorded in its global_index (set by Self::from_array) — this only correctly inverts chunks produced by from_array’s own axis-0 splitting; arbitrary externally-constructed chunks (via Self::new) with different placement semantics are not supported.

Errors

Returns CoreError if array conversion fails.

pub fn map<F, R>(&self, f: F) -> Vec<R>

where F: Fn(&ArrayChunk<T, D>) -> R + Send + Sync, R: Send + 'static,

Execute a function on each chunk in parallel.

pub fn map_reduce<F, R, G>(&self, map_fn: F, reducefn: G) -> R

where F: Fn(&ArrayChunk<T, D>) -> R + Send + Sync, G: Fn(R, R) -> R + Send + Sync, R: Send + Clone + 'static,

Reduce the results of mapping a function across all chunks.

Panics

Panics if the chunks collection is empty and no initial value can be reduced.

Trait Implementations

impl<T, D> ArrayProtocol for DistributedNdarray<T, D>

where T: Clone + Send + Sync + 'static + Zero + Div<f64, Output = T> + Default + Add<Output = T> + Mul<Output = T>, D: Dimension + Clone + Send + Sync + 'static + RemoveAxis,

fn array_function( &self, func: &ArrayFunction, _types: &[TypeId], args: &[Box<dyn Any>], kwargs: &HashMap<String, Box<dyn Any>>, ) -> Result<Box<dyn Any>, NotImplemented>

Implementation of the array protocol.

fn as_any(&self) -> &dyn Any

Get the array as Any for downcasting

fn shape(&self) -> &[usize]

Get the shape of the array (default implementation returns empty slice)

fn box_clone(&self) -> Box<dyn ArrayProtocol>

Clone this array protocol object.

fn dtype(&self) -> TypeId

Get the data type of the array (default implementation returns f64)

fn supports_op(&self, _opname: &str) -> bool

Whether this concrete array type provides its own array_function handling for the named operation.

impl<T, D> Debug for DistributedNdarray<T, D>

where T: Clone + Debug + 'static, D: Dimension + Debug + 'static,

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<T, D> DistributedArray for DistributedNdarray<T, D>

where T: Clone + Send + Sync + 'static + Zero + Div<f64, Output = T> + Default + One, D: Dimension + Clone + Send + Sync + 'static + RemoveAxis,

fn gather(&self) -> CoreResult<Box<dyn ArrayProtocol>>

where D: RemoveAxis, T: Default + Clone + One,

Errors

Returns CoreError if gathering fails.

fn scatter(&self, chunks: usize) -> CoreResult<Box<dyn DistributedArray>>

Errors

Returns CoreError if scattering fails.

fn distribution_info(&self) -> HashMap<String, String>

Get information about the distribution of this array.

fn is_distributed(&self) -> bool

Check if this array is distributed.