Struct ArrayFile 

pub struct ArrayFile { /* private fields */ }

The top-level file handle.

Layers are stacked oldest → newest in deltas. Uncommitted writes accumulate in a disk-backed mutable delta (pending) and are flushed by flush. The mutable delta is created lazily on the first mutation after open/flush.

Implementations

impl ArrayFile

pub async fn create<C: CompressionCodec + 'static>( store: Arc<dyn ObjectStore>, path: Path, config: FileConfig<C>, ) -> Result<Self>

Creates a new empty file at path within store.

path is the base file object and should end in .af; sidecars ({stem}.N.af) and the stats file ({stem}.stats) are written alongside it in the same prefix. Fails if an object already exists at path only insofar as the backend allows overwriting — the base is (re)written empty.

pub async fn open<C: CompressionCodec + 'static>( store: Arc<dyn ObjectStore>, path: Path, config: FileConfig<C>, ) -> Result<Self>

Opens an existing file from store, discovering the base and all sidecar layers under the same stem.

path must end in .af. Aggregate statistics are loaded from {stem}.stats if present; a missing or unreadable stats file is not an error (see array_stats).

pub async fn create_memory<C: CompressionCodec + 'static>( config: FileConfig<C>, ) -> Result<Self>

Creates a new empty in-memory file.

Backed by object_store’s in-memory backend, so it behaves exactly like an on-disk file (commit pending writes with flush) but keeps everything in process. Useful for tests and ephemeral pipelines.

impl ArrayFile

pub fn get_array(&self, name: &str) -> Result<&ArrayMeta>

Returns a reference to the merged array metadata for name, searching from the newest layer towards the oldest.

pub fn list_arrays(&self) -> Vec<MergedArrayMeta>

Returns all non-deleted visible arrays (newest-wins merge).

pub fn array_stats(&self, name: &str) -> Option<&ArrayStats>

Returns aggregate statistics for name, or None if no stats exist yet.

pub fn num_layers(&self) -> usize

Number of committed (immutable) delta layers.

pub fn get_attribute( &self, name: &str, key: &str, ) -> Result<Option<&AttributeValue>>

Returns the value of attribute key on array name, or None if the array has no such attribute. Errors if the array does not exist.

pub fn set_attribute( &mut self, name: &str, key: &str, value: AttributeValue, ) -> Result<()>

Sets attribute key on array name to value, inserting or replacing any existing entry. The change lands in the pending layer and is persisted on the next flush. Errors if the array does not exist.

impl ArrayFile

pub fn define_array<T: ArrayElement>( &mut self, name: impl Into<String>, dimension_names: Vec<String>, shape: Vec<usize>, chunk_shape: Option<Vec<usize>>, fill_value: Option<FillValue>, ) -> Result<()>

Defines a new array in the pending layer.

shape is the full array shape; chunk_shape tiles it into a grid of independently stored chunks, or None to store the whole array as a single chunk. If dimension_names does not have one entry per dimension it is replaced with dim0, dim1, … . fill_value is returned for elements that are never written.

Errors with Error::ArrayAlreadyExists if an array of this name is already visible. The definition is persisted on the next flush.

pub fn delete(&mut self, name: &str) -> Result<()>

Logically deletes an array by writing a tombstone to the pending layer.

The array is excluded from list_arrays and all reads immediately, but its bytes remain on disk until compact reclaims them.

impl ArrayFile

pub async fn write_array<T: ArrayElement>( &mut self, name: &str, start: Vec<usize>, data: ArrayView<'_, T, IxDyn>, ) -> Result<()>

Writes data into array name with its origin at coordinate start.

The region may span multiple chunks and need not be chunk-aligned; partially covered chunks are read-modify-written automatically. T must match the array’s declared dtype, otherwise Error::DTypeMismatch is returned. Writes accumulate in the pending layer until flush.

pub async fn read_array<T: ArrayElement>( &self, name: &str, start: Vec<usize>, shape: Vec<usize>, ) -> Result<ArcArray<T, IxDyn>>

Reads the sub-region of array name starting at start with the given shape.

Pass vec![], vec![] to read the whole array. Chunks that were never written are materialized from the array’s fill value. T must match the array’s declared dtype, otherwise Error::DTypeMismatch is returned.

impl ArrayFile

pub async fn flush(&mut self) -> Result<()>

Commits pending writes to a new sidecar layer and refreshes the {stem}.stats file.

A no-op if there are no pending changes.

impl ArrayFile

pub async fn compact(&mut self) -> Result<()>

Merges all committed layers into a single new base file, deleting the sidecars and reclaiming space held by overwritten and tombstoned chunks.

After a successful compaction num_layers returns 1. Recomputes and rewrites the {stem}.stats file.