Skip to main content

Module sparse_io

Module sparse_io 

Source

Re-exports§

pub use meta::Metadata;
pub use prelude::ArcArray;
pub use prelude::Array;
pub use prelude::ArrayBase;
pub use prelude::ArrayRef;
pub use prelude::ArrayView;
pub use prelude::ArrayViewMut;
pub use prelude::CowArray;
pub use prelude::LayoutRef;
pub use prelude::RawArrayView;
pub use prelude::RawArrayViewMut;
pub use prelude::RawRef;
pub use prelude::Axis;
pub use prelude::Dim;
pub use prelude::Dim;
pub use prelude::Dimension;
pub use prelude::ArrayRef0;
pub use prelude::ArrayRef1;
pub use prelude::ArrayRef2;
pub use prelude::ArrayRef3;
pub use prelude::ArrayRef4;
pub use prelude::ArrayRef5;
pub use prelude::ArrayRef6;
pub use prelude::ArrayRefD;
pub use prelude::Array0;
pub use prelude::Array1;
pub use prelude::Array2;
pub use prelude::Array3;
pub use prelude::Array4;
pub use prelude::Array5;
pub use prelude::Array6;
pub use prelude::ArrayD;
pub use prelude::ArrayView0;
pub use prelude::ArrayView1;
pub use prelude::ArrayView2;
pub use prelude::ArrayView3;
pub use prelude::ArrayView4;
pub use prelude::ArrayView5;
pub use prelude::ArrayView6;
pub use prelude::ArrayViewD;
pub use prelude::ArrayViewMut0;
pub use prelude::ArrayViewMut1;
pub use prelude::ArrayViewMut2;
pub use prelude::ArrayViewMut3;
pub use prelude::ArrayViewMut4;
pub use prelude::ArrayViewMut5;
pub use prelude::ArrayViewMut6;
pub use prelude::ArrayViewMutD;
pub use prelude::Ix0;
pub use prelude::Ix0;
pub use prelude::Ix1;
pub use prelude::Ix1;
pub use prelude::Ix2;
pub use prelude::Ix2;
pub use prelude::Ix3;
pub use prelude::Ix3;
pub use prelude::Ix4;
pub use prelude::Ix4;
pub use prelude::Ix5;
pub use prelude::Ix5;
pub use prelude::Ix6;
pub use prelude::Ix6;
pub use prelude::IxDyn;
pub use prelude::IxDyn;
pub use prelude::arr0;
pub use prelude::arr1;
pub use prelude::arr2;
pub use prelude::aview0;
pub use prelude::aview1;
pub use prelude::aview2;
pub use prelude::aview_mut1;
pub use prelude::ShapeBuilder;
pub use prelude::NewAxis;
pub use prelude::AsArray;
pub use prelude::NdFloat;

Modules§

meta
Metadata: short strings stored with a backend that say what its data is, so a reader need not infer it from the file name. Any key may be used; the ones below are shared between the programs that write and read backends, and mean the same everywhere.

Macros§

array
Create an Array with one, two, three, four, five, or six dimensions.
azip
Array zip macro: lock step function application across several arrays and producers.
s
Slice argument constructor.

Structs§

CscMatrix
A CSC representation of a sparse matrix.
CsrMatrix
A CSR representation of a sparse matrix.
PreloadReservation
A preload’s share of the process-wide preload budget, given back when it is dropped. Keep it next to the preloaded arrays and drop it with them, so a later preload in the same process sees the memory as free again.
Tensor
The core struct for manipulating tensors.

Enums§

CsKey
Identifies one of the six 1-D datasets inside a sparse backend. Used by the streaming write API so we don’t have to add six separate abstract methods per dtype × (csc|csr) × (data|indices|indptr).
SparseIoBackend

Constants§

COLUMN_SEP
MAX_COLUMN_NAME_IDX
MAX_ROW_NAME_IDX
ROW_SEP
SLAB_BUDGET_BYTES
Bytes of (u64, u64, f32) triplets a streaming-write slab may hold, and the padded size of one such triplet. One definition, because the two streaming pipelines (the subset trait method and the handlers’ column-selection writer) each carried their own copy and two memory ceilings drift apart.
TRIPLET_BYTES
(u64, u64, f32) padded.

Traits§

SparseIo

Functions§

auto_preload_enabled
Whether loaders preload columns without being asked. On unless LEGUME_AUTO_PRELOAD is 0, false or off; the preload itself still answers to preload_within_budget, so data that does not fit in memory stays on the streaming path.
build_name2index_map
create_sparse_from_dmatrix
Create a sparse matrix io (backend) with dense DMatrix
create_sparse_from_mtx_file
Create a sparse matrix io (backend) with 10x mtx
create_sparse_from_ndarray
Create a sparse matrix io (backend) with dense Array2
create_sparse_from_triplets
Create a sparse backend from a borrowed triplet slice.
create_sparse_from_triplets_owned
Create a sparse backend from an owned triplet Vec.
create_sparse_streaming_empty
Create an empty sparse backend ready for streaming CSC writes.
dmatrix_to_triplets
ndarray_to_triplets
open_sparse_matrix
Open a sparse matrix io (backend)
open_sparse_matrix_by_path
Open a sparse matrix, choosing the backend from the file name (.h5, .zarr, or .zarr.zip) via crate::hdf5_io::resolve_backend_file.
preload_reserved_bytes
Bytes currently held by live PreloadReservations in this process.
preload_within_budget
Whether a preload of nnz entries fits the budget. Unlike reserve_preload, the reservation is held for the life of the process; prefer reserve_preload and keep its guard with the preloaded arrays.
remove_backend_path
Remove a backend at path, whether it is a file (.h5, .zarr.zip) or a directory (.zarr). Nothing happens when the path does not exist.
reserve_preload
Reserve room for a preload of nnz entries, or None when it does not fit the budget.
sparse_io_box_to_arc
take_subset_indices_names
take_subset_indices_names_if_needed

Type Aliases§

DMatrix
A dynamically sized column-major matrix.