Struct ndarray_npz::NpzView
source · pub struct NpzView<'a> { /* private fields */ }Expand description
Immutable view for memory-mapped .npz files.
The primary use-case for this is viewing .npy files within a memory-mapped
.npz archive.
Notes
- For types for which not all bit patterns are valid, such as
bool, the implementation iterates over all of the elements when creating the view to ensure they have a valid bit pattern. - The data in the buffer containing an
.npzarchive must be properly aligned for its.npyfile with the maximum alignment requirement for its element type. Typically, this should not be a concern for memory-mapped files (unless an option likeMAP_FIXEDis used), since memory mappings are usually aligned to a page boundary. - The
.npyfiles within the.npzarchive must be properly aligned for their element type. Archives not created by this crate can be aligned with the help of the CLI toolrezipas inrezip in.npz -o out.npz.
Example
This is an example of opening an immutably memory-mapped .npz archive as
an NpzView providing an NpyView for each non-compressed and
non-encrypted .npy file within the archive which can be accessed via
NpyView::view as immutable ArrayView.
This example uses the memmap2 crate
because that appears to be the best-maintained memory-mapping crate at the
moment, but Self::new takes a &mut [u8] instead of a file so that you
can use the memory-mapping crate you’re most comfortable with.
use std::fs::OpenOptions;
use memmap2::MmapOptions;
use ndarray::Ix1;
use ndarray_npz::{NpzView, ViewNpzError};
// Open `.npz` archive of non-compressed and non-encrypted `.npy` files in
// native endian.
#[cfg(target_endian = "little")]
let file = OpenOptions::new()
.read(true)
.open("tests/examples_little_endian_64_byte_aligned.npz")
.unwrap();
#[cfg(target_endian = "big")]
let file = OpenOptions::new()
.read(true)
.open("tests/examples_big_endian_64_byte_aligned.npz")
.unwrap();
// Memory-map `.npz` archive of 64-byte aligned `.npy` files.
let mmap = unsafe { MmapOptions::new().map(&file).unwrap() };
let npz = NpzView::new(&mmap)?;
// List non-compressed and non-encrypted files only.
for npy in npz.names() {
println!("{}", npy);
}
// Get immutable `.npy` views.
let mut x_npy_view = npz.by_name("i64.npy")?;
let mut y_npy_view = npz.by_name("f64.npy")?;
// Optionally verify CRC-32 checksums.
x_npy_view.verify()?;
y_npy_view.verify()?;
// Get and print immutable `ArrayView`s.
let x_array_view = x_npy_view.view::<i64, Ix1>()?;
let y_array_view = y_npy_view.view::<f64, Ix1>()?;
println!("{}", x_array_view);
println!("{}", y_array_view);Implementations§
source§impl<'a> NpzView<'a>
impl<'a> NpzView<'a>
sourcepub fn new(bytes: &'a [u8]) -> Result<Self, ViewNpzError>
pub fn new(bytes: &'a [u8]) -> Result<Self, ViewNpzError>
sourcepub fn is_empty(&self) -> bool
pub fn is_empty(&self) -> bool
Returns true iff the .npz file doesn’t contain any viewable arrays.
Viewable arrays are neither directories, nor compressed, nor encrypted.
sourcepub fn len(&self) -> usize
pub fn len(&self) -> usize
Returns the number of viewable arrays in the .npz file.
Viewable arrays are neither directories, nor compressed, nor encrypted.
sourcepub fn names(&self) -> impl Iterator<Item = &str>
pub fn names(&self) -> impl Iterator<Item = &str>
Returns the names of all of the viewable arrays in the .npz file.
Viewable arrays are neither directories, nor compressed, nor encrypted.
sourcepub fn directory_names(&self) -> impl Iterator<Item = &str>
pub fn directory_names(&self) -> impl Iterator<Item = &str>
Returns the names of all of the directories in the .npz file.
sourcepub fn compressed_names(&self) -> impl Iterator<Item = &str>
pub fn compressed_names(&self) -> impl Iterator<Item = &str>
Returns the names of all of the compressed files in the .npz file.
sourcepub fn encrypted_names(&self) -> impl Iterator<Item = &str>
pub fn encrypted_names(&self) -> impl Iterator<Item = &str>
Returns the names of all of the encrypted files in the .npz file.
sourcepub fn by_name(&self, name: &str) -> Result<NpyView<'a>, ViewNpzError>
pub fn by_name(&self, name: &str) -> Result<NpyView<'a>, ViewNpzError>
Returns an immutable .npy file view by name.
Errors
Viewing an .npy file can fail with ViewNpyError. Trying to view a directory,
compressed file, or encrypted file, fails with ViewNpzError::Directory,
ViewNpzError::CompressedFile, or ViewNpzError::CompressedFile. Fails with
ZipError::FileNotFound if the name is not found.
sourcepub fn by_index(&self, index: usize) -> Result<NpyView<'a>, ViewNpzError>
pub fn by_index(&self, index: usize) -> Result<NpyView<'a>, ViewNpzError>
Returns an immutable .npy file view by index in 0..len().
The index does not necessarily correspond to the index of the zip archive as directories, compressed files, and encrypted files are skipped.
Errors
Viewing an .npy file can fail with ViewNpyError. Fails with ZipError::FileNotFound
if the index is not found.