Struct npyz::NpyFile

pub struct NpyFile<R: Read> { /* private fields */ }

Object for reading an npy file.

This type represents a partially read npy file, where the header has been parsed and we are ready to begin parsing data.

use std::fs::File;
use std::io;

let file = io::BufReader::new(File::open("./test-data/c-order.npy")?);
let npy = npyz::NpyFile::new(file)?;

// Helper methods for inspecting the layout of the data.
assert_eq!(npy.shape(), &[2, 3, 4]);
assert_eq!(npy.strides(), &[12, 4, 1]);
assert_eq!(npy.order(), npyz::Order::C);

// Get the data!
let data: Vec<i64> = npy.into_vec()?;
assert_eq!(data.len(), 24);

Working with large files

For large files, it may be undesirable to read all of the data into a Vec. The NpyFile::data method allows you to iterate over the data instead.

let npy = npyz::NpyFile::new(file)?;

let mut sum = 0;
for x in npy.data::<i64>()? {
    sum += x?;  // items are Result
}
assert_eq!(sum, 84);

Related types

Is a read adaptor too heavy for you? NpyFile is ultimately just a NpyHeader paired with its input stream, so you may consider parsing NpyHeader instead if you need something easier to clone or send.

let mut file = io::BufReader::new(File::open("./test-data/c-order.npy")?);
let header = npyz::NpyHeader::from_reader(&mut file)?;
assert_eq!(header.shape(), &[2, 3, 4]);

// now you can store `header` somewhere
// ...
// and later in your program you can construct an NpyFile to read the data

let npy = npyz::NpyFile::with_header(header, file);
let data: Vec<i64> = npy.into_vec()?;
assert_eq!(data.len(), 24);

Migrating from npy-rs 0.4.0

NpyData is still provided, but it is deprecated in favor of NpyFile. At construction, since &[u8] impls Read, you can still use them as input.

was:
    npyz::NpyData::<i64>::from_bytes(&bytes).to_vec()
now:
    npyz::NpyFile::new(&bytes[..]).into_vec::<i64>()

If you were using the iterator API of NpyData, this is now on NpyReader, which requires us to call NpyFile::data:

was:
    let iter = npyz::NpyData::<i64>::new(&bytes)?;
now:
    let iter = npyz::NpyFile::new(&bytes[..]).data::<i64>.map_err(invalid_data)?;

where the following function has been used to paper over the fact that NpyFile::data has a different Error type:

fn invalid_data<S: ToString>(err: S) -> std::io::Error {
    std::io::Error::new(std::io::ErrorKind::InvalidData, err.to_string())
}

NpyData::is_empty is gone due to possible ambiguity between NpyReader::len and NpyReader::total_len. Use the one that is appropriate for what you are doing.

If you were using NpyData::get… well, honestly, you should first consider whether you could just iterate over the reader instead. But if you were using NpyData::get because you genuinely need random access, then there is NpyReader::read_at.

was:
    // note:  0 <= i < arr.len()
    arr.get(i)
now:
    // note:  0 <= i < reader.total_len()
    reader.read_at(i)?

Implementations

impl<R: Read> NpyFile<R>

pub fn new(reader: R) -> Result<Self>

Read the header of an npy file and construct an NpyFile for reading the data.

pub fn with_header(header: NpyHeader, data_reader: R) -> Self

Construct from a previously parsed header and a reader for the raw data bytes.

pub fn header(&self) -> &NpyHeader

Access the underlying NpyHeader object.

impl<R: Read> NpyFile<R>

pub fn into_vec<T: Deserialize>(self) -> Result<Vec<T>>

Read all elements into a flat Vec, in the order they are stored as.

This is a convenience wrapper around Self::data and Iterator::collect.

pub fn data<T: Deserialize>(self) -> Result<NpyReader<T, R>, DTypeError>

Produce an NpyReader to begin reading elements, if T can be deserialized from the file’s dtype.

The returned type implements Iterator<Item=io::Result<T>>, and provides additional methods for random access when R: Seek. See NpyReader for more details.

pub fn try_data<T: Deserialize>(self) -> Result<NpyReader<T, R>, Self>

Produce an NpyReader to begin reading elements, if T can be deserialized from the file’s dtype.

This fallible form of the function returns self on error, so that you can try again with a different T.

Methods from Deref<Target = NpyHeader>

pub fn dtype(&self) -> DType

Get the dtype as written in the file.

pub fn shape(&self) -> &[u64]

Get the shape as written in the file.

pub fn strides(&self) -> &[u64]

Get strides for each of the dimensions.

This is the amount by which the item index changes as you move along each dimension. It is a function of both Self::order and Self::shape, provided for your convenience.

pub fn order(&self) -> Order

Get whether the data is in C order or fortran order.

pub fn len(&self) -> u64

Get the total number of elements in the file. (This is the product of Self::shape)

Trait Implementations

impl<R: Clone + Read> Clone for NpyFile<R>

fn clone(&self) -> NpyFile<R>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<R: Read> Deref for NpyFile<R>

type Target = NpyHeader

The resulting type after dereferencing.

fn deref(&self) -> &NpyHeader

Dereferences the value.