Struct NpyFile 

pub struct NpyFile<R: Read> { /* private fields */ }

Object for reading an npy file.

This type represents a partially read npy file, where the header has been parsed and we are ready to begin parsing data.

use std::fs::File;
use std::io;

let file = io::BufReader::new(File::open("./test-data/c-order.npy")?);
let npy = npyz::NpyFile::new(file)?;

// Helper methods for inspecting the layout of the data.
assert_eq!(npy.shape(), &[2, 3, 4]);
assert_eq!(npy.strides(), &[12, 4, 1]);
assert_eq!(npy.order(), npyz::Order::C);

// Get the data!
let data: Vec<i64> = npy.into_vec()?;
assert_eq!(data.len(), 24);

Working with large files

For large files, it may be undesirable to read all of the data into a Vec. The NpyFile::data method allows you to iterate over the data instead.

let npy = npyz::NpyFile::new(file)?;

let mut sum = 0;
for x in npy.data::<i64>()? {
    sum += x?;  // items are Result
}
assert_eq!(sum, 84);

Related types

Is a read adaptor too heavy for you? NpyFile is ultimately just a NpyHeader paired with its input stream, so you may consider parsing NpyHeader instead if you need something easier to clone or send.

let mut file = io::BufReader::new(File::open("./test-data/c-order.npy")?);
let header = npyz::NpyHeader::from_reader(&mut file)?;
assert_eq!(header.shape(), &[2, 3, 4]);

// now you can store `header` somewhere
// ...
// and later in your program you can construct an NpyFile to read the data

let npy = npyz::NpyFile::with_header(header, file);
let data: Vec<i64> = npy.into_vec()?;
assert_eq!(data.len(), 24);

Migrating from npy-rs 0.4.0

NpyData is still provided, but it is deprecated in favor of NpyFile. At construction, since &[u8] impls Read, you can still use them as input.

was:
    npyz::NpyData::<i64>::from_bytes(&bytes).to_vec()
now:
    npyz::NpyFile::new(&bytes[..]).into_vec::<i64>()

If you were using the iterator API of NpyData, this is now on NpyReader, which requires us to call NpyFile::data:

was:
    let iter = npyz::NpyData::<i64>::new(&bytes)?;
now:
    let iter = npyz::NpyFile::new(&bytes[..]).data::<i64>.map_err(invalid_data)?;

where the following function has been used to paper over the fact that NpyFile::data has a different Error type:

fn invalid_data<S: ToString>(err: S) -> std::io::Error {
    std::io::Error::new(std::io::ErrorKind::InvalidData, err.to_string())
}

NpyData::is_empty is gone due to possible ambiguity between NpyReader::len and NpyReader::total_len. Use the one that is appropriate for what you are doing.

If you were using NpyData::get… well, honestly, you should first consider whether you could just iterate over the reader instead. But if you were using NpyData::get because you genuinely need random access, then there is NpyReader::read_at.

was:
    // note:  0 <= i < arr.len()
    arr.get(i)
now:
    // note:  0 <= i < reader.total_len()
    reader.read_at(i)?

Implementations

impl<R: Read> NpyFile<R>

pub fn new(reader: R) -> Result<Self>

Read the header of an npy file and construct an NpyFile for reading the data.

pub fn with_header(header: NpyHeader, data_reader: R) -> Self

Construct from a previously parsed header and a reader for the raw data bytes.

pub fn header(&self) -> &NpyHeader

Access the underlying NpyHeader object.

pub fn into_inner(self) -> R

Recover the wrapped io::Read, which now points at the beginning of the raw data bytes.

impl<R: Read> NpyFile<R>

pub fn into_vec<T: Deserialize>(self) -> Result<Vec<T>>

Read all elements into a flat Vec, in the order they are stored as.

This is a convenience wrapper around Self::data and Iterator::collect.

pub fn data<T: Deserialize>(self) -> Result<NpyReader<T, R>, DTypeError>

Produce an NpyReader to begin reading elements, if T can be deserialized from the file’s dtype.

The returned type implements Iterator<Item=io::Result<T>>, and provides additional methods for random access when R: Seek. See NpyReader for more details.

pub fn try_data<T: Deserialize>(self) -> Result<NpyReader<T, R>, Self>

Produce an NpyReader to begin reading elements, if T can be deserialized from the file’s dtype.

This fallible form of the function returns self on error, so that you can try again with a different T.

Methods from Deref<Target = NpyHeader>

pub fn dtype(&self) -> DType

Get the dtype as written in the file.

pub fn shape(&self) -> &[u64]

Get the shape as written in the file.

pub fn strides(&self) -> &[u64]

Get strides for each of the dimensions.

This is the amount by which the item index changes as you move along each dimension. It is a function of both Self::order and Self::shape, provided for your convenience.

pub fn order(&self) -> Order

Get whether the data is in C order or fortran order.

pub fn len(&self) -> u64

Get the total number of elements in the file. (This is the product of Self::shape)

pub fn uses_pickled_array(&self) -> bool

true if an array using this DType is stored using pickle.

This is true if and only if the type contains an 'O' dtype.

Pickled arrays present unique challenges, and you will be unable to extract items from this array using this crate. However, you will be able to extract array metadata such as Self::shape.

Trait Implementations

impl<R: Clone + Read> Clone for NpyFile<R>

fn clone(&self) -> NpyFile<R>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<R: Read> Deref for NpyFile<R>

type Target = NpyHeader

The resulting type after dereferencing.

fn deref(&self) -> &NpyHeader

Dereferences the value.