Struct DataArray 

pub struct DataArray {
    pub data: Bytes,
    pub dtype: BinaryDataArrayType,
    pub compression: BinaryCompressionType,
    pub name: ArrayType,
    pub params: Option<Box<ParamList>>,
    pub unit: Unit,
    /* private fields */
}

Represents a data array that holds a byte buffer that may be compressed, base64 encoded, or raw little endian bytes, and provides views of those bytes as a small range of supported types.

This type is modeled after the <binaryDataArray> element in mzML.

Note

This type tries to walk a fine line between convenience and performance and as such is easy to misuse. All operations that view the byte buffer as arbitrary data need that data to be decoded and decompressed in order to borrow it. If the byte buffer is not already stored decoded, the operation will copy and decode the buffer in its entirety before performing any other operation, so repeated method calls may incur excessive overhead. If this is happening, please use DataArray::decode_and_store to store the decoded representation explicitly.

Normally, SpectrumSource-implementing file readers will eagerly decode all arrays as soon as they are ready. If they are operating in lazy mode, the buffers will need to be decoded explicitly, again using DataArray::decode_and_store or operations should make as much use of the copied arrays as possible instead.

Fields

data: Bytes

The raw data of the array, stored in a variety of states controlled by [Self::Compression], as native bytes

dtype: BinaryDataArrayType

The kind of data stored in the array, as in bit-width primitive types

compression: BinaryCompressionType

How Self::data is encoded and compressed.

name: ArrayType

The what the data stored in the array was measuring, e.g. an m/z array

params: Option<Box<ParamList>>

Additional metadata parameters

unit: Unit

The Unit for the data measurements

Implementations

impl<'transient, 'lifespan: 'transient> DataArray

pub fn new() -> DataArray

An alias of Default::default, no configuration is performed whatsoever.

pub fn from_name(name: &ArrayType) -> DataArray

Create a new DataArray for a given name in its preferred data type returned by ArrayType::preferred_dtype and unit returned by ArrayType::preferred_unit

pub fn from_name_and_type( name: &ArrayType, dtype: BinaryDataArrayType, ) -> DataArray

Create a new DataArray with a given name and dtype, using name’s preferred units. Does not allocate any additional storage.

pub fn from_name_type_size( name: &ArrayType, dtype: BinaryDataArrayType, size: usize, ) -> DataArray

Create a new DataArray with a given name and dtype, using name’s preferred units. Pre-allocates but does not fill size storage capacity in bytes (not elements of dtype).

pub fn slice( &self, start: usize, end: usize, ) -> Result<DataArray, ArrayRetrievalError>

Extract a subrange of the encoded byte buffer as a new DataArray.

start and end are byte offsets into the decoded buffer, not element indices, so they must be aligned to a multiple of DataArray::dtype’s size.

Examples

use mzdata_bindata::{ArrayType, BinaryDataArrayType, ByteArrayView, DataArray};

let mut array = DataArray::from_name_and_type(&ArrayType::MZArray, BinaryDataArrayType::Float64);
array.extend(&[1.0f64, 2.0, 3.0, 4.0]).unwrap();

// Each `f64` is 8 bytes wide, so the 2nd and 3rd elements occupy bytes [8, 24).
let middle = array.slice(8, 24).unwrap();
assert_eq!(&*middle.to_f64().unwrap(), &[2.0f64, 3.0f64][..]);

pub fn slice_buffer( &self, start: usize, end: usize, ) -> Result<Cow<'_, [u8]>, ArrayRetrievalError>

Retrieve a slice of raw bytes with no concern for value type alignment.

start and end are byte offsets into the decoded buffer

pub fn wrap( name: &ArrayType, dtype: BinaryDataArrayType, data: Bytes, ) -> DataArray

This method assumes the data are already in native byte order.

This is not endian-safe yet, but is valid for in-memory operations.

Examples

use mzdata_bindata::{ArrayType, BinaryDataArrayType, ByteArrayView, DataArray};

let values: [f64; 3] = [100.5, 200.25, 300.0];
let bytes: Vec<u8> = values.iter().flat_map(|v| v.to_le_bytes()).collect();
let array = DataArray::wrap(&ArrayType::MZArray, BinaryDataArrayType::Float64, bytes);

let decoded = array.to_f64().unwrap();
assert_eq!(&*decoded, &values[..]);

pub fn update_buffer<T: Pod + ToBytes>( &mut self, data_buffer: &[T], ) -> Result<usize, ArrayRetrievalError>

Directly set the data buffer from a slice of a Pod type, copying it.

This will return an error if std::mem::size_of::<T>() is not equal to the size of DataArray::dtype.

pub fn push<T: Pod>(&mut self, value: T) -> Result<(), ArrayRetrievalError>

Add the value of a Pod implementing type T to the array, converting it to native byte representation.

This will return an error if std::mem::size_of::<T>() is not equal to the size of DataArray::dtype.

Examples

use mzdata_bindata::{ArrayType, BinaryDataArrayType, ByteArrayView, DataArray};

let mut array = DataArray::from_name_and_type(&ArrayType::MZArray, BinaryDataArrayType::Float64);
array.push(500.5f64).unwrap();
array.push(501.0f64).unwrap();

assert_eq!(&*array.to_f64().unwrap(), &[500.5f64, 501.0f64][..]);

pub fn extend<T: Pod>( &mut self, values: &[T], ) -> Result<(), ArrayRetrievalError>

Add the values from a slice of a Pod implementing type T to the array, converting the slice a slice of T in its native byte representation.

This will return an error if std::mem::size_of::<T>() is not equal to the size of DataArray::dtype.

Examples

use mzdata_bindata::{ArrayType, BinaryDataArrayType, ByteArrayView, DataArray};

let mut array = DataArray::from_name_and_type(&ArrayType::IntensityArray, BinaryDataArrayType::Float32);
array.extend(&[10.0f32, 20.0, 30.0]).unwrap();

assert_eq!(&*array.to_f32().unwrap(), &[10.0f32, 20.0f32, 30.0f32][..]);

pub fn extend_raw(&mut self, values: &[u8]) -> Result<(), ArrayRetrievalError>

Add the values from a buffer of bytes to the array directly.

This will return an error if values.len() is not a multiple of the size of DataArray::dtype.

pub fn extend_iter<T: Pod>( &mut self, iter: impl IntoIterator<Item = T>, ) -> Result<(), ArrayRetrievalError>

Add the values from an iterator over Pod type T.

This is more efficient than repeated calls to DataArray::push or collecting to a Vec and then using DataArray::extend, but equivalent to either.

This will return an error if std::mem::size_of::<T>() is not equal to the size of DataArray::dtype.

pub fn encode_bytestring(&self, compression: BinaryCompressionType) -> Bytes

Encode the data buffer to a byte array using the requested BinaryCompressionType to compress it and then base64 encode it.

This method takes the current byte buffer in Self::data and does the following: 1. If Self::compression is compression, just return a copy of the data as-is, it is pre-coded properly. 2. If Self::compression is not BinaryCompressionType::Decoded, copy and decode Self::data, otherwise borrow the raw byte data. 3. Apply compression to the decoded raw bytes, performing one or more transformations and then base64 encode the resulting compressed byte array. 4. Return the base64 encoded, compressed byte array

Panics

This method will panic if compression fails for any reason, or if an unsupported compression method is requested. Some compression methods are only available if appropriate features are enabled.

pub fn decode_and_store( &mut self, ) -> Result<BinaryCompressionType, ArrayRetrievalError>

Decode the compressed data, if needed, and store that buffer in self.data. After decoding self.compression will always be BinaryCompressionType::Decoded.

The return value is the content of self.compression after decoding.

This may fail if the decoding fails for any reason.

Examples

use mzdata_bindata::{ArrayType, BinaryCompressionType, BinaryDataArrayType, ByteArrayView, DataArray};

let values: [f64; 4] = [1.0, 2.0, 3.0, 4.0];
let mut array = DataArray::from_name_and_type(&ArrayType::MZArray, BinaryDataArrayType::Float64);
array.extend(&values).unwrap();

array.store_compressed(BinaryCompressionType::Zlib).unwrap();
assert_eq!(array.compression, BinaryCompressionType::Zlib);

array.decode_and_store().unwrap();
assert_eq!(array.compression, BinaryCompressionType::Decoded);
assert_eq!(&*array.to_f64().unwrap(), &values[..]);

pub fn decode( &'lifespan self, ) -> Result<Cow<'lifespan, [u8]>, ArrayRetrievalError>

Decompress and base64-decode encoded bytes, and return the data.

If the data were already decoded, the existing bytes are returned. Otherwise one or more buffers may be allocated to hold the decompressed and decoded bytes.

Panics

This method panics if Self::compression fails for any reason or is not supported. Several compression methods require certain features be enabled.

pub fn decode_mut( &'transient mut self, ) -> Result<&'transient mut Bytes, ArrayRetrievalError>

Get a mutable view of the raw bytes.

If the data are already decoded, this just borrows Self::data, otherwise the Self::decode is called Self::data is updated to the decoded copy first, then borrowed.

Note that care MUST be taken here to avoid writing invalid values when working on the raw bytes directly. Prefer to update the native values indirectly via ByteArrayViewMut::coerce_mut.

pub fn clear(&mut self)

Erase all the data in the array, calling Vec::clear on Self::data, and erasing all [Param] values. All other properties remain unchanged.

pub fn store_compressed( &mut self, compression: BinaryCompressionType, ) -> Result<(), ArrayRetrievalError>

The reverse of DataArray::decode_and_store, this method compresses self.data to the desired compression method and stores that buffer as self.data.

pub fn store_as( &mut self, dtype: BinaryDataArrayType, ) -> Result<usize, ArrayRetrievalError>

Recode the stored data as the requested binary data type.

This is a no-op if the Self::dtype matches the dtype argument.

Note

When down-casting to a lower precision type, e.g. f64 to f32, data may be lost. Take care to do this only when necessary or when the loss of precision is acceptable.

pub const fn is_ion_mobility(&self) -> bool

Test if the the array describes an ion mobility quantity.

See also

ArrayType::is_ion_mobility

pub fn raw_len(&self) -> usize

The size of the raw byte buffer

pub fn data_processing_reference(&self) -> Option<&str>

Get the identifier referencing a DataProcessing

pub fn set_data_processing_reference( &mut self, data_processing_reference: Option<Box<str>>, )

Set the identifier referencing a DataProcessing

Trait Implementations

impl<'transient, 'lifespan: 'transient> ByteArrayView<'transient, 'lifespan> for DataArray

fn view(&'lifespan self) -> Result<Cow<'lifespan, [u8]>, ArrayRetrievalError>

The core method that exposes the decoded byte array that the type must provide.

fn dtype(&self) -> BinaryDataArrayType

The real data type encoded in bytes

fn data_len(&'lifespan self) -> Result<usize, ArrayRetrievalError>

The size of encoded array in terms of # of elements of the BinaryDataArrayType given by ByteArrayView::dtype

fn unit(&self) -> Unit

The unit of measurement each data point is in

fn data_processing_reference(&self) -> Option<&str>

Get the identifier referencing a DataProcessing

fn name(&self) -> &ArrayType

The kind of array this is

fn coerce_from<T: Pod>( buffer: Cow<'transient, [u8]>, ) -> Result<Cow<'transient, [T]>, ArrayRetrievalError>

This is a helper method for the various to_<type> methods like Self::to_f64. It should not need to be called directly. This method relies on bytemuck::try_cast_slice to do the heavy lifting.

fn coerce<T: Pod>( &'lifespan self, ) -> Result<Cow<'transient, [T]>, ArrayRetrievalError>

This is a helper method for the various to_<type> methods like Self::to_f64. It should not need to be called directly. This calls into Self::coerce_from internally after decoding.

fn convert<S: Num + Clone + AsPrimitive<D> + Pod, D: Num + Clone + Copy + 'static>( &'lifespan self, ) -> Result<Cow<'transient, [D]>, ArrayRetrievalError>

Decode the array, then copy it to a new array, converting each element from type D to to type S

fn to_f32( &'lifespan self, ) -> Result<Cow<'transient, [f32]>, ArrayRetrievalError>

Get a view of the data as f32. If the data are stored that way already, no copy is required.

fn to_f64( &'lifespan self, ) -> Result<Cow<'transient, [f64]>, ArrayRetrievalError>

Get a view of the data as f64. If the data are stored that way already, no copy is required.

fn to_i32( &'lifespan self, ) -> Result<Cow<'transient, [i32]>, ArrayRetrievalError>

Get a view of the data as i32. If the data are stored that way already, no copy is required.

fn to_i64( &'lifespan self, ) -> Result<Cow<'transient, [i64]>, ArrayRetrievalError>

Get a view of the data as i64. If the data are stored that way already, no copy is required.

fn iter_type<T: Pod>( &'lifespan self, ) -> Result<DataSliceIter<'lifespan, T>, ArrayRetrievalError>

fn iter_u8( &'lifespan self, ) -> Result<DataSliceIter<'lifespan, u8>, ArrayRetrievalError>

fn iter_f32( &'lifespan self, ) -> Result<DataSliceIter<'lifespan, f32>, ArrayRetrievalError>

fn iter_f64( &'lifespan self, ) -> Result<DataSliceIter<'lifespan, f64>, ArrayRetrievalError>

fn iter_i32( &'lifespan self, ) -> Result<DataSliceIter<'lifespan, i32>, ArrayRetrievalError>

fn iter_i64( &'lifespan self, ) -> Result<DataSliceIter<'lifespan, i64>, ArrayRetrievalError>

impl<'transient, 'lifespan: 'transient> ByteArrayViewMut<'transient, 'lifespan> for DataArray

fn view_mut( &'transient mut self, ) -> Result<&'transient mut Bytes, ArrayRetrievalError>

Get a mutable view of the bytes backing this data array.

fn unit_mut(&mut self) -> &mut Unit

Specify the unit of the data array

fn set_data_processing_reference( &mut self, data_processing_reference: Option<Box<str>>, )

Set the identifier referencing a DataProcessing

fn coerce_from_mut<T: Clone + Sized>( buffer: &mut [u8], ) -> Result<&'transient mut [T], ArrayRetrievalError>

fn coerce_mut<T: Clone + Sized>( &'lifespan mut self, ) -> Result<&'transient mut [T], ArrayRetrievalError>

impl Clone for DataArray

fn clone(&self) -> DataArray

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DataArray

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for DataArray

fn default() -> DataArray

Returns the “default value” for a type.

impl ParamDescribed for DataArray

fn params(&self) -> &[Param]

Obtain an immutable slice over the encapsulated Param list

fn params_mut(&mut self) -> &mut ParamList

Obtain an mutable slice over the encapsulated Param list

fn add_param(&mut self, param: Param)

Add a new Param to the entity

fn extend_params(&mut self, it: impl IntoIterator<Item = Param>)

Add all parameters from an iterator of Param to the entity

fn remove_param(&mut self, index: usize) -> Param

Remove the ith Param from the entity.

fn get_param_by_name(&self, name: &str) -> Option<&Param>

Find the first Param whose name matches name

fn get_param_by_curie(&self, curie: &CURIE) -> Option<&Param>

Find the first Param whose CURIE matches curie

fn get_param_by_accession(&self, accession: &str) -> Option<&Param>

Find the first Param whose Param::accession matches accession

fn iter_params(&self) -> Iter<'_, Param>

Iterate over the encapsulated parameter list

fn iter_params_mut(&mut self) -> IterMut<'_, Param>

Iterate mutably over the encapsulated parameter list