Struct flatk::ClumpedOffsets

pub struct ClumpedOffsets<O = Vec<usize>> {
    pub chunk_offsets: Offsets<O>,
    pub offsets: Offsets<O>,
}

A collection of clumped offsets into another collection.

A version of Offsets that combines consecutive equidistant offsets.

Example

To see the correspondence between Offsets and ClumpedOffsets, consider the following example

    use flatk::{Offsets, ClumpedOffsets};

    // with `Offsets` we might have offsets like
    let off = Offsets::new(vec![0, 3, 6, 9, 12, 16, 20, 24, 27, 30, 33, 36, 39]);
    // which can be represented in `ClumpedOffsets` as
    let clumped_off = ClumpedOffsets::new(vec![0, 4, 7, 12], vec![0, 12, 24, 39]);
    // Note that ClumpedOffsets would be more compact if the triplets could be combined,
    // which would require reorganizing the data indexed by the offsets.

    assert_eq!(ClumpedOffsets::from(off), clumped_off);

ClumpedOffsets can be a lot more memory efficient when a chunked collection is mostly uniformly chunked. However, in the worst case, when each consecutive stride is different, this representation can be three times the size of standard Offsets.

Fields

chunk_offsets: Offsets<O>

offsets: Offsets<O>

Implementations

impl<O: AsRef<[usize]>> ClumpedOffsets<O>

pub fn double_ended_sizes(&self) -> DEUnclumpedSizes<'_>

An iterator over effective (unclumped) sizes that implements DoubleEndedIterator.

Notes

This iterator is used to implement the parallel sizes iterator to enable parallel iteration of Chunked types with clumped offsets. However, this iterator can also be used as is for efficient backwards iteration.

pub fn par_offset_values_and_sizes(
    &self
) -> ParUnclumpedOffsetValuesAndSizes<'_>

Parallel version of offsets_and_sizes.

impl<O: AsRef<[usize]>> ClumpedOffsets<O>

pub fn first_chunk_offset_value(&self) -> usize

pub fn last_chunk_offset_value(&self) -> usize

pub fn num_clump_offsets(&self) -> usize

Returns the number of clump offsets represented by ClumpedOffsets.

This is typically significantly smaller than self.num_offsets().

pub fn num_clumps(&self) -> usize

Returns the number of clumps represented by ClumpedOffsets.

pub fn clump_stride(&self, index: usize) -> usize

Compute the stride of the clump at the given index.

Panics

This function panics if index is greater than or equal to self.num_clumps().

impl<O: AsRef<[usize]>> ClumpedOffsets<O>

pub fn sizes(&self) -> UnclumpedSizes<'_>

An iterator over effective (unclumped) sizes producing an increment for advancing the data pointer.

This helps for implementing iterators over Chunked types.

Example

    use flatk::{Offsets, ClumpedOffsets};

    let off = Offsets::new(vec![0, 3, 6, 9, 12, 16, 20, 24, 27, 30, 33, 36, 39]);
    let clumped = ClumpedOffsets::from(off);
    let mut clumped_iter = clumped.sizes();
    for _ in 0..4 {
        assert_eq!(clumped_iter.next(), Some(3));
    }
    for _ in 0..3 {
        assert_eq!(clumped_iter.next(), Some(4));
    }
    for _ in 0..5 {
        assert_eq!(clumped_iter.next(), Some(3));
    }
    assert_eq!(clumped_iter.next(), None);

pub fn iter<'a>(&'a self) -> impl Iterator<Item = usize> + 'a

An iterator over unclumped offsets.

This is equivalent to iterating over Offsets after conversion, but it doesn't require any additional allocations.

pub fn values(&self) -> UnclumpedOffsetValues<'_>

An iterator over unclumped offsets.

This is equivalent to iterating over Offsets after conversion, but it doesn't require any additional allocations.

impl<O: AsRef<[usize]>> ClumpedOffsets<O>

pub fn new(chunk_offsets: O, offsets: O) -> Self

Construct new ClumpedOffsets from a given valid set of clumped offsets represented as an array of usizes.

It is possible to create an invalid ClumpedOffsets struct using this constructor, however it is safe.

Panics

This function will panic if either chunk_offsets or offsets is empty.

pub unsafe fn from_raw(chunk_offsets: O, offsets: O) -> Self

An unchecked version of the new constructor.

Safety

Calling this function with empty chunk_offsets or offsets may cause undefined behaviour in safe APIs.

Trait Implementations

impl Clear for ClumpedOffsets

fn clear(&mut self)

Remove all elements from the current set without necessarily deallocating the space previously used.

impl<O: Clone> Clone for ClumpedOffsets<O>

fn clone(&self) -> ClumpedOffsets<O>

Returns a copy of the value.

pub fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<O: Copy> Copy for ClumpedOffsets<O>

impl<O: Debug> Debug for ClumpedOffsets<O>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ClumpedOffsets<Vec<usize>>

A default set of offsets must allocate.

fn default() -> Self

Returns the "default value" for a type.

impl<O: Dummy> Dummy for ClumpedOffsets<O>

A dummy set of offsets will not allocate, but the resulting type does not correspond to valid offsets.

unsafe fn dummy() -> Self

This function creates an invalid ClumpedOffsets, but it does not allocate.

impl<O: AsRef<[usize]> + Set> From<ClumpedOffsets<O>> for Offsets

fn from(clumped_offsets: ClumpedOffsets<O>) -> Self

Convert ClumpedOffsets into owned Offsets.

This function causes allocations.

impl<O: AsRef<[usize]> + Set> From<Offsets<O>> for ClumpedOffsets

fn from(offsets: Offsets<O>) -> Self

Convert Offsets to owned ClumpedOffsets.

This function causes allocations.

Example

use flatk::{Offsets, ClumpedOffsets};
let offsets = Offsets::new(vec![0, 3, 6, 9, 12, 16, 20, 24, 27, 30, 33, 36, 39]);
let clumped_offsets = ClumpedOffsets::new(vec![0, 4, 7, 12], vec![0, 12, 24, 39]);
assert_eq!(ClumpedOffsets::from(offsets), clumped_offsets);

impl FromIterator<usize> for ClumpedOffsets

fn from_iter<T>(iter: T) -> Self where
    T: IntoIterator<Item = usize>, 

Creates a value from an iterator.

impl<'a, O> GetIndex<'a, ClumpedOffsets<O>> for usize where
    ClumpedOffsets<O>: GetOffset, 

type Output = usize

fn get(self, clumped_offsets: &ClumpedOffsets<O>) -> Option<Self::Output>

impl<'a, O> GetIndex<'a, ClumpedOffsets<O>> for &usize where
    ClumpedOffsets<O>: GetOffset, 

type Output = usize

fn get(self, clumped_offsets: &ClumpedOffsets<O>) -> Option<Self::Output>

impl<O: AsRef<[usize]>> GetOffset for ClumpedOffsets<O>

unsafe fn offset_value_unchecked(&self, index: usize) -> usize

A version of offset_value without bounds checking.

Safety

It is assumed that index is strictly less than self.num_offsets().

fn num_offsets(&self) -> usize

Get the total number of offsets.

This is one more than the number of chunks represented.

fn chunk_len(&self, chunk_index: usize) -> usize

Get the length of the chunk at the given index.

unsafe fn chunk_len_unchecked(&self, chunk_index: usize) -> usize

Get the length of the chunk at the given index without bounds checking.

fn offset_value(&self, index: usize) -> usize

Return the raw value corresponding to the offset at the given index.

fn offset(&self, index: usize) -> usize

Returns the offset at the given index with respect to (minus) the first offset. This function returns the total length of data if index is equal to self.len().

unsafe fn offset_unchecked(&self, index: usize) -> usize

A version of offset without bounds checking.

fn last_offset(&self) -> usize

Get the last offset.

fn first_offset(&self) -> usize

Get the first offset.

fn last_offset_value(&self) -> usize

Get the raw value corresponding to the last offset.

fn first_offset_value(&self) -> usize

Get the raw value corresponding to the first offset.

impl<O> IndexRange for ClumpedOffsets<O> where
    Self: GetOffset, 

unsafe fn index_range_unchecked(&self, range: Range<usize>) -> Range<usize>

fn index_range(&self, range: Range<usize>) -> Option<Range<usize>>

Return the [begin..end) bound of the chunk at the given index.

impl<'a> IntoOffsetValuesAndSizes for ClumpedOffsets<&'a [usize]>

type Iter = UnclumpedOffsetValuesAndSizes<'a>

fn into_offset_values_and_sizes(self) -> UnclumpedOffsetValuesAndSizes<'a>

impl<O: IntoOwned> IntoOwned for ClumpedOffsets<O>

type Owned = ClumpedOffsets<O::Owned>

fn into_owned(self) -> Self::Owned

fn clone_into(self, target: &mut Self::Owned)

impl<'a> IntoParOffsetValuesAndSizes for ClumpedOffsets<&'a [usize]>

type ParIter = ParUnclumpedOffsetValuesAndSizes<'a>

fn into_par_offset_values_and_sizes(self) -> Self::ParIter

Returns a parallel iterator over chunk offsets and sizes represented by the stored Offsets.

impl<'a> IntoSizes for ClumpedOffsets<&'a [usize]>

type Iter = UnclumpedSizes<'a>

fn into_sizes(self) -> UnclumpedSizes<'a>

Returns an iterator over chunk sizes represented by the stored ClumpedOffsets.

impl<'a> IntoValues for ClumpedOffsets<&'a [usize]>

type Iter = UnclumpedOffsetValues<'a>

fn into_values(self) -> UnclumpedOffsetValues<'a>

Returns an iterator over offset values represented by the stored Offsets.

impl<O> IsolateIndex<ClumpedOffsets<O>> for usize where
    ClumpedOffsets<O>: GetOffset, 

type Output = usize

unsafe fn isolate_unchecked(
    self,
    clumped_offsets: ClumpedOffsets<O>
) -> Self::Output

fn try_isolate(self, clumped_offsets: ClumpedOffsets<O>) -> Option<Self::Output>

impl<O: PartialEq> PartialEq<ClumpedOffsets<O>> for ClumpedOffsets<O>

fn eq(&self, other: &ClumpedOffsets<O>) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &ClumpedOffsets<O>) -> bool

This method tests for !=.

impl<O: RemovePrefix + Set> RemovePrefix for ClumpedOffsets<O>

fn remove_prefix(&mut self, n: usize)

Remove n elements from the beginning.

impl<O: Reserve> Reserve for ClumpedOffsets<O>

fn reserve_with_storage(&mut self, n: usize, storage_n: usize)

fn reserve(&mut self, n: usize)

impl<O: AsRef<[usize]>> Set for ClumpedOffsets<O>

type Elem = usize

Owned element of the set.

type Atom = usize

The most basic element contained by this collection. If this collection contains other collections, this type should be different than Elem.

fn len(&self) -> usize

fn is_empty(&self) -> bool

impl<'a> SplitOffsetsAt for ClumpedOffsets<&'a [usize]>

fn split_offsets_with_intersection_at(
    self,
    mid_off: usize
) -> (ClumpedOffsets<&'a [usize]>, ClumpedOffsets<&'a [usize]>, usize)

Same as split_offsets_at, but in addition, return the offset of the middle element (intersection): this is the value offsets[mid] - offsets[0].

Panics

Calling this function with an empty slice or with mid greater than or equal to its length will cause a panic.

This function will also panic when trying to split a clump since the offsets cannot be modified or created.

fn split_offsets_at(
    self,
    mid: usize
) -> (ClumpedOffsets<&'a [usize]>, ClumpedOffsets<&'a [usize]>)

Splits clumped offsets at the given index into two such that each resulting ClumpedOffsets is valid. This means that the element at index mid is shared between the two outputs.

Panics

Calling this function with an empty slice or with mid greater than or equal to its length will cause a panic.

This function will also panic when trying to split a clump since the offsets cannot be modified or created.

impl<O> StructuralPartialEq for ClumpedOffsets<O>

impl<O: Truncate> Truncate for ClumpedOffsets<O>

fn truncate(&mut self, new_len: usize)

impl<'a, O: AsRef<[usize]>> View<'a> for ClumpedOffsets<O>

type Type = ClumpedOffsets<&'a [usize]>

fn view(&'a self) -> Self::Type

impl<'a, O: AsMut<[usize]>> ViewMut<'a> for ClumpedOffsets<O>

type Type = ClumpedOffsets<&'a mut [usize]>

fn view_mut(&'a mut self) -> Self::Type

impl<O: Viewed> Viewed for ClumpedOffsets<O>