Struct ClumpedOffsets 

pub struct ClumpedOffsets<O = Vec<usize>> {
    pub chunk_offsets: Offsets<O>,
    pub offsets: Offsets<O>,
}

A collection of clumped offsets into another collection.

A version of Offsets that combines consecutive equidistant offsets.

Example

To see the correspondence between Offsets and ClumpedOffsets, consider the following example

    use flatk::{Offsets, ClumpedOffsets};

    // with `Offsets` we might have offsets like
    let off = Offsets::new(vec![0, 3, 6, 9, 12, 16, 20, 24, 27, 30, 33, 36, 39]);
    // which can be represented in `ClumpedOffsets` as
    let clumped_off = ClumpedOffsets::new(vec![0, 4, 7, 12], vec![0, 12, 24, 39]);
    // Note that ClumpedOffsets would be more compact if the triplets could be combined,
    // which would require reorganizing the data indexed by the offsets.

    assert_eq!(ClumpedOffsets::from(off), clumped_off);

ClumpedOffsets can be a lot more memory efficient when a chunked collection is mostly uniformly chunked. However, in the worst case, when each consecutive stride is different, this representation can be three times the size of standard Offsets.

Fields

chunk_offsets: Offsets<O>

offsets: Offsets<O>

Implementations

impl<O: AsRef<[usize]>> ClumpedOffsets<O>

pub fn double_ended_sizes(&self) -> DEUnclumpedSizes<'_>

An iterator over effective (unclumped) sizes that implements DoubleEndedIterator.

Notes

This iterator is used to implement the parallel sizes iterator to enable parallel iteration of Chunked types with clumped offsets. However, this iterator can also be used as is for efficient backwards iteration.

pub fn par_offset_values_and_sizes( &self, ) -> ParUnclumpedOffsetValuesAndSizes<'_>

Parallel version of offsets_and_sizes.

impl<O: AsRef<[usize]>> ClumpedOffsets<O>

pub fn first_chunk_offset_value(&self) -> usize

pub fn last_chunk_offset_value(&self) -> usize

pub fn num_clump_offsets(&self) -> usize

Returns the number of clump offsets represented by ClumpedOffsets.

This is typically significantly smaller than self.num_offsets().

pub fn num_clumps(&self) -> usize

Returns the number of clumps represented by ClumpedOffsets.

pub fn clump_stride(&self, index: usize) -> usize

Compute the stride of the clump at the given index.

Panics

This function panics if index is greater than or equal to self.num_clumps().

impl<O: AsRef<[usize]>> ClumpedOffsets<O>

pub fn sizes(&self) -> UnclumpedSizes<'_>

An iterator over effective (unclumped) sizes producing an increment for advancing the data pointer.

This helps for implementing iterators over Chunked types.

Example

    use flatk::{Offsets, ClumpedOffsets};

    let off = Offsets::new(vec![0, 3, 6, 9, 12, 16, 20, 24, 27, 30, 33, 36, 39]);
    let clumped = ClumpedOffsets::from(off);
    let mut clumped_iter = clumped.sizes();
    for _ in 0..4 {
        assert_eq!(clumped_iter.next(), Some(3));
    }
    for _ in 0..3 {
        assert_eq!(clumped_iter.next(), Some(4));
    }
    for _ in 0..5 {
        assert_eq!(clumped_iter.next(), Some(3));
    }
    assert_eq!(clumped_iter.next(), None);

pub fn iter(&self) -> impl Iterator<Item = usize> + '_

An iterator over unclumped offsets.

This is equivalent to iterating over Offsets after conversion, but it doesn’t require any additional allocations.

pub fn values(&self) -> UnclumpedOffsetValues<'_>

An iterator over unclumped offsets.

This is equivalent to iterating over Offsets after conversion, but it doesn’t require any additional allocations.

impl<O: AsRef<[usize]>> ClumpedOffsets<O>

pub fn new(chunk_offsets: O, offsets: O) -> Self

Construct new ClumpedOffsets from a given valid set of clumped offsets represented as an array of usizes.

It is possible to create an invalid ClumpedOffsets struct using this constructor, however it is safe.

Panics

This function will panic if either chunk_offsets or offsets is empty.

pub unsafe fn from_raw(chunk_offsets: O, offsets: O) -> Self

An unchecked version of the new constructor.

Safety

Calling this function with empty chunk_offsets or offsets may cause undefined behaviour in safe APIs.

Trait Implementations

impl Clear for ClumpedOffsets

fn clear(&mut self)

Remove all elements from the current set without necessarily deallocating the space previously used.

impl<O: Clone> Clone for ClumpedOffsets<O>

fn clone(&self) -> ClumpedOffsets<O>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<O: Debug> Debug for ClumpedOffsets<O>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ClumpedOffsets<Vec<usize>>

A default set of offsets must allocate.

fn default() -> Self

Returns the “default value” for a type.

impl<O: Dummy> Dummy for ClumpedOffsets<O>

A dummy set of offsets will not allocate, but the resulting type does not correspond to valid offsets.

unsafe fn dummy() -> Self

This function creates an invalid ClumpedOffsets, but it does not allocate.

impl Extend<(usize, usize)> for ClumpedOffsets

fn extend<T: IntoIterator<Item = (usize, usize)>>(&mut self, iter: T)

Extend this set of clumped offsets with a given iterator over chunk-offset and offset pairs.

This operation automatically shifts the merged offsets in the iterator to start from the last offset in self.

Note that there will be 1 less offset added to self than produced by iter since the first offset is only used to determine the relative magnitude of the rest and corresponds to the last offset in self.

fn extend_one(&mut self, item: A)

🔬This is a nightly-only experimental API. (extend_one)

Extends a collection with exactly one element.

fn extend_reserve(&mut self, additional: usize)

🔬This is a nightly-only experimental API. (extend_one)

Reserves capacity in a collection for the given number of additional elements.

impl<O: AsRef<[usize]> + Set> From<ClumpedOffsets<O>> for Offsets

fn from(clumped_offsets: ClumpedOffsets<O>) -> Self

Convert ClumpedOffsets into owned Offsets.

This function causes allocations.

impl<O: AsRef<[usize]> + Set> From<Offsets<O>> for ClumpedOffsets

fn from(offsets: Offsets<O>) -> Self

Convert Offsets to owned ClumpedOffsets.

This function causes allocations.

Example

use flatk::{Offsets, ClumpedOffsets};
let offsets = Offsets::new(vec![0, 3, 6, 9, 12, 16, 20, 24, 27, 30, 33, 36, 39]);
let clumped_offsets = ClumpedOffsets::new(vec![0, 4, 7, 12], vec![0, 12, 24, 39]);
assert_eq!(ClumpedOffsets::from(offsets), clumped_offsets);

impl FromIterator<usize> for ClumpedOffsets

fn from_iter<T>(iter: T) -> Self

where T: IntoIterator<Item = usize>,

Creates a value from an iterator.

impl<'a, O> GetIndex<'a, ClumpedOffsets<O>> for &usize

where ClumpedOffsets<O>: GetOffset,

type Output = usize

fn get(self, clumped_offsets: &ClumpedOffsets<O>) -> Option<Self::Output>

Gets the value in the set at this index.

unsafe fn at_unchecked(self, set: &S) -> Self::Output

where Self: Sized,

Gets the value in the set at this index.

impl<'a, O> GetIndex<'a, ClumpedOffsets<O>> for usize

where ClumpedOffsets<O>: GetOffset,

type Output = usize

fn get(self, clumped_offsets: &ClumpedOffsets<O>) -> Option<Self::Output>

Gets the value in the set at this index.

unsafe fn at_unchecked(self, set: &S) -> Self::Output

where Self: Sized,

Gets the value in the set at this index.

impl<O: AsRef<[usize]>> GetOffset for ClumpedOffsets<O>

unsafe fn offset_value_unchecked(&self, index: usize) -> usize

A version of offset_value without bounds checking.

Safety

It is assumed that index is strictly less than self.num_offsets().

fn num_offsets(&self) -> usize

Get the total number of offsets.

This is one more than the number of chunks represented.

fn chunk_len(&self, chunk_index: usize) -> usize

Get the length of the chunk at the given index.

unsafe fn chunk_len_unchecked(&self, chunk_index: usize) -> usize

Get the length of the chunk at the given index without bounds checking.

fn offset_value(&self, index: usize) -> usize

Return the raw value corresponding to the offset at the given index.

fn offset(&self, index: usize) -> usize

Returns the offset at the given index with respect to (minus) the first offset. This function returns the total length of data if index is equal to self.len().

unsafe fn offset_unchecked(&self, index: usize) -> usize

A version of offset without bounds checking.

fn last_offset(&self) -> usize

Get the last offset.

fn first_offset(&self) -> usize

Get the first offset.

fn last_offset_value(&self) -> usize

Get the raw value corresponding to the last offset.

fn first_offset_value(&self) -> usize

Get the raw value corresponding to the first offset.

impl<O> IndexRange for ClumpedOffsets<O>

where Self: GetOffset,

fn index_range(&self, range: Range<usize>) -> Option<Range<usize>>

Return the [begin..end) bound of the chunk at the given index.

unsafe fn index_range_unchecked(&self, range: Range<usize>) -> Range<usize>

Safety

impl<'a> IntoOffsetValuesAndSizes for ClumpedOffsets<&'a [usize]>

type Iter = UnclumpedOffsetValuesAndSizes<'a>

fn into_offset_values_and_sizes(self) -> UnclumpedOffsetValuesAndSizes<'a>

impl<O: IntoOwned> IntoOwned for ClumpedOffsets<O>

type Owned = ClumpedOffsets<<O as IntoOwned>::Owned>

fn into_owned(self) -> Self::Owned

fn clone_into(self, target: &mut Self::Owned)

impl<'a> IntoParOffsetValuesAndSizes for ClumpedOffsets<&'a [usize]>

fn into_par_offset_values_and_sizes(self) -> Self::ParIter

Returns a parallel iterator over chunk offsets and sizes represented by the stored Offsets.

type ParIter = ParUnclumpedOffsetValuesAndSizes<'a>

impl<'a> IntoSizes for ClumpedOffsets<&'a [usize]>

fn into_sizes(self) -> UnclumpedSizes<'a>

Returns an iterator over chunk sizes represented by the stored ClumpedOffsets.

type Iter = UnclumpedSizes<'a>

impl<'a> IntoValues for ClumpedOffsets<&'a [usize]>

fn into_values(self) -> UnclumpedOffsetValues<'a>

Returns an iterator over offset values represented by the stored Offsets.

type Iter = UnclumpedOffsetValues<'a>

impl<O> IsolateIndex<ClumpedOffsets<O>> for usize

where ClumpedOffsets<O>: GetOffset,

type Output = usize

unsafe fn isolate_unchecked( self, clumped_offsets: ClumpedOffsets<O>, ) -> Self::Output

Attempts to isolate a value in the given set at this index.

fn try_isolate(self, clumped_offsets: ClumpedOffsets<O>) -> Option<Self::Output>

Attempts to isolate a value in the given set at this index.

impl<O: PartialEq> PartialEq for ClumpedOffsets<O>

fn eq(&self, other: &ClumpedOffsets<O>) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl<O: RemovePrefix + Set> RemovePrefix for ClumpedOffsets<O>

fn remove_prefix(&mut self, n: usize)

Remove n elements from the beginning.

impl<O: Reserve> Reserve for ClumpedOffsets<O>

fn reserve_with_storage(&mut self, n: usize, storage_n: usize)

fn reserve(&mut self, n: usize)

impl<O: AsRef<[usize]>> Set for ClumpedOffsets<O>

type Elem = usize

Owned element of the set.

type Atom = usize

The most basic element contained by this collection. If this collection contains other collections, this type should be different than Elem.

fn len(&self) -> usize

fn is_empty(&self) -> bool

impl<'a> SplitOffsetsAt for ClumpedOffsets<&'a [usize]>

fn split_offsets_with_intersection_at( self, mid_off: usize, ) -> (ClumpedOffsets<&'a [usize]>, ClumpedOffsets<&'a [usize]>, usize)

Same as split_offsets_at, but in addition, return the offset of the middle element (intersection): this is the value offsets[mid] - offsets[0].

Panics

Calling this function with an empty slice or with mid greater than or equal to its length will cause a panic.

This function will also panic when trying to split a clump since the offsets cannot be modified or created.

fn split_offsets_at( self, mid: usize, ) -> (ClumpedOffsets<&'a [usize]>, ClumpedOffsets<&'a [usize]>)

Splits clumped offsets at the given index into two such that each resulting ClumpedOffsets is valid. This means that the element at index mid is shared between the two outputs.

Panics

Calling this function with an empty slice or with mid greater than or equal to its length will cause a panic.

This function will also panic when trying to split a clump since the offsets cannot be modified or created.

impl<O: Truncate> Truncate for ClumpedOffsets<O>

fn truncate(&mut self, new_len: usize)

impl<'a, O: AsRef<[usize]>> View<'a> for ClumpedOffsets<O>

type Type = ClumpedOffsets<&'a [usize]>

fn view(&'a self) -> Self::Type

impl<'a, O: AsMut<[usize]>> ViewMut<'a> for ClumpedOffsets<O>

type Type = ClumpedOffsets<&'a mut [usize]>

fn view_mut(&'a mut self) -> Self::Type

impl<O: Copy> Copy for ClumpedOffsets<O>

impl<O> StructuralPartialEq for ClumpedOffsets<O>

impl<O: Viewed> Viewed for ClumpedOffsets<O>