Struct deltalake::arrow::ipc::SparseTensorIndexCSF

pub struct SparseTensorIndexCSF<'a> {
    pub _tab: Table<'a>,
}

Compressed Sparse Fiber (CSF) sparse tensor index.

Fields

_tab: Table<'a>

Implementations

impl<'a> SparseTensorIndexCSF<'a>

pub const VT_INDPTRTYPE: u16 = 4u16

pub const VT_INDPTRBUFFERS: u16 = 6u16

pub const VT_INDICESTYPE: u16 = 8u16

pub const VT_INDICESBUFFERS: u16 = 10u16

pub const VT_AXISORDER: u16 = 12u16

pub unsafe fn init_from_table(table: Table<'a>) -> SparseTensorIndexCSF<'a>

pub fn create<'bldr, 'args, 'mut_bldr>( _fbb: &'mut_bldr mut FlatBufferBuilder<'bldr>, args: &'args SparseTensorIndexCSFArgs<'args> ) -> WIPOffset<SparseTensorIndexCSF<'bldr>>

where 'bldr: 'args, 'args: 'mut_bldr,

pub fn indptrType(&self) -> Int<'a>

CSF is a generalization of compressed sparse row (CSR) index. See smith2017knl

CSF index recursively compresses each dimension of a tensor into a set of prefix trees. Each path from a root to leaf forms one tensor non-zero index. CSF is implemented with two arrays of buffers and one arrays of integers.

For example, let X be a 2x3x4x5 tensor and let it have the following 8 non-zero values:

  X[0, 0, 0, 1] := 1
  X[0, 0, 0, 2] := 2
  X[0, 1, 0, 0] := 3
  X[0, 1, 0, 2] := 4
  X[0, 1, 1, 0] := 5
  X[1, 1, 1, 0] := 6
  X[1, 1, 1, 1] := 7
  X[1, 1, 1, 2] := 8

As a prefix tree this would be represented as:

        0          1
       / \         |
      0   1        1
     /   / \       |
    0   0   1      1
   /|  /|   |    /| |
  1 2 0 2   0   0 1 2

The type of values in indptrBuffers

pub fn indptrBuffers(&self) -> Vector<'a, Buffer>

indptrBuffers stores the sparsity structure. Each two consecutive dimensions in a tensor correspond to a buffer in indptrBuffers. A pair of consecutive values at indptrBuffers[dim][i] and indptrBuffers[dim][i + 1] signify a range of nodes in indicesBuffers[dim + 1] who are children of indicesBuffers[dim][i] node.

For example, the indptrBuffers for the above X is:

  indptrBuffer(X) = [
                      [0, 2, 3],
                      [0, 1, 3, 4],
                      [0, 2, 4, 5, 8]
                    ].

pub fn indicesType(&self) -> Int<'a>

The type of values in indicesBuffers

pub fn indicesBuffers(&self) -> Vector<'a, Buffer>

indicesBuffers stores values of nodes. Each tensor dimension corresponds to a buffer in indicesBuffers. For example, the indicesBuffers for the above X is:

  indicesBuffer(X) = [
                       [0, 1],
                       [0, 1, 1],
                       [0, 0, 1, 1],
                       [1, 2, 0, 2, 0, 0, 1, 2]
                     ].

pub fn axisOrder(&self) -> Vector<'a, i32>

axisOrder stores the sequence in which dimensions were traversed to produce the prefix tree. For example, the axisOrder for the above X is:

  axisOrder(X) = [0, 1, 2, 3].

Trait Implementations

impl<'a> Clone for SparseTensorIndexCSF<'a>

fn clone(&self) -> SparseTensorIndexCSF<'a>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SparseTensorIndexCSF<'_>

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl<'a> Follow<'a> for SparseTensorIndexCSF<'a>

type Inner = SparseTensorIndexCSF<'a>

unsafe fn follow( buf: &'a [u8], loc: usize ) -> <SparseTensorIndexCSF<'a> as Follow<'a>>::Inner

Safety

impl<'a> PartialEq for SparseTensorIndexCSF<'a>

fn eq(&self, other: &SparseTensorIndexCSF<'a>) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Verifiable for SparseTensorIndexCSF<'_>

fn run_verifier( v: &mut Verifier<'_, '_>, pos: usize ) -> Result<(), InvalidFlatbuffer>

Runs the verifier for this type, assuming its at position pos in the verifier’s buffer. Should not need to be called directly.

impl<'a> Copy for SparseTensorIndexCSF<'a>

impl<'a> StructuralPartialEq for SparseTensorIndexCSF<'a>