Struct ArrowRleBatch 

pub struct ArrowRleBatch { /* private fields */ }

Many Arrow-backed sequences, each identified by a row of a key table.

The keys are a RecordBatch with any number of columns of any types (for example a chromosome and a strand column), one row per sequence. Sequences are stored back to back: sequence i is runs offsets[i]..offsets[i + 1] of ends and values, with ends restarting at 0. Batch operations run in parallel across sequences.

Implementations

impl ArrowRleBatch

pub fn try_new( keys: RecordBatch, offsets: Vec<usize>, ends: Vec<i64>, values: ArrayRef, ) -> Result<Self>

Builds a batch from its flat parts, validating them. Keys must be unique.

pub fn from_rles( keys: RecordBatch, rles: Vec<ArrowRle>, data_type: &DataType, ) -> Result<Self>

Builds a batch from one key row and one sequence per entry. All sequences must have the same value type, data_type.

pub fn len(&self) -> usize

The number of sequences.

pub fn is_empty(&self) -> bool

pub fn keys(&self) -> &RecordBatch

The key table, one row per sequence.

pub fn offsets(&self) -> &[usize]

pub fn ends(&self) -> &[i64]

The run ends of all sequences, back to back.

pub fn values(&self) -> &ArrayRef

The run values of all sequences, back to back.

pub fn get_index(&self, position: usize) -> Option<ArrowRle>

The sequence at position. The values are a zero-copy slice.

pub fn iter(&self) -> impl Iterator<Item = ArrowRle> + '_

The sequences in key order.

pub fn positions(&self, keys: &RecordBatch) -> Result<Vec<Option<usize>>>

The position of each row of keys in this batch, or None if absent. keys must have the same column types as this batch’s keys.

pub fn zip_with<F>( &self, other: &ArrowRleBatch, how: How, fill_a: Option<&dyn Array>, fill_b: Option<&dyn Array>, f: F, ) -> Result<ArrowRleBatch>

where F: Fn(&dyn Array, &dyn Array) -> Result<ArrayRef, ArrowError> + Sync,

Combines two batches key by key, in parallel, merging each result.

Both key tables must have the same column names and types. how picks the keys, which appear in the left batch’s order followed by keys only in the right batch. A key missing on one side is an empty sequence there, padded with that side’s fill value. See ArrowRle::zip_with for f.

pub fn to_record_batch(&self) -> Result<RecordBatch>

The batch as one long table: the key columns, then start, end and value, with one row per run. This is the natural shape for polars or pandas.

pub fn zip_with_default( &self, other: &ArrowRleBatch, f: Kernel, ) -> Result<ArrowRleBatch>

Combines with other by an outer join, using default_fill on each side, and merges.

impl ArrowRleBatch

pub fn add(&self, other: &ArrowRleBatch) -> Result<ArrowRleBatch>

Element-wise addition (integer overflow is an error). Keys are outer-joined and shorter sides padded with default_fill.

pub fn sub(&self, other: &ArrowRleBatch) -> Result<ArrowRleBatch>

Element-wise subtraction (integer overflow is an error). Keys are outer-joined and shorter sides padded with default_fill.

pub fn mul(&self, other: &ArrowRleBatch) -> Result<ArrowRleBatch>

Element-wise multiplication (integer overflow is an error). Keys are outer-joined and shorter sides padded with default_fill.

pub fn div(&self, other: &ArrowRleBatch) -> Result<ArrowRleBatch>

Element-wise division (integer division by zero is an error). Keys are outer-joined and shorter sides padded with default_fill.

pub fn elem_eq(&self, other: &ArrowRleBatch) -> Result<ArrowRleBatch>

Element-wise ==. Keys are outer-joined and shorter sides padded with default_fill.

pub fn elem_ne(&self, other: &ArrowRleBatch) -> Result<ArrowRleBatch>

Element-wise !=. Keys are outer-joined and shorter sides padded with default_fill.

pub fn elem_lt(&self, other: &ArrowRleBatch) -> Result<ArrowRleBatch>

Element-wise <. Keys are outer-joined and shorter sides padded with default_fill.

pub fn elem_le(&self, other: &ArrowRleBatch) -> Result<ArrowRleBatch>

Element-wise <=. Keys are outer-joined and shorter sides padded with default_fill.

pub fn elem_gt(&self, other: &ArrowRleBatch) -> Result<ArrowRleBatch>

Element-wise >. Keys are outer-joined and shorter sides padded with default_fill.

pub fn elem_ge(&self, other: &ArrowRleBatch) -> Result<ArrowRleBatch>

Element-wise >=. Keys are outer-joined and shorter sides padded with default_fill.

pub fn and(&self, other: &ArrowRleBatch) -> Result<ArrowRleBatch>

Element-wise logical and, with SQL semantics for nulls. Keys are outer-joined and shorter sides padded with default_fill.

pub fn or(&self, other: &ArrowRleBatch) -> Result<ArrowRleBatch>

Element-wise logical or, with SQL semantics for nulls. Keys are outer-joined and shorter sides padded with default_fill.

impl ArrowRleBatch

pub fn slice_agg( &self, keys: &RecordBatch, starts: &[i64], ends: &[i64], agg: SliceAgg, ) -> Result<ArrayRef>

Like ArrowRle::slice_agg, where slice k is in the sequence of row k of keys. A key not in the batch gives an empty slice. keys must have the same column names and types as the batch’s keys.

impl ArrowRleBatch

pub fn collapse(&self, keep: &[&str], kernel: Kernel) -> Result<ArrowRleBatch>

Combines the sequences whose keys agree on the keep columns, folding each group with kernel in key order, in parallel across groups.

For example, keeping only Chromosome with kernels::add adds the strands of each chromosome together. Sequences of different lengths are padded with default_fill.

impl ArrowRleBatch

pub fn shift(&self, distance: i64, fill: &dyn Array) -> Result<ArrowRleBatch>

Shifts every sequence, in parallel; see ArrowRle::shift.

pub fn reversed(&self) -> Result<ArrowRleBatch>

Reverses every sequence, in parallel.

pub fn map_rles<F>(&self, f: F) -> Result<ArrowRleBatch>

where F: Fn(&ArrowRle) -> Result<ArrowRle> + Sync,

Transforms every sequence in parallel, keeping the keys. All results must have the same value type.

pub fn concat(batches: &[ArrowRleBatch]) -> Result<ArrowRleBatch>

Joins batches end to end key by key, in parallel: each key’s sequences are concatenated in batch order, and a batch without the key adds nothing. Keys appear in order of first appearance. All batches must have the same key columns and value type.

Trait Implementations

impl Clone for ArrowRleBatch

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ArrowRleBatch

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for ArrowRleBatch

fn eq(&self, other: &Self) -> bool

Same keys in the same order, and equal sequences for each.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.