Skip to main content

Module super_array_view

Module super_array_view 

Source
Expand description

§SuperArrayView Module - Chunked, Windowed View over 1:M Arrays

SuperArrayV is a borrowed, chunked view over a single logical array, exposing an arbitrary [offset .. offset + len) window that may span multiple underlying chunks. It presents those chunks as one continuous logical range without copying the underlying memory.

§Role

  • Represents a mini-batch window for one Array (or one entry inside a SuperArray), useful for streaming, batching, or region-wise processing.
  • Similar in shape to SuperArray, but semantically a view over a single column’s region rather than a bag of heterogeneous columns.

§Interop

  • Constructed by higher-level chunked containers (SuperArray::slice).
  • Can be materialised to a contiguous Array via SuperArrayV::consolidate.
  • Preserves schema via an Arc<Field>; no schema cloning per row.

§Features

§Performance Notes

  • Iterating rows may touch non-contiguous memory if the window crosses chunk boundaries. For hot loops, prefer contiguous runs or materialise with consolidate().

§Invariants

  • len is the logical row count of this view.
  • slices are ordered, non-overlapping, and cover at most len rows.
  • field is the schema for the underlying array and is shared by all slices.

Structs§

SuperArrayV
SuperArrayView