pub struct RepartitionWriter { /* private fields */ }Expand description
Repartition a stream of arrays into blocks.
Each emitted block (except the last) is at least block_size_minimum bytes and contains a
multiple of block_len_multiple rows.
Implementations§
Source§impl RepartitionWriter
impl RepartitionWriter
pub fn new( dtype: DType, writer: Box<dyn LayoutWriter>, options: RepartitionWriterOptions, ) -> Self
Trait Implementations§
Source§impl LayoutWriter for RepartitionWriter
impl LayoutWriter for RepartitionWriter
Source§fn push_chunk(
&mut self,
segment_writer: &mut dyn SegmentWriter,
chunk: ArrayRef,
) -> VortexResult<()>
fn push_chunk( &mut self, segment_writer: &mut dyn SegmentWriter, chunk: ArrayRef, ) -> VortexResult<()>
Push a chunk into the layout writer.
Source§fn flush(&mut self, segment_writer: &mut dyn SegmentWriter) -> VortexResult<()>
fn flush(&mut self, segment_writer: &mut dyn SegmentWriter) -> VortexResult<()>
Flush any buffered chunks.
Source§fn finish(
&mut self,
segment_writer: &mut dyn SegmentWriter,
) -> VortexResult<Layout>
fn finish( &mut self, segment_writer: &mut dyn SegmentWriter, ) -> VortexResult<Layout>
Write any final data (e.g. stats) and return the finished
Layout.Auto Trait Implementations§
impl Freeze for RepartitionWriter
impl !RefUnwindSafe for RepartitionWriter
impl Send for RepartitionWriter
impl !Sync for RepartitionWriter
impl Unpin for RepartitionWriter
impl !UnwindSafe for RepartitionWriter
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§impl<L> LayoutWriterExt for Lwhere
L: LayoutWriter + ?Sized,
impl<L> LayoutWriterExt for Lwhere
L: LayoutWriter + ?Sized,
Source§fn push_one(
&mut self,
segment_writer: &mut dyn SegmentWriter,
chunk: ArrayRef,
) -> VortexResult<Layout>
fn push_one( &mut self, segment_writer: &mut dyn SegmentWriter, chunk: ArrayRef, ) -> VortexResult<Layout>
Push a single chunk into the layout writer and return the finished
Layout.Source§fn push_all<I: IntoIterator<Item = VortexResult<ArrayRef>>>(
&mut self,
segment_writer: &mut dyn SegmentWriter,
iter: I,
) -> VortexResult<Layout>
fn push_all<I: IntoIterator<Item = VortexResult<ArrayRef>>>( &mut self, segment_writer: &mut dyn SegmentWriter, iter: I, ) -> VortexResult<Layout>
Push all chunks of the iterator into the layout writer and return the finished
Layout.