Struct CompressingStrategy 

pub struct CompressingStrategy { /* private fields */ }

A layout writer that compresses chunks.

Implementations

impl CompressingStrategy

pub fn new_btrblocks<S: LayoutStrategy>( child: S, exclude_int_dict_encoding: bool, ) -> Self

Create a new writer that uses the BtrBlocks-style cascading compressor to compress chunks.

This provides a good balance between decoding speed and small file size.

Set exclude_int_dict_encoding to true to prevent dictionary encoding of integer arrays, which is useful when compressing dictionary codes to avoid recursive dictionary encoding.

pub fn new_compact<S: LayoutStrategy>( child: S, compressor: CompactCompressor, ) -> Self

Create a new writer that compresses using a CompactCompressor to compress chunks.

This may create smaller files than the BtrBlocks writer, in exchange for some penalty to decoding performance. This is only recommended for datasets that make heavy use of floating point numbers.

pub fn new_opaque<S: LayoutStrategy, C: CompressorPlugin>( child: S, compressor: C, ) -> Self

Create a new compressor from a plugin interface.

pub fn with_concurrency(self, concurrency: usize) -> Self

Trait Implementations

impl Clone for CompressingStrategy

fn clone(&self) -> CompressingStrategy

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl LayoutStrategy for CompressingStrategy

fn write_stream<'life0, 'async_trait>( &'life0 self, ctx: ArrayContext, segment_sink: SegmentSinkRef, stream: SendableSequentialStream, eof: SequencePointer, handle: Handle, ) -> Pin<Box<dyn Future<Output = VortexResult<LayoutRef>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Asynchronously process an ordered stream of array chunks, emitting them into a sink and returning the Layout instance that can be parsed to retrieve the data from rest.

fn buffered_bytes(&self) -> u64

Returns the number of bytes currently buffered by this strategy and any child strategies.