Struct General 

pub struct General {
    pub compression: Option<BufferCompression>,
    pub values: Option<Box<CompressiveEncoding>>,
}

A compression scheme that wraps the underlying data with general compression

Note: The application of wrapped compression will depend on the layout of the data. If we apply it to mini-block data then we compress entire mini-blocks. If we apply it to full-zip data then we compress each value individually.

Note: Wrapped compression is somewhat unique at the moment as it is applied to the output of the inner encoding and not the input like all other compressive encodings.

Note: General compression can usually be applied in two spots. We can apply it to individual buffers or we can apply it here, to the entire array.

For example, let’s say we are storing mini-blocks of strings and we are using FSST and bitpacking the offsets. We have something like this…

WRAPPED(†3) -> FSST -> VARIABLE -(offsets)-> INLINE_BITPACKING -(data)-> FLAT -> BUFFER (†1) -(data)-> BUFFER (†2)

General compression can be applied at †1, †2, or †3 (or any combination of these).

If we apply it at †1 then we apply it just to the bitpacked offsets If we apply it at †2 then we apply it just to the FSST compressed data If we apply it at †3 then we apply it to the entire mini-block (both offsets and data)

The input is a single data block of any kind. The output is a single data block of the same kind as the input.

Fields

compression: Option<BufferCompression>

The compression to apply to the values

values: Option<Box<CompressiveEncoding>>

The compression used to store the output data block

Trait Implementations

impl Clone for General

fn clone(&self) -> General

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for General

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for General

fn default() -> Self

Returns the “default value” for a type.

impl Message for General

fn encoded_len(&self) -> usize

Returns the encoded length of the message without a length delimiter.

fn clear(&mut self)

Clears the message, resetting all fields to their default.

fn encode(&self, buf: &mut impl BufMut) -> Result<(), EncodeError>

where Self: Sized,

Encodes the message to a buffer.

fn encode_to_vec(&self) -> Vec<u8>

where Self: Sized,

Encodes the message to a newly allocated buffer.

fn encode_length_delimited( &self, buf: &mut impl BufMut, ) -> Result<(), EncodeError>

where Self: Sized,

Encodes the message with a length-delimiter to a buffer.

fn encode_length_delimited_to_vec(&self) -> Vec<u8>

where Self: Sized,

Encodes the message with a length-delimiter to a newly allocated buffer.

fn decode(buf: impl Buf) -> Result<Self, DecodeError>

where Self: Default,

Decodes an instance of the message from a buffer.

fn decode_length_delimited(buf: impl Buf) -> Result<Self, DecodeError>

where Self: Default,

Decodes a length-delimited instance of the message from the buffer.

fn merge(&mut self, buf: impl Buf) -> Result<(), DecodeError>

where Self: Sized,

Decodes an instance of the message from a buffer, and merges it into self.

fn merge_length_delimited(&mut self, buf: impl Buf) -> Result<(), DecodeError>

where Self: Sized,

Decodes a length-delimited instance of the message from buffer, and merges it into self.

impl Name for General

const NAME: &'static str = "General"

Simple name for this Message. This name is the same as it appears in the source .proto file, e.g. FooBar.

const PACKAGE: &'static str = "lance.encodings21"

Package name this message type is contained in. They are domain-like and delimited by ., e.g. google.protobuf.

fn full_name() -> String

Fully-qualified unique name for this Message. It’s prefixed with the package name and names of any parent messages, e.g. google.rpc.BadRequest.FieldViolation. By default, this is the package name followed by the message name. Fully-qualified names must be unique within a domain of Type URLs.

fn type_url() -> String

Type URL for this Message, which by default is the full name with a leading slash, but may also include a leading domain name, e.g. type.googleapis.com/google.profile.Person. This can be used when serializing into the google.protobuf.Any type.

impl PartialEq for General

fn eq(&self, other: &General) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for General