Struct q_compress::Flags
source · #[non_exhaustive]pub struct Flags {
pub use_5_bit_code_len: bool,
pub delta_encoding_order: usize,
pub use_min_count_encoding: bool,
pub use_gcds: bool,
pub use_wrapped_mode: bool,
}
Expand description
The configuration stored in a Quantile-compressed header.
During compression, flags are determined based on your CompressorConfig
and the q_compress
version.
Flags affect the encoding of the rest of the file, so decompressing with
the wrong flags will likely cause a corruption error.
You will not need to manually instantiate flags; that should be done
internally by Compressor::from_config
.
However, in some circumstances you may want to inspect flags during
decompression.
Fields (Non-exhaustive)§
This struct is marked as non-exhaustive
Struct { .. }
syntax; cannot be matched against without a wildcard ..
; and struct update syntax will not work.use_5_bit_code_len: bool
Whether to use 5 bits to encode the length of a prefix Huffman code,
as opposed to 4.
The first versions of q_compress
used 4, which was insufficient for
Huffman codes that could reach up to 23 in length
(23 >= 16 = 2^4)
in spiky distributions with high compression level.
In later versions, this flag is always true.
Introduced in 0.5.0.
delta_encoding_order: usize
How many times delta encoding was applied during compression.
This is stored as 3 bits to express 0-7.
See CompressorConfig
for more details.
Introduced in 0.6.0.
use_min_count_encoding: bool
Whether to use the minimum number of bits to encode the count of each prefix, rather than using a constant number of bits. This can reduce file size slightly for small data. In later versions, this flag is always true.
Introduced in 0.9.1.
use_gcds: bool
Whether to enable greatest common divisor multipliers for each
prefix.
This adds an optional multiplier to each prefix metadata, so that each
unsigned number is decoded as x = prefix_lower + offset * gcd
.
This can improve compression ratio in some cases, e.g. when the
numbers are all integer multiples of 100 or all integer-valued floats.
Introduced in 0.10.0.
use_wrapped_mode: bool
Whether to release control to a wrapping columnar format. This causes q_compress to omit count and compressed size metadata and also break each chuk into finer data pages.
Introduced in 0.11.2.