Struct DictionaryCompressor 

pub struct DictionaryCompressor;

Dictionary compression for categorical data

Replaces frequently occurring values with indices into a dictionary. Excellent for high-cardinality categorical columns.

Examples

let values = vec!["red", "blue", "red", "green", "blue", "red"];
let (compressed, dict) = DictionaryCompressor::compress(&values)?;
// Dictionary: {"red": 0, "blue": 1, "green": 2}
// Compressed: [0, 1, 0, 2, 1, 0]

Implementations

impl DictionaryCompressor

pub fn compress_strings( values: &[&str], ) -> Result<(Vec<u8>, HashMap<String, u32>), BinaryFormatError>

Compress string values using dictionary encoding

Returns (compressed indices, dictionary)

pub fn decompress_strings( bytes: &[u8], dictionary: &HashMap<String, u32>, ) -> Result<Vec<String>, BinaryFormatError>

Decompress dictionary-encoded values

Trait Implementations

impl Debug for DictionaryCompressor

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.