Skip to main content

Module categorical

simd_kernels::structs::variants

Module categorical 

Source
Expand description

§CategoricalArray Module - Mid-Level, Inner Typed Categorical Array

CategoricalArray uses dictionary-encoded strings where each row stores a small integer “code” that references a per-column dictionary of unique strings. This saves memory and accelerates comparisons/joins when many values repeat.

§Interop

  • Arrow-compatible dictionary layout (indices + string dictionary), and round-trips over the Arrow C Data Interface to/from the Dictionary array type.
  • Index width is the generic T (e.g., u8/u16/u32/u64) and corresponds to Arrow’s CategoricalIndexType.

§Features

  • Optional null_mask: bit-packed, where 1 = valid, 0 = null
  • Builders from raw values (from_values, from_vec64) and from raw parts.
  • Iterators over indices and over resolved strings (nullable and non-nullable).
  • Convert to a dense StringArray via to_string_array() when needed.
  • Parallel helpers behind parallel_proc feature.

§When to use

Use for arrays with repeated strings to reduce memory and speed up operations.

Structs§

CategoricalArray
CategoricalArray