Struct DictionaryEncoder 

pub struct DictionaryEncoder<T: Eq + Hash + Clone> {
    pub vocab: Vec<T>,
    pub unknown_policy: UnknownPolicy,
    /* private fields */
}

Dictionary encoder for a generic value type T.

T must be Eq + Hash + Clone + Ord; the Ord bound is used only for tie-breaking when two values have the same frequency.

Fields

vocab: Vec<T>

Vocabulary: vocab[code] → value. Sorted by descending frequency.

unknown_policy: UnknownPolicy

Policy for values not present in the vocabulary.

Implementations

impl<T: Eq + Hash + Clone + Ord> DictionaryEncoder<T>

pub fn new(unknown_policy: UnknownPolicy) -> Self

Create an empty encoder with the given unknown policy.

pub fn fit(data: &[T], unknown_policy: UnknownPolicy) -> Result<Self>

Build a DictionaryEncoder by scanning data and counting frequencies.

The vocabulary is sorted by descending frequency (ties broken by value ordering). Codes 0..vocab.len() are assigned in that order.

Errors

Returns an error if the number of unique values exceeds u16::MAX - 1 (= 65534).

pub fn encode(&self, data: &[T]) -> Result<Vec<u16>>

where T: Debug,

Encode a slice of values to codes.

Behaviour for unknown values is governed by self.unknown_policy.

pub fn decode(&self, codes: &[u16]) -> Result<Vec<T>>

Decode a slice of codes back to values.

Returns an error if any code equals u16::MAX (null sentinel) or is out of bounds.

pub fn vocab_size(&self) -> usize

Return the current vocabulary size.

pub fn encode_single(&self, value: &T) -> Result<u16>

where T: Debug,

Encode a single value to its code.

pub fn encode_single_mut(&mut self, value: T) -> Result<u16>

where T: Debug,

Encode a single value, potentially adding it to the vocabulary (only meaningful when unknown_policy == AddToVocab).

pub fn decode_single(&self, code: u16) -> Result<&T>

Decode a single code to a reference to the corresponding value.

Trait Implementations

impl<T: Clone + Eq + Hash + Clone> Clone for DictionaryEncoder<T>

fn clone(&self) -> DictionaryEncoder<T>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<T: Debug + Eq + Hash + Clone> Debug for DictionaryEncoder<T>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.