Struct datafusion::common::arrow::ipc::writer::DictionaryTracker

pub struct DictionaryTracker { /* private fields */ }

Keeps track of dictionaries that have been written, to avoid emitting the same dictionary multiple times. Can optionally error if an update to an existing dictionary is attempted, which isn’t allowed in the FileWriter.

Implementations

impl DictionaryTracker

pub fn new(error_on_replacement: bool) -> DictionaryTracker

Create a new DictionaryTracker. If error_on_replacement is true, an error will be generated if an update to an existing dictionary is attempted.

pub fn insert( &mut self, dict_id: i64, column: &Arc<dyn Array> ) -> Result<bool, ArrowError>

Keep track of the dictionary with the given ID and values. Behavior:

• If this ID has been written already and has the same data, return Ok(false) to indicate that the dictionary was not actually inserted (because it’s already been seen).
• If this ID has been written already but with different data, and this tracker is configured to return an error, return an error.
• If the tracker has not been configured to error on replacement or this dictionary has never been seen before, return Ok(true) to indicate that the dictionary was just inserted.