Struct lance_file::v2::writer::FileWriterOptions

pub struct FileWriterOptions {
    pub data_cache_bytes: Option<u64>,
}

Fields

data_cache_bytes: Option<u64>

How many bytes to use for buffering column data

When data comes in small batches the writer will buffer column data so that larger pages can be created. This value will be divided evenly across all of the columns. Generally you want this to be at least large enough to match your filesystem’s ideal read size per column.

In some cases you might want this value to be even larger if you have highly compressible data. However, if this is too large, then the writer could require a lot of memory and write performance may suffer if the CPU-expensive encoding falls behind and can’t be interleaved with the I/O expensive flushing.

The default will use 8MiB per column which should be reasonable for most cases.

Trait Implementations

impl Clone for FileWriterOptions

fn clone(&self) -> FileWriterOptions

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for FileWriterOptions

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for FileWriterOptions

fn default() -> FileWriterOptions

Returns the “default value” for a type.