Struct VortexTableOptions 

#[non_exhaustive]
pub struct VortexTableOptions {
    pub footer_initial_read_size_bytes: usize,
    pub projection_pushdown: bool,
    pub predicate_pushdown: bool,
    pub scan_concurrency: Option<usize>,
}

Options to configure VortexFormat and VortexSource.

The API follows DataFusion’s built-in Parquet and JSON format factories: a format factory may carry customized defaults, the session may carry format defaults, and CREATE EXTERNAL TABLE ... OPTIONS(...) can override individual fields for one table.

FileFormatFactory::create builds the VortexTableOptions copied into each VortexFormat as follows:

1. If the factory has explicit options from VortexFormatFactory::with_options or VortexFormatFactory::new_with_options, start from that complete VortexTableOptions value. This matches ParquetFormatFactory::new_with_options and JsonFormatFactory::new_with_options: factory options replace session defaults; they are not merged with them field-by-field.
2. If the factory does not have explicit options, read the session’s vortex extension at the time create is called. This is the value changed by SET vortex.<option> = ....
3. If the session has no vortex extension, start from VortexTableOptions::default().
4. Apply table OPTIONS(...) last. Each option overwrites only its matching field, so per-table settings can override either the factory options or the session/default value.

In SQL, session settings use the vortex. prefix. Table options use the field names directly, the same style as Parquet or JSON table options:

SET vortex.predicate_pushdown = false;

CREATE EXTERNAL TABLE t (x BIGINT)
STORED AS vortex
LOCATION 's3://bucket/path/'
OPTIONS(predicate_pushdown 'true');

Example

use vortex_datafusion::{VortexFormatFactory, VortexTableOptions};

let mut options = VortexTableOptions::default();
options.predicate_pushdown = true;
options.projection_pushdown = true;
options.scan_concurrency = Some(8);

let factory = VortexFormatFactory::new().with_options(options);

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

footer_initial_read_size_bytes: usize

The number of bytes to read when parsing a file footer.

Values smaller than MAX_POSTSCRIPT_SIZE + EOF_SIZE will be clamped to that minimum during footer parsing.

projection_pushdown: bool

Whether to enable projection pushdown into the underlying Vortex scan.

When enabled, projection expressions may be partially evaluated during the scan. When disabled, Vortex reads only the referenced columns and all expressions are evaluated after the scan.

Disabled by default.

predicate_pushdown: bool

Whether to enable predicate pushdown into the underlying Vortex scan.

When enabled, supported filters are evaluated during the scan. When disabled, DataFusion evaluates filters after the scan, while VortexSource can still use the full predicate for file pruning.

Enabled by default.

scan_concurrency: Option<usize>

The intra-partition scan concurrency, controlling the number of row splits to process concurrently per-thread within each file.

This does not affect the overall parallelism across partitions, which is controlled by DataFusion’s execution configuration.

Leave as None to use Vortex’s scan default. Override per session with SET vortex.scan_concurrency = <n>, or per table with OPTIONS(scan_concurrency '<n>').

Trait Implementations

impl Clone for VortexTableOptions

fn clone(&self) -> VortexTableOptions

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl ConfigExtension for VortexTableOptions

const PREFIX: &'static str = "vortex"

Configuration namespace prefix to use

impl ConfigField for VortexTableOptions

fn set(&mut self, key: &str, value: &str) -> Result<()>

fn visit<V: Visit>( &self, v: &mut V, _key_prefix: &str, _description: &'static str, )

fn reset(&mut self, key: &str) -> Result<(), DataFusionError>

impl Debug for VortexTableOptions

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for VortexTableOptions

fn default() -> Self

Returns the “default value” for a type.

impl ExtensionOptions for VortexTableOptions

fn as_any(&self) -> &dyn Any

Return self as Any

fn as_any_mut(&mut self) -> &mut dyn Any

Return self as Any

fn cloned(&self) -> Box<dyn ExtensionOptions>

Return a deep clone of this ExtensionOptions

fn set(&mut self, key: &str, value: &str) -> Result<()>

Set the given key, value pair

fn entries(&self) -> Vec<ConfigEntry>

Returns the ConfigEntry stored in this ExtensionOptions