Struct SDFConfig 

pub struct SDFConfig {
    pub batch_size: usize,
    pub file_schema: SchemaRef,
    pub object_store: Arc<dyn ObjectStore>,
    pub projection: Option<Vec<usize>>,
    pub limit: Option<usize>,
}

Configuration for a SDF data source.

Fields

batch_size: usize

The number of rows to read at a time.

file_schema: SchemaRef

The schema of the file.

object_store: Arc<dyn ObjectStore>

The object store to use.

projection: Option<Vec<usize>>

Any projections to apply to the resulting batches.

limit: Option<usize>

The limit of rows to read.

Implementations

impl SDFConfig

pub fn new( object_store: Arc<dyn ObjectStore>, batch_size: usize, file_schema: SchemaRef, ) -> Self

pub fn effective_batch_size(&self) -> usize

Get the effective batch size, which is the minimum of the batch size and the limit.

pub fn with_limit_opt(self, limit: Option<usize>) -> Self

Set the limit.

pub fn projection(&self) -> Vec<usize>

Get the projection.

pub fn projected_schema(&self) -> Result<SchemaRef>

Get the projected schema.

pub fn with_projection(self, projection: Vec<usize>) -> Self

Create a new SDF configuration with a given projection.

Trait Implementations

impl Debug for SDFConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.