Struct CsvSourceConfig 

pub struct CsvSourceConfig {
    pub path: String,
    pub has_headers: bool,
    pub delimiter: u8,
    pub quote: u8,
    pub batch_size: usize,
    pub compression: CompressionConfig,
}

Available on crate features source-csv and source-rest only.

Configuration for the CSV file source.

Fields

path: String

Path to the CSV file.

has_headers: bool

Whether the file has a header row. Defaults to true.

delimiter: u8

Field delimiter byte. Defaults to b','.

quote: u8

Quote character byte. Defaults to b'"'.

batch_size: usize

Records per emitted StreamPage. Rows are parsed line-by-line from a tokio BufReader and yielded whenever the buffer reaches this size. Defaults to DEFAULT_BATCH_SIZE.

batch_size = 0 is the “no batching” sentinel: the file is fully drained and the entire result set is emitted in a single page. Useful for small lookup tables or for sinks (e.g. SQL COPY, BigQuery load jobs) that prefer one large request to many small ones.

compression: CompressionConfig

Available on crate feature compression only.

Compression codec for the input file. Defaults to CompressionConfig::Auto — .gz / .zst suffix selects gzip / zstd. Requires the crate-local compression feature.

Implementations

impl CsvSourceConfig

pub fn new(path: impl Into<String>) -> CsvSourceConfig

Create a new config with the required file path and sensible defaults.

pub fn has_headers(self, v: bool) -> CsvSourceConfig

Set whether the file has a header row.

pub fn delimiter(self, d: u8) -> CsvSourceConfig

Set the field delimiter byte.

pub fn quote(self, q: u8) -> CsvSourceConfig

Set the quote character byte.

pub fn with_batch_size(self, batch_size: usize) -> CsvSourceConfig

Set the per-page row count for Source::stream_pages.

Pass 0 to opt out of batching — the entire file is emitted in a single StreamPage.

pub fn compression(self, c: CompressionConfig) -> CsvSourceConfig

Available on crate feature compression only.

Set the compression codec. Available only with the compression feature.

Trait Implementations

impl Clone for CsvSourceConfig

fn clone(&self) -> CsvSourceConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for CsvSourceConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for CsvSourceConfig

fn deserialize<__D>( __deserializer: __D, ) -> Result<CsvSourceConfig, <__D as Deserializer<'de>>::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl JsonSchema for CsvSourceConfig

fn schema_name() -> Cow<'static, str>

The name of the generated JSON Schema.

fn schema_id() -> Cow<'static, str>

Returns a string that uniquely identifies the schema produced by this type.

fn json_schema(generator: &mut SchemaGenerator) -> Schema

Generates a JSON Schema for this type.

fn inline_schema() -> bool

Whether JSON Schemas generated for this type should be included directly in parent schemas, rather than being re-used where possible using the $ref keyword.

impl Serialize for CsvSourceConfig

fn serialize<__S>( &self, __serializer: __S, ) -> Result<<__S as Serializer>::Ok, <__S as Serializer>::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.