Struct aws_sdk_glue::types::builders::S3CsvSourceBuilder

#[non_exhaustive]
pub struct S3CsvSourceBuilder { /* private fields */ }

A builder for S3CsvSource.

Implementations

impl S3CsvSourceBuilder

pub fn name(self, input: impl Into<String>) -> Self

The name of the data store.

This field is required.

pub fn set_name(self, input: Option<String>) -> Self

The name of the data store.

pub fn get_name(&self) -> &Option<String>

The name of the data store.

pub fn paths(self, input: impl Into<String>) -> Self

Appends an item to paths.

To override the contents of this collection use set_paths.

A list of the Amazon S3 paths to read from.

pub fn set_paths(self, input: Option<Vec<String>>) -> Self

A list of the Amazon S3 paths to read from.

pub fn get_paths(&self) -> &Option<Vec<String>>

A list of the Amazon S3 paths to read from.

pub fn compression_type(self, input: CompressionType) -> Self

Specifies how the data is compressed. This is generally not necessary if the data has a standard file extension. Possible values are "gzip" and "bzip").

pub fn set_compression_type(self, input: Option<CompressionType>) -> Self

Specifies how the data is compressed. This is generally not necessary if the data has a standard file extension. Possible values are "gzip" and "bzip").

pub fn get_compression_type(&self) -> &Option<CompressionType>

Specifies how the data is compressed. This is generally not necessary if the data has a standard file extension. Possible values are "gzip" and "bzip").

pub fn exclusions(self, input: impl Into<String>) -> Self

Appends an item to exclusions.

To override the contents of this collection use set_exclusions.

A string containing a JSON list of Unix-style glob patterns to exclude. For example, "[\"**.pdf\"]" excludes all PDF files.

pub fn set_exclusions(self, input: Option<Vec<String>>) -> Self

A string containing a JSON list of Unix-style glob patterns to exclude. For example, "[\"**.pdf\"]" excludes all PDF files.

pub fn get_exclusions(&self) -> &Option<Vec<String>>

A string containing a JSON list of Unix-style glob patterns to exclude. For example, "[\"**.pdf\"]" excludes all PDF files.

pub fn group_size(self, input: impl Into<String>) -> Self

The target group size in bytes. The default is computed based on the input data size and the size of your cluster. When there are fewer than 50,000 input files, "groupFiles" must be set to "inPartition" for this to take effect.

pub fn set_group_size(self, input: Option<String>) -> Self

The target group size in bytes. The default is computed based on the input data size and the size of your cluster. When there are fewer than 50,000 input files, "groupFiles" must be set to "inPartition" for this to take effect.

pub fn get_group_size(&self) -> &Option<String>

The target group size in bytes. The default is computed based on the input data size and the size of your cluster. When there are fewer than 50,000 input files, "groupFiles" must be set to "inPartition" for this to take effect.

pub fn group_files(self, input: impl Into<String>) -> Self

Grouping files is turned on by default when the input contains more than 50,000 files. To turn on grouping with fewer than 50,000 files, set this parameter to "inPartition". To disable grouping when there are more than 50,000 files, set this parameter to "none".

pub fn set_group_files(self, input: Option<String>) -> Self

Grouping files is turned on by default when the input contains more than 50,000 files. To turn on grouping with fewer than 50,000 files, set this parameter to "inPartition". To disable grouping when there are more than 50,000 files, set this parameter to "none".

pub fn get_group_files(&self) -> &Option<String>

Grouping files is turned on by default when the input contains more than 50,000 files. To turn on grouping with fewer than 50,000 files, set this parameter to "inPartition". To disable grouping when there are more than 50,000 files, set this parameter to "none".

pub fn recurse(self, input: bool) -> Self

If set to true, recursively reads files in all subdirectories under the specified paths.

pub fn set_recurse(self, input: Option<bool>) -> Self

If set to true, recursively reads files in all subdirectories under the specified paths.

pub fn get_recurse(&self) -> &Option<bool>

If set to true, recursively reads files in all subdirectories under the specified paths.

pub fn max_band(self, input: i32) -> Self

This option controls the duration in milliseconds after which the s3 listing is likely to be consistent. Files with modification timestamps falling within the last maxBand milliseconds are tracked specially when using JobBookmarks to account for Amazon S3 eventual consistency. Most users don't need to set this option. The default is 900000 milliseconds, or 15 minutes.

pub fn set_max_band(self, input: Option<i32>) -> Self

This option controls the duration in milliseconds after which the s3 listing is likely to be consistent. Files with modification timestamps falling within the last maxBand milliseconds are tracked specially when using JobBookmarks to account for Amazon S3 eventual consistency. Most users don't need to set this option. The default is 900000 milliseconds, or 15 minutes.

pub fn get_max_band(&self) -> &Option<i32>

This option controls the duration in milliseconds after which the s3 listing is likely to be consistent. Files with modification timestamps falling within the last maxBand milliseconds are tracked specially when using JobBookmarks to account for Amazon S3 eventual consistency. Most users don't need to set this option. The default is 900000 milliseconds, or 15 minutes.

pub fn max_files_in_band(self, input: i32) -> Self

This option specifies the maximum number of files to save from the last maxBand seconds. If this number is exceeded, extra files are skipped and only processed in the next job run.

pub fn set_max_files_in_band(self, input: Option<i32>) -> Self

This option specifies the maximum number of files to save from the last maxBand seconds. If this number is exceeded, extra files are skipped and only processed in the next job run.

pub fn get_max_files_in_band(&self) -> &Option<i32>

This option specifies the maximum number of files to save from the last maxBand seconds. If this number is exceeded, extra files are skipped and only processed in the next job run.

pub fn additional_options(self, input: S3DirectSourceAdditionalOptions) -> Self

Specifies additional connection options.

pub fn set_additional_options( self, input: Option<S3DirectSourceAdditionalOptions> ) -> Self

Specifies additional connection options.

pub fn get_additional_options(&self) -> &Option<S3DirectSourceAdditionalOptions>

Specifies additional connection options.

pub fn separator(self, input: Separator) -> Self

Specifies the delimiter character. The default is a comma: ",", but any other character can be specified.

This field is required.

pub fn set_separator(self, input: Option<Separator>) -> Self

Specifies the delimiter character. The default is a comma: ",", but any other character can be specified.

pub fn get_separator(&self) -> &Option<Separator>

Specifies the delimiter character. The default is a comma: ",", but any other character can be specified.

pub fn escaper(self, input: impl Into<String>) -> Self

Specifies a character to use for escaping. This option is used only when reading CSV files. The default value is none. If enabled, the character which immediately follows is used as-is, except for a small set of well-known escapes (\n, \r, \t, and \0).

pub fn set_escaper(self, input: Option<String>) -> Self

Specifies a character to use for escaping. This option is used only when reading CSV files. The default value is none. If enabled, the character which immediately follows is used as-is, except for a small set of well-known escapes (\n, \r, \t, and \0).

pub fn get_escaper(&self) -> &Option<String>

Specifies a character to use for escaping. This option is used only when reading CSV files. The default value is none. If enabled, the character which immediately follows is used as-is, except for a small set of well-known escapes (\n, \r, \t, and \0).

pub fn quote_char(self, input: QuoteChar) -> Self

Specifies the character to use for quoting. The default is a double quote: '"'. Set this to -1 to turn off quoting entirely.

This field is required.

pub fn set_quote_char(self, input: Option<QuoteChar>) -> Self

Specifies the character to use for quoting. The default is a double quote: '"'. Set this to -1 to turn off quoting entirely.

pub fn get_quote_char(&self) -> &Option<QuoteChar>

Specifies the character to use for quoting. The default is a double quote: '"'. Set this to -1 to turn off quoting entirely.

pub fn multiline(self, input: bool) -> Self

A Boolean value that specifies whether a single record can span multiple lines. This can occur when a field contains a quoted new-line character. You must set this option to True if any record spans multiple lines. The default value is False, which allows for more aggressive file-splitting during parsing.

pub fn set_multiline(self, input: Option<bool>) -> Self

A Boolean value that specifies whether a single record can span multiple lines. This can occur when a field contains a quoted new-line character. You must set this option to True if any record spans multiple lines. The default value is False, which allows for more aggressive file-splitting during parsing.

pub fn get_multiline(&self) -> &Option<bool>

A Boolean value that specifies whether a single record can span multiple lines. This can occur when a field contains a quoted new-line character. You must set this option to True if any record spans multiple lines. The default value is False, which allows for more aggressive file-splitting during parsing.

pub fn with_header(self, input: bool) -> Self

A Boolean value that specifies whether to treat the first line as a header. The default value is False.

pub fn set_with_header(self, input: Option<bool>) -> Self

A Boolean value that specifies whether to treat the first line as a header. The default value is False.

pub fn get_with_header(&self) -> &Option<bool>

A Boolean value that specifies whether to treat the first line as a header. The default value is False.

pub fn write_header(self, input: bool) -> Self

A Boolean value that specifies whether to write the header to output. The default value is True.

pub fn set_write_header(self, input: Option<bool>) -> Self

A Boolean value that specifies whether to write the header to output. The default value is True.

pub fn get_write_header(&self) -> &Option<bool>

A Boolean value that specifies whether to write the header to output. The default value is True.

pub fn skip_first(self, input: bool) -> Self

A Boolean value that specifies whether to skip the first data line. The default value is False.

pub fn set_skip_first(self, input: Option<bool>) -> Self

A Boolean value that specifies whether to skip the first data line. The default value is False.

pub fn get_skip_first(&self) -> &Option<bool>

A Boolean value that specifies whether to skip the first data line. The default value is False.

pub fn optimize_performance(self, input: bool) -> Self

A Boolean value that specifies whether to use the advanced SIMD CSV reader along with Apache Arrow based columnar memory formats. Only available in Glue version 3.0.

pub fn set_optimize_performance(self, input: Option<bool>) -> Self

A Boolean value that specifies whether to use the advanced SIMD CSV reader along with Apache Arrow based columnar memory formats. Only available in Glue version 3.0.

pub fn get_optimize_performance(&self) -> &Option<bool>

A Boolean value that specifies whether to use the advanced SIMD CSV reader along with Apache Arrow based columnar memory formats. Only available in Glue version 3.0.

pub fn output_schemas(self, input: GlueSchema) -> Self

Appends an item to output_schemas.

To override the contents of this collection use set_output_schemas.

Specifies the data schema for the S3 CSV source.

pub fn set_output_schemas(self, input: Option<Vec<GlueSchema>>) -> Self

Specifies the data schema for the S3 CSV source.

pub fn get_output_schemas(&self) -> &Option<Vec<GlueSchema>>

Specifies the data schema for the S3 CSV source.

pub fn build(self) -> Result<S3CsvSource, BuildError>

Consumes the builder and constructs a S3CsvSource. This method will fail if any of the following fields are not set:

• name
• paths
• separator
• quote_char

Trait Implementations

impl Clone for S3CsvSourceBuilder

fn clone(&self) -> S3CsvSourceBuilder

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for S3CsvSourceBuilder

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for S3CsvSourceBuilder

fn default() -> S3CsvSourceBuilder

Returns the “default value” for a type.

impl PartialEq for S3CsvSourceBuilder

fn eq(&self, other: &S3CsvSourceBuilder) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for S3CsvSourceBuilder