Struct aws_sdk_glue::types::builders::S3JsonSourceBuilder
source · #[non_exhaustive]pub struct S3JsonSourceBuilder { /* private fields */ }
Expand description
A builder for S3JsonSource
.
Implementations§
source§impl S3JsonSourceBuilder
impl S3JsonSourceBuilder
sourcepub fn name(self, input: impl Into<String>) -> Self
pub fn name(self, input: impl Into<String>) -> Self
The name of the data store.
This field is required.sourcepub fn paths(self, input: impl Into<String>) -> Self
pub fn paths(self, input: impl Into<String>) -> Self
Appends an item to paths
.
To override the contents of this collection use set_paths
.
A list of the Amazon S3 paths to read from.
sourcepub fn set_paths(self, input: Option<Vec<String>>) -> Self
pub fn set_paths(self, input: Option<Vec<String>>) -> Self
A list of the Amazon S3 paths to read from.
sourcepub fn compression_type(self, input: CompressionType) -> Self
pub fn compression_type(self, input: CompressionType) -> Self
Specifies how the data is compressed. This is generally not necessary if the data has a standard file extension. Possible values are "gzip"
and "bzip"
).
sourcepub fn set_compression_type(self, input: Option<CompressionType>) -> Self
pub fn set_compression_type(self, input: Option<CompressionType>) -> Self
Specifies how the data is compressed. This is generally not necessary if the data has a standard file extension. Possible values are "gzip"
and "bzip"
).
sourcepub fn get_compression_type(&self) -> &Option<CompressionType>
pub fn get_compression_type(&self) -> &Option<CompressionType>
Specifies how the data is compressed. This is generally not necessary if the data has a standard file extension. Possible values are "gzip"
and "bzip"
).
sourcepub fn exclusions(self, input: impl Into<String>) -> Self
pub fn exclusions(self, input: impl Into<String>) -> Self
Appends an item to exclusions
.
To override the contents of this collection use set_exclusions
.
A string containing a JSON list of Unix-style glob patterns to exclude. For example, "\[\"**.pdf\"\]" excludes all PDF files.
sourcepub fn set_exclusions(self, input: Option<Vec<String>>) -> Self
pub fn set_exclusions(self, input: Option<Vec<String>>) -> Self
A string containing a JSON list of Unix-style glob patterns to exclude. For example, "\[\"**.pdf\"\]" excludes all PDF files.
sourcepub fn get_exclusions(&self) -> &Option<Vec<String>>
pub fn get_exclusions(&self) -> &Option<Vec<String>>
A string containing a JSON list of Unix-style glob patterns to exclude. For example, "\[\"**.pdf\"\]" excludes all PDF files.
sourcepub fn group_size(self, input: impl Into<String>) -> Self
pub fn group_size(self, input: impl Into<String>) -> Self
The target group size in bytes. The default is computed based on the input data size and the size of your cluster. When there are fewer than 50,000 input files, "groupFiles"
must be set to "inPartition"
for this to take effect.
sourcepub fn set_group_size(self, input: Option<String>) -> Self
pub fn set_group_size(self, input: Option<String>) -> Self
The target group size in bytes. The default is computed based on the input data size and the size of your cluster. When there are fewer than 50,000 input files, "groupFiles"
must be set to "inPartition"
for this to take effect.
sourcepub fn get_group_size(&self) -> &Option<String>
pub fn get_group_size(&self) -> &Option<String>
The target group size in bytes. The default is computed based on the input data size and the size of your cluster. When there are fewer than 50,000 input files, "groupFiles"
must be set to "inPartition"
for this to take effect.
sourcepub fn group_files(self, input: impl Into<String>) -> Self
pub fn group_files(self, input: impl Into<String>) -> Self
Grouping files is turned on by default when the input contains more than 50,000 files. To turn on grouping with fewer than 50,000 files, set this parameter to "inPartition". To disable grouping when there are more than 50,000 files, set this parameter to "none"
.
sourcepub fn set_group_files(self, input: Option<String>) -> Self
pub fn set_group_files(self, input: Option<String>) -> Self
Grouping files is turned on by default when the input contains more than 50,000 files. To turn on grouping with fewer than 50,000 files, set this parameter to "inPartition". To disable grouping when there are more than 50,000 files, set this parameter to "none"
.
sourcepub fn get_group_files(&self) -> &Option<String>
pub fn get_group_files(&self) -> &Option<String>
Grouping files is turned on by default when the input contains more than 50,000 files. To turn on grouping with fewer than 50,000 files, set this parameter to "inPartition". To disable grouping when there are more than 50,000 files, set this parameter to "none"
.
sourcepub fn recurse(self, input: bool) -> Self
pub fn recurse(self, input: bool) -> Self
If set to true, recursively reads files in all subdirectories under the specified paths.
sourcepub fn set_recurse(self, input: Option<bool>) -> Self
pub fn set_recurse(self, input: Option<bool>) -> Self
If set to true, recursively reads files in all subdirectories under the specified paths.
sourcepub fn get_recurse(&self) -> &Option<bool>
pub fn get_recurse(&self) -> &Option<bool>
If set to true, recursively reads files in all subdirectories under the specified paths.
sourcepub fn max_band(self, input: i32) -> Self
pub fn max_band(self, input: i32) -> Self
This option controls the duration in milliseconds after which the s3 listing is likely to be consistent. Files with modification timestamps falling within the last maxBand milliseconds are tracked specially when using JobBookmarks to account for Amazon S3 eventual consistency. Most users don't need to set this option. The default is 900000 milliseconds, or 15 minutes.
sourcepub fn set_max_band(self, input: Option<i32>) -> Self
pub fn set_max_band(self, input: Option<i32>) -> Self
This option controls the duration in milliseconds after which the s3 listing is likely to be consistent. Files with modification timestamps falling within the last maxBand milliseconds are tracked specially when using JobBookmarks to account for Amazon S3 eventual consistency. Most users don't need to set this option. The default is 900000 milliseconds, or 15 minutes.
sourcepub fn get_max_band(&self) -> &Option<i32>
pub fn get_max_band(&self) -> &Option<i32>
This option controls the duration in milliseconds after which the s3 listing is likely to be consistent. Files with modification timestamps falling within the last maxBand milliseconds are tracked specially when using JobBookmarks to account for Amazon S3 eventual consistency. Most users don't need to set this option. The default is 900000 milliseconds, or 15 minutes.
sourcepub fn max_files_in_band(self, input: i32) -> Self
pub fn max_files_in_band(self, input: i32) -> Self
This option specifies the maximum number of files to save from the last maxBand seconds. If this number is exceeded, extra files are skipped and only processed in the next job run.
sourcepub fn set_max_files_in_band(self, input: Option<i32>) -> Self
pub fn set_max_files_in_band(self, input: Option<i32>) -> Self
This option specifies the maximum number of files to save from the last maxBand seconds. If this number is exceeded, extra files are skipped and only processed in the next job run.
sourcepub fn get_max_files_in_band(&self) -> &Option<i32>
pub fn get_max_files_in_band(&self) -> &Option<i32>
This option specifies the maximum number of files to save from the last maxBand seconds. If this number is exceeded, extra files are skipped and only processed in the next job run.
sourcepub fn additional_options(self, input: S3DirectSourceAdditionalOptions) -> Self
pub fn additional_options(self, input: S3DirectSourceAdditionalOptions) -> Self
Specifies additional connection options.
sourcepub fn set_additional_options(
self,
input: Option<S3DirectSourceAdditionalOptions>,
) -> Self
pub fn set_additional_options( self, input: Option<S3DirectSourceAdditionalOptions>, ) -> Self
Specifies additional connection options.
sourcepub fn get_additional_options(&self) -> &Option<S3DirectSourceAdditionalOptions>
pub fn get_additional_options(&self) -> &Option<S3DirectSourceAdditionalOptions>
Specifies additional connection options.
sourcepub fn json_path(self, input: impl Into<String>) -> Self
pub fn json_path(self, input: impl Into<String>) -> Self
A JsonPath string defining the JSON data.
sourcepub fn set_json_path(self, input: Option<String>) -> Self
pub fn set_json_path(self, input: Option<String>) -> Self
A JsonPath string defining the JSON data.
sourcepub fn get_json_path(&self) -> &Option<String>
pub fn get_json_path(&self) -> &Option<String>
A JsonPath string defining the JSON data.
sourcepub fn multiline(self, input: bool) -> Self
pub fn multiline(self, input: bool) -> Self
A Boolean value that specifies whether a single record can span multiple lines. This can occur when a field contains a quoted new-line character. You must set this option to True if any record spans multiple lines. The default value is False
, which allows for more aggressive file-splitting during parsing.
sourcepub fn set_multiline(self, input: Option<bool>) -> Self
pub fn set_multiline(self, input: Option<bool>) -> Self
A Boolean value that specifies whether a single record can span multiple lines. This can occur when a field contains a quoted new-line character. You must set this option to True if any record spans multiple lines. The default value is False
, which allows for more aggressive file-splitting during parsing.
sourcepub fn get_multiline(&self) -> &Option<bool>
pub fn get_multiline(&self) -> &Option<bool>
A Boolean value that specifies whether a single record can span multiple lines. This can occur when a field contains a quoted new-line character. You must set this option to True if any record spans multiple lines. The default value is False
, which allows for more aggressive file-splitting during parsing.
sourcepub fn output_schemas(self, input: GlueSchema) -> Self
pub fn output_schemas(self, input: GlueSchema) -> Self
Appends an item to output_schemas
.
To override the contents of this collection use set_output_schemas
.
Specifies the data schema for the S3 JSON source.
sourcepub fn set_output_schemas(self, input: Option<Vec<GlueSchema>>) -> Self
pub fn set_output_schemas(self, input: Option<Vec<GlueSchema>>) -> Self
Specifies the data schema for the S3 JSON source.
sourcepub fn get_output_schemas(&self) -> &Option<Vec<GlueSchema>>
pub fn get_output_schemas(&self) -> &Option<Vec<GlueSchema>>
Specifies the data schema for the S3 JSON source.
sourcepub fn build(self) -> Result<S3JsonSource, BuildError>
pub fn build(self) -> Result<S3JsonSource, BuildError>
Consumes the builder and constructs a S3JsonSource
.
This method will fail if any of the following fields are not set:
Trait Implementations§
source§impl Clone for S3JsonSourceBuilder
impl Clone for S3JsonSourceBuilder
source§fn clone(&self) -> S3JsonSourceBuilder
fn clone(&self) -> S3JsonSourceBuilder
1.0.0 · source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source
. Read moresource§impl Debug for S3JsonSourceBuilder
impl Debug for S3JsonSourceBuilder
source§impl Default for S3JsonSourceBuilder
impl Default for S3JsonSourceBuilder
source§fn default() -> S3JsonSourceBuilder
fn default() -> S3JsonSourceBuilder
source§impl PartialEq for S3JsonSourceBuilder
impl PartialEq for S3JsonSourceBuilder
source§fn eq(&self, other: &S3JsonSourceBuilder) -> bool
fn eq(&self, other: &S3JsonSourceBuilder) -> bool
self
and other
values to be equal, and is used
by ==
.impl StructuralPartialEq for S3JsonSourceBuilder
Auto Trait Implementations§
impl Freeze for S3JsonSourceBuilder
impl RefUnwindSafe for S3JsonSourceBuilder
impl Send for S3JsonSourceBuilder
impl Sync for S3JsonSourceBuilder
impl Unpin for S3JsonSourceBuilder
impl UnwindSafe for S3JsonSourceBuilder
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
source§impl<T> Instrument for T
impl<T> Instrument for T
source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
source§impl<T> IntoEither for T
impl<T> IntoEither for T
source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left
is true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moresource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left(&self)
returns true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read more