Struct ProcessingS3Input

#[non_exhaustive]
pub struct ProcessingS3Input {
    pub s3_uri: Option<String>,
    pub local_path: Option<String>,
    pub s3_data_type: Option<ProcessingS3DataType>,
    pub s3_input_mode: Option<ProcessingS3InputMode>,
    pub s3_data_distribution_type: Option<ProcessingS3DataDistributionType>,
    pub s3_compression_type: Option<ProcessingS3CompressionType>,
}

Configuration for downloading input data from Amazon S3 into the processing container.

Fields (Non-exhaustive)

Non-exhaustive structs could have additional fields added in future. Therefore, non-exhaustive structs cannot be constructed in external crates using the traditional Struct { .. } syntax; cannot be matched against without a wildcard ..; and struct update syntax will not work.

s3_uri: Option<String>

The URI of the Amazon S3 prefix Amazon SageMaker downloads data required to run a processing job.

local_path: Option<String>

The local path in your container where you want Amazon SageMaker to write input data to. LocalPath is an absolute path to the input data and must begin with /opt/ml/processing/. LocalPath is a required parameter when AppManaged is False (default).

s3_data_type: Option<ProcessingS3DataType>

Whether you use an S3Prefix or a ManifestFile for the data type. If you choose S3Prefix, S3Uri identifies a key name prefix. Amazon SageMaker uses all objects with the specified key name prefix for the processing job. If you choose ManifestFile, S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for the processing job.

s3_input_mode: Option<ProcessingS3InputMode>

Whether to use File or Pipe input mode. In File mode, Amazon SageMaker copies the data from the input source onto the local ML storage volume before starting your processing container. This is the most commonly used input mode. In Pipe mode, Amazon SageMaker streams input data from the source directly to your processing container into named pipes without using the ML storage volume.

s3_data_distribution_type: Option<ProcessingS3DataDistributionType>

Whether to distribute the data from Amazon S3 to all processing instances with FullyReplicated, or whether the data from Amazon S3 is shared by Amazon S3 key, downloading one shard of data to each processing instance.

s3_compression_type: Option<ProcessingS3CompressionType>

Whether to GZIP-decompress the data in Amazon S3 as it is streamed into the processing container. Gzip can only be used when Pipe mode is specified as the S3InputMode. In Pipe mode, Amazon SageMaker streams input data from the source directly to your container without using the EBS volume.

Implementations

impl ProcessingS3Input

pub fn s3_uri(&self) -> Option<&str>

The URI of the Amazon S3 prefix Amazon SageMaker downloads data required to run a processing job.

pub fn local_path(&self) -> Option<&str>

The local path in your container where you want Amazon SageMaker to write input data to. LocalPath is an absolute path to the input data and must begin with /opt/ml/processing/. LocalPath is a required parameter when AppManaged is False (default).

pub fn s3_data_type(&self) -> Option<&ProcessingS3DataType>

Whether you use an S3Prefix or a ManifestFile for the data type. If you choose S3Prefix, S3Uri identifies a key name prefix. Amazon SageMaker uses all objects with the specified key name prefix for the processing job. If you choose ManifestFile, S3Uri identifies an object that is a manifest file containing a list of object keys that you want Amazon SageMaker to use for the processing job.

pub fn s3_input_mode(&self) -> Option<&ProcessingS3InputMode>

Whether to use File or Pipe input mode. In File mode, Amazon SageMaker copies the data from the input source onto the local ML storage volume before starting your processing container. This is the most commonly used input mode. In Pipe mode, Amazon SageMaker streams input data from the source directly to your processing container into named pipes without using the ML storage volume.

pub fn s3_data_distribution_type( &self, ) -> Option<&ProcessingS3DataDistributionType>

Whether to distribute the data from Amazon S3 to all processing instances with FullyReplicated, or whether the data from Amazon S3 is shared by Amazon S3 key, downloading one shard of data to each processing instance.

pub fn s3_compression_type(&self) -> Option<&ProcessingS3CompressionType>

Whether to GZIP-decompress the data in Amazon S3 as it is streamed into the processing container. Gzip can only be used when Pipe mode is specified as the S3InputMode. In Pipe mode, Amazon SageMaker streams input data from the source directly to your container without using the EBS volume.

impl ProcessingS3Input

pub fn builder() -> ProcessingS3InputBuilder

Creates a new builder-style object to manufacture ProcessingS3Input.

Trait Implementations

impl Clone for ProcessingS3Input

fn clone(&self) -> ProcessingS3Input

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ProcessingS3Input

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for ProcessingS3Input

fn eq(&self, other: &ProcessingS3Input) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for ProcessingS3Input