Struct rusoto_sagemaker::Channel
source · [−]pub struct Channel {
pub channel_name: String,
pub compression_type: Option<String>,
pub content_type: Option<String>,
pub data_source: DataSource,
pub input_mode: Option<String>,
pub record_wrapper_type: Option<String>,
pub shuffle_config: Option<ShuffleConfig>,
}
Expand description
A channel is a named input source that training algorithms can consume.
Fields
channel_name: String
The name of the channel.
compression_type: Option<String>
If training data is compressed, the compression type. The default value is None
. CompressionType
is used only in Pipe input mode. In File mode, leave this field unset or set it to None.
content_type: Option<String>
The MIME type of the data.
data_source: DataSource
The location of the channel data.
input_mode: Option<String>
(Optional) The input mode to use for the data channel in a training job. If you don't set a value for InputMode
, Amazon SageMaker uses the value set for TrainingInputMode
. Use this parameter to override the TrainingInputMode
setting in a AlgorithmSpecification request when you have a channel that needs a different input mode from the training job's general setting. To download the data from Amazon Simple Storage Service (Amazon S3) to the provisioned ML storage volume, and mount the directory to a Docker volume, use File
input mode. To stream data directly from Amazon S3 to the container, choose Pipe
input mode.
To use a model for incremental training, choose File
input model.
record_wrapper_type: Option<String>
Specify RecordIO as the value when input data is in raw format but the training algorithm requires the RecordIO format. In this case, Amazon SageMaker wraps each individual S3 object in a RecordIO record. If the input data is already in RecordIO format, you don't need to set this attribute. For more information, see Create a Dataset Using RecordIO.
In File mode, leave this field unset or set it to None.
shuffle_config: Option<ShuffleConfig>
A configuration for a shuffle option for input data in a channel. If you use S3Prefix
for S3DataType
, this shuffles the results of the S3 key prefix matches. If you use ManifestFile
, the order of the S3 object references in the ManifestFile
is shuffled. If you use AugmentedManifestFile
, the order of the JSON lines in the AugmentedManifestFile
is shuffled. The shuffling order is determined using the Seed
value.
For Pipe input mode, shuffling is done at the start of every epoch. With large datasets this ensures that the order of the training data is different for each epoch, it helps reduce bias and possible overfitting. In a multi-node training job when ShuffleConfig is combined with S3DataDistributionType
of ShardedByS3Key
, the data is shuffled across nodes so that the content sent to a particular node on the first epoch might be sent to a different node on the second epoch.
Trait Implementations
sourceimpl<'de> Deserialize<'de> for Channel
impl<'de> Deserialize<'de> for Channel
sourcefn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error> where
__D: Deserializer<'de>,
fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error> where
__D: Deserializer<'de>,
Deserialize this value from the given Serde deserializer. Read more
impl StructuralPartialEq for Channel
Auto Trait Implementations
impl RefUnwindSafe for Channel
impl Send for Channel
impl Sync for Channel
impl Unpin for Channel
impl UnwindSafe for Channel
Blanket Implementations
sourceimpl<T> BorrowMut<T> for T where
T: ?Sized,
impl<T> BorrowMut<T> for T where
T: ?Sized,
const: unstable · sourcefn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
sourceimpl<T> Instrument for T
impl<T> Instrument for T
sourcefn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
sourcefn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
sourceimpl<T> ToOwned for T where
T: Clone,
impl<T> ToOwned for T where
T: Clone,
type Owned = T
type Owned = T
The resulting type after obtaining ownership.
sourcefn clone_into(&self, target: &mut T)
fn clone_into(&self, target: &mut T)
toowned_clone_into
)Uses borrowed data to replace owned data, usually by cloning. Read more
sourceimpl<T> WithSubscriber for T
impl<T> WithSubscriber for T
sourcefn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self> where
S: Into<Dispatch>,
fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self> where
S: Into<Dispatch>,
Attaches the provided Subscriber
to this type, returning a
WithDispatch
wrapper. Read more
sourcefn with_current_subscriber(self) -> WithDispatch<Self>
fn with_current_subscriber(self) -> WithDispatch<Self>
Attaches the current default Subscriber
to this type, returning a
WithDispatch
wrapper. Read more