Struct Client

pub struct Client { /* private fields */ }

Client for Amazon SageMaker Runtime

Client for invoking operations on Amazon SageMaker Runtime. Each operation on Amazon SageMaker Runtime is a method on this this struct. .send() MUST be invoked on the generated operations to dispatch the request to the service.

Constructing a Client

A Config is required to construct a client. For most use cases, the aws-config crate should be used to automatically resolve this config using aws_config::load_from_env(), since this will resolve an SdkConfig which can be shared across multiple different AWS SDK clients. This config resolution process can be customized by calling aws_config::from_env() instead, which returns a ConfigLoader that uses the builder pattern to customize the default config.

In the simplest case, creating a client looks as follows:

let config = aws_config::load_from_env().await;
let client = aws_sdk_sagemakerruntime::Client::new(&config);

Occasionally, SDKs may have additional service-specific values that can be set on the Config that is absent from SdkConfig, or slightly different settings for a specific client may be desired. The Builder struct implements From<&SdkConfig>, so setting these specific settings can be done as follows:

let sdk_config = ::aws_config::load_from_env().await;
let config = aws_sdk_sagemakerruntime::config::Builder::from(&sdk_config)
    .some_service_specific_setting("value")
    .build();

See the aws-config docs and Config for more information on customizing configuration.

Note: Client construction is expensive due to connection thread pool initialization, and should be done once at application start-up.

Using the Client

A client has a function for every operation that can be performed by the service. For example, the InvokeEndpoint operation has a Client::invoke_endpoint, function which returns a builder for that operation. The fluent builder ultimately has a send() function that returns an async future that returns a result, as illustrated below:

let result = client.invoke_endpoint()
    .endpoint_name("example")
    .send()
    .await;

The underlying HTTP requests that get made by this can be modified with the customize_operation function on the fluent builder. See the customize module for more information.

Implementations

impl Client

pub fn invoke_endpoint(&self) -> InvokeEndpointFluentBuilder

Constructs a fluent builder for the InvokeEndpoint operation.

• The fluent builder is configurable:
  • endpoint_name(impl Into<String>) / set_endpoint_name(Option<String>):
    required: true
    

    The name of the endpoint that you specified when you created the endpoint using the CreateEndpoint API.

    
    
  • body(Blob) / set_body(Option<Blob>):
    required: true
    

    Provides input data, in the format specified in the ContentType request header. Amazon SageMaker passes all of the data in the body to the model.

    For information about the format of the request body, see Common Data Formats-Inference.

    
    
  • content_type(impl Into<String>) / set_content_type(Option<String>):
    required: false
    

    The MIME type of the input data in the request body.

    
    
  • accept(impl Into<String>) / set_accept(Option<String>):
    required: false
    

    The desired MIME type of the inference response from the model container.

    
    
  • custom_attributes(impl Into<String>) / set_custom_attributes(Option<String>):
    required: false
    

    Provides additional information about a request for an inference submitted to a model hosted at an Amazon SageMaker endpoint. The information is an opaque value that is forwarded verbatim. You could use this value, for example, to provide an ID that you can use to track a request or to provide other metadata that a service endpoint was programmed to process. The value must consist of no more than 1024 visible US-ASCII characters as specified in Section 3.3.6. Field Value Components of the Hypertext Transfer Protocol (HTTP/1.1).

    The code in your model is responsible for setting or updating any custom attributes in the response. If your code does not set this value in the response, an empty value is returned. For example, if a custom attribute represents the trace ID, your model can prepend the custom attribute with Trace ID: in your post-processing function.

    This feature is currently supported in the Amazon Web Services SDKs but not in the Amazon SageMaker Python SDK.

    
    
  • target_model(impl Into<String>) / set_target_model(Option<String>):
    required: false
    

    The model to request for inference when invoking a multi-model endpoint.

    
    
  • target_variant(impl Into<String>) / set_target_variant(Option<String>):
    required: false
    

    Specify the production variant to send the inference request to when invoking an endpoint that is running two or more variants. Note that this parameter overrides the default behavior for the endpoint, which is to distribute the invocation traffic based on the variant weights.

    For information about how to use variant targeting to perform a/b testing, see Test models in production

    
    
  • target_container_hostname(impl Into<String>) / set_target_container_hostname(Option<String>):
    required: false
    

    If the endpoint hosts multiple containers and is configured to use direct invocation, this parameter specifies the host name of the container to invoke.

    
    
  • inference_id(impl Into<String>) / set_inference_id(Option<String>):
    required: false
    

    If you provide a value, it is added to the captured data when you enable data capture on the endpoint. For information about data capture, see Capture Data.

    
    
  • enable_explanations(impl Into<String>) / set_enable_explanations(Option<String>):
    required: false
    

    An optional JMESPath expression used to override the EnableExplanations parameter of the ClarifyExplainerConfig API. See the EnableExplanations section in the developer guide for more information.

    
    
  • inference_component_name(impl Into<String>) / set_inference_component_name(Option<String>):
    required: false
    

    If the endpoint hosts one or more inference components, this parameter specifies the name of inference component to invoke.

    
    
  • session_id(impl Into<String>) / set_session_id(Option<String>):
    required: false
    

    Creates a stateful session or identifies an existing one. You can do one of the following:

    • Create a stateful session by specifying the value NEW_SESSION.

    • Send your request to an existing stateful session by specifying the ID of that session.

    With a stateful session, you can send multiple requests to a stateful model. When you create a session with a stateful model, the model must create the session ID and set the expiration time. The model must also provide that information in the response to your request. You can get the ID and timestamp from the NewSessionId response parameter. For any subsequent request where you specify that session ID, SageMaker routes the request to the same instance that supports the session.

    
    
• On success, responds with InvokeEndpointOutput with field(s):
  • body(Option<Blob>):

    Includes the inference provided by the model.

    For information about the format of the response body, see Common Data Formats-Inference.

    If the explainer is activated, the body includes the explanations provided by the model. For more information, see the Response section under Invoke the Endpoint in the Developer Guide.

  • content_type(Option<String>):

    The MIME type of the inference returned from the model container.

  • invoked_production_variant(Option<String>):

    Identifies the production variant that was invoked.

  • custom_attributes(Option<String>):

    Provides additional information in the response about the inference returned by a model hosted at an Amazon SageMaker endpoint. The information is an opaque value that is forwarded verbatim. You could use this value, for example, to return an ID received in the CustomAttributes header of a request or other metadata that a service endpoint was programmed to produce. The value must consist of no more than 1024 visible US-ASCII characters as specified in Section 3.3.6. Field Value Components of the Hypertext Transfer Protocol (HTTP/1.1). If the customer wants the custom attribute returned, the model must set the custom attribute to be included on the way back.

    The code in your model is responsible for setting or updating any custom attributes in the response. If your code does not set this value in the response, an empty value is returned. For example, if a custom attribute represents the trace ID, your model can prepend the custom attribute with Trace ID: in your post-processing function.

    This feature is currently supported in the Amazon Web Services SDKs but not in the Amazon SageMaker Python SDK.

  • new_session_id(Option<String>):

    If you created a stateful session with your request, the ID and expiration time that the model assigns to that session.

  • closed_session_id(Option<String>):

    If you closed a stateful session with your request, the ID of that session.

• On failure, responds with SdkError<InvokeEndpointError>

impl Client

pub fn invoke_endpoint_async(&self) -> InvokeEndpointAsyncFluentBuilder

Constructs a fluent builder for the InvokeEndpointAsync operation.

• The fluent builder is configurable:
  • endpoint_name(impl Into<String>) / set_endpoint_name(Option<String>):
    required: true
    

    The name of the endpoint that you specified when you created the endpoint using the CreateEndpoint API.

    
    
  • content_type(impl Into<String>) / set_content_type(Option<String>):
    required: false
    

    The MIME type of the input data in the request body.

    
    
  • accept(impl Into<String>) / set_accept(Option<String>):
    required: false
    

    The desired MIME type of the inference response from the model container.

    
    
  • custom_attributes(impl Into<String>) / set_custom_attributes(Option<String>):
    required: false
    

    Provides additional information about a request for an inference submitted to a model hosted at an Amazon SageMaker endpoint. The information is an opaque value that is forwarded verbatim. You could use this value, for example, to provide an ID that you can use to track a request or to provide other metadata that a service endpoint was programmed to process. The value must consist of no more than 1024 visible US-ASCII characters as specified in Section 3.3.6. Field Value Components of the Hypertext Transfer Protocol (HTTP/1.1).

    The code in your model is responsible for setting or updating any custom attributes in the response. If your code does not set this value in the response, an empty value is returned. For example, if a custom attribute represents the trace ID, your model can prepend the custom attribute with Trace ID: in your post-processing function.

    This feature is currently supported in the Amazon Web Services SDKs but not in the Amazon SageMaker Python SDK.

    
    
  • inference_id(impl Into<String>) / set_inference_id(Option<String>):
    required: false
    

    The identifier for the inference request. Amazon SageMaker will generate an identifier for you if none is specified.

    
    
  • input_location(impl Into<String>) / set_input_location(Option<String>):
    required: true
    

    The Amazon S3 URI where the inference request payload is stored.

    
    
  • request_ttl_seconds(i32) / set_request_ttl_seconds(Option<i32>):
    required: false
    

    Maximum age in seconds a request can be in the queue before it is marked as expired. The default is 6 hours, or 21,600 seconds.

    
    
  • invocation_timeout_seconds(i32) / set_invocation_timeout_seconds(Option<i32>):
    required: false
    

    Maximum amount of time in seconds a request can be processed before it is marked as expired. The default is 15 minutes, or 900 seconds.

    
    
• On success, responds with InvokeEndpointAsyncOutput with field(s):
  • inference_id(Option<String>):

    Identifier for an inference request. This will be the same as the InferenceId specified in the input. Amazon SageMaker will generate an identifier for you if you do not specify one.

  • output_location(Option<String>):

    The Amazon S3 URI where the inference response payload is stored.

  • failure_location(Option<String>):

    The Amazon S3 URI where the inference failure response payload is stored.

• On failure, responds with SdkError<InvokeEndpointAsyncError>

impl Client

pub fn invoke_endpoint_with_response_stream( &self, ) -> InvokeEndpointWithResponseStreamFluentBuilder

Constructs a fluent builder for the InvokeEndpointWithResponseStream operation.

• The fluent builder is configurable:
  • endpoint_name(impl Into<String>) / set_endpoint_name(Option<String>):
    required: true
    

    The name of the endpoint that you specified when you created the endpoint using the CreateEndpoint API.

    
    
  • body(Blob) / set_body(Option<Blob>):
    required: true
    

    Provides input data, in the format specified in the ContentType request header. Amazon SageMaker passes all of the data in the body to the model.

    For information about the format of the request body, see Common Data Formats-Inference.

    
    
  • content_type(impl Into<String>) / set_content_type(Option<String>):
    required: false
    

    The MIME type of the input data in the request body.

    
    
  • accept(impl Into<String>) / set_accept(Option<String>):
    required: false
    

    The desired MIME type of the inference response from the model container.

    
    
  • custom_attributes(impl Into<String>) / set_custom_attributes(Option<String>):
    required: false
    

    Provides additional information about a request for an inference submitted to a model hosted at an Amazon SageMaker endpoint. The information is an opaque value that is forwarded verbatim. You could use this value, for example, to provide an ID that you can use to track a request or to provide other metadata that a service endpoint was programmed to process. The value must consist of no more than 1024 visible US-ASCII characters as specified in Section 3.3.6. Field Value Components of the Hypertext Transfer Protocol (HTTP/1.1).

    The code in your model is responsible for setting or updating any custom attributes in the response. If your code does not set this value in the response, an empty value is returned. For example, if a custom attribute represents the trace ID, your model can prepend the custom attribute with Trace ID: in your post-processing function.

    This feature is currently supported in the Amazon Web Services SDKs but not in the Amazon SageMaker Python SDK.

    
    
  • target_variant(impl Into<String>) / set_target_variant(Option<String>):
    required: false
    

    Specify the production variant to send the inference request to when invoking an endpoint that is running two or more variants. Note that this parameter overrides the default behavior for the endpoint, which is to distribute the invocation traffic based on the variant weights.

    For information about how to use variant targeting to perform a/b testing, see Test models in production

    
    
  • target_container_hostname(impl Into<String>) / set_target_container_hostname(Option<String>):
    required: false
    

    If the endpoint hosts multiple containers and is configured to use direct invocation, this parameter specifies the host name of the container to invoke.

    
    
  • inference_id(impl Into<String>) / set_inference_id(Option<String>):
    required: false
    

    An identifier that you assign to your request.

    
    
  • inference_component_name(impl Into<String>) / set_inference_component_name(Option<String>):
    required: false
    

    If the endpoint hosts one or more inference components, this parameter specifies the name of inference component to invoke for a streaming response.

    
    
  • session_id(impl Into<String>) / set_session_id(Option<String>):
    required: false
    

    The ID of a stateful session to handle your request.

    You can’t create a stateful session by using the InvokeEndpointWithResponseStream action. Instead, you can create one by using the InvokeEndpoint action. In your request, you specify NEW_SESSION for the SessionId request parameter. The response to that request provides the session ID for the NewSessionId response parameter.

    
    
• On success, responds with InvokeEndpointWithResponseStreamOutput with field(s):
  • body(EventReceiver<ResponseStream, ResponseStreamError>):

    A stream of payload parts. Each part contains a portion of the response for a streaming inference request.

  • content_type(Option<String>):

    The MIME type of the inference returned from the model container.

  • invoked_production_variant(Option<String>):

    Identifies the production variant that was invoked.

  • custom_attributes(Option<String>):

    Provides additional information in the response about the inference returned by a model hosted at an Amazon SageMaker endpoint. The information is an opaque value that is forwarded verbatim. You could use this value, for example, to return an ID received in the CustomAttributes header of a request or other metadata that a service endpoint was programmed to produce. The value must consist of no more than 1024 visible US-ASCII characters as specified in Section 3.3.6. Field Value Components of the Hypertext Transfer Protocol (HTTP/1.1). If the customer wants the custom attribute returned, the model must set the custom attribute to be included on the way back.

    The code in your model is responsible for setting or updating any custom attributes in the response. If your code does not set this value in the response, an empty value is returned. For example, if a custom attribute represents the trace ID, your model can prepend the custom attribute with Trace ID: in your post-processing function.

    This feature is currently supported in the Amazon Web Services SDKs but not in the Amazon SageMaker Python SDK.

• On failure, responds with SdkError<InvokeEndpointWithResponseStreamError>

impl Client

pub fn from_conf(conf: Config) -> Self

Creates a new client from the service Config.

Panics

This method will panic in the following cases:

• Retries or timeouts are enabled without a sleep_impl configured.
• Identity caching is enabled without a sleep_impl and time_source configured.
• No behavior_version is provided.

The panic message for each of these will have instructions on how to resolve them.

pub fn config(&self) -> &Config

Returns the client’s configuration.

impl Client

pub fn new(sdk_config: &SdkConfig) -> Self

Creates a new client from an SDK Config.

Panics

• This method will panic if the sdk_config is missing an async sleep implementation. If you experience this panic, set the sleep_impl on the Config passed into this function to fix it.
• This method will panic if the sdk_config is missing an HTTP connector. If you experience this panic, set the http_connector on the Config passed into this function to fix it.
• This method will panic if no BehaviorVersion is provided. If you experience this panic, set behavior_version on the Config or enable the behavior-version-latest Cargo feature.

Trait Implementations

impl Clone for Client

fn clone(&self) -> Client

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Client

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.