Struct deltalake::storage::object_store::aws::AmazonS3Builder

pub struct AmazonS3Builder { /* private fields */ }

Configure a connection to Amazon S3 using the specified credentials in the specified Amazon region and bucket.

Example

let s3 = AmazonS3Builder::new()
 .with_region(REGION)
 .with_bucket_name(BUCKET_NAME)
 .with_access_key_id(ACCESS_KEY_ID)
 .with_secret_access_key(SECRET_KEY)
 .build();

Implementations

impl AmazonS3Builder

pub fn new() -> AmazonS3Builder

Create a new AmazonS3Builder with default values.

pub fn from_env() -> AmazonS3Builder

Fill the AmazonS3Builder with regular AWS environment variables

Variables extracted from environment:

• AWS_ACCESS_KEY_ID -> access_key_id
• AWS_SECRET_ACCESS_KEY -> secret_access_key
• AWS_DEFAULT_REGION -> region
• AWS_ENDPOINT -> endpoint
• AWS_SESSION_TOKEN -> token
• AWS_CONTAINER_CREDENTIALS_RELATIVE_URI -> https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html
• AWS_ALLOW_HTTP -> set to “true” to permit HTTP connections without TLS

Example

use object_store::aws::AmazonS3Builder;

let s3 = AmazonS3Builder::from_env()
    .with_bucket_name("foo")
    .build();

pub fn with_url(self, url: impl Into<String>) -> AmazonS3Builder

Parse available connection info form a well-known storage URL.

The supported url schemes are:

• s3://<bucket>/<path>
• s3a://<bucket>/<path>
• https://s3.<region>.amazonaws.com/<bucket>
• https://<bucket>.s3.<region>.amazonaws.com
• https://ACCOUNT_ID.r2.cloudflarestorage.com/bucket

Note: Settings derived from the URL will override any others set on this builder

Example

use object_store::aws::AmazonS3Builder;

let s3 = AmazonS3Builder::from_env()
    .with_url("s3://bucket/path")
    .build();

pub fn with_config( self, key: AmazonS3ConfigKey, value: impl Into<String> ) -> AmazonS3Builder

Set an option on the builder via a key - value pair.

pub fn get_config_value(&self, key: &AmazonS3ConfigKey) -> Option<String>

Get config value via a AmazonS3ConfigKey.

Example

use object_store::aws::{AmazonS3Builder, AmazonS3ConfigKey};

let builder = AmazonS3Builder::from_env()
    .with_bucket_name("foo");
let bucket_name = builder.get_config_value(&AmazonS3ConfigKey::Bucket).unwrap_or_default();
assert_eq!("foo", &bucket_name);

pub fn with_access_key_id( self, access_key_id: impl Into<String> ) -> AmazonS3Builder

Set the AWS Access Key

pub fn with_secret_access_key( self, secret_access_key: impl Into<String> ) -> AmazonS3Builder

Set the AWS Secret Access Key

pub fn with_token(self, token: impl Into<String>) -> AmazonS3Builder

Set the AWS Session Token to use for requests

pub fn with_region(self, region: impl Into<String>) -> AmazonS3Builder

Set the region, defaults to us-east-1

pub fn with_bucket_name(self, bucket_name: impl Into<String>) -> AmazonS3Builder

Set the bucket_name (required)

pub fn with_endpoint(self, endpoint: impl Into<String>) -> AmazonS3Builder

Sets the endpoint for communicating with AWS S3, defaults to the region endpoint

For example, this might be set to "http://localhost:4566: for testing against a localstack instance.

The endpoint field should be consistent with Self::with_virtual_hosted_style_request, i.e. if virtual_hosted_style_request is set to true then endpoint should have the bucket name included.

By default, only HTTPS schemes are enabled. To connect to an HTTP endpoint, enable Self::with_allow_http.

pub fn with_credentials( self, credentials: Arc<dyn CredentialProvider<Credential = AwsCredential>> ) -> AmazonS3Builder

Set the credential provider overriding any other options

pub fn with_allow_http(self, allow_http: bool) -> AmazonS3Builder

Sets what protocol is allowed. If allow_http is :

• false (default): Only HTTPS are allowed
• true: HTTP and HTTPS are allowed

pub fn with_virtual_hosted_style_request( self, virtual_hosted_style_request: bool ) -> AmazonS3Builder

Sets if virtual hosted style request has to be used.

If virtual_hosted_style_request is:

• false (default): Path style request is used
• true: Virtual hosted style request is used

If the endpoint is provided then it should be consistent with virtual_hosted_style_request. i.e. if virtual_hosted_style_request is set to true then endpoint should have bucket name included.

pub fn with_s3_express(self, s3_express: bool) -> AmazonS3Builder

Configure this as an S3 Express One Zone Bucket

pub fn with_retry(self, retry_config: RetryConfig) -> AmazonS3Builder

Set the retry configuration

pub fn with_imdsv1_fallback(self) -> AmazonS3Builder

By default instance credentials will only be fetched over IMDSv2, as AWS recommends against having IMDSv1 enabled on EC2 instances as it is vulnerable to SSRF attack

However, certain deployment environments, such as those running old versions of kube2iam, may not support IMDSv2. This option will enable automatic fallback to using IMDSv1 if the token endpoint returns a 403 error indicating that IMDSv2 is not supported.

This option has no effect if not using instance credentials

pub fn with_unsigned_payload(self, unsigned_payload: bool) -> AmazonS3Builder

Sets if unsigned payload option has to be used. See unsigned payload option

• false (default): Signed payload option is used, where the checksum for the request body is computed and included when constructing a canonical request.
• true: Unsigned payload option is used. UNSIGNED-PAYLOAD literal is included when constructing a canonical request,

pub fn with_skip_signature(self, skip_signature: bool) -> AmazonS3Builder

If enabled, AmazonS3 will not fetch credentials and will not sign requests

This can be useful when interacting with public S3 buckets that deny authorized requests

pub fn with_checksum_algorithm( self, checksum_algorithm: Checksum ) -> AmazonS3Builder

Sets the checksum algorithm which has to be used for object integrity check during upload.

pub fn with_metadata_endpoint( self, endpoint: impl Into<String> ) -> AmazonS3Builder

Set the instance metadata endpoint, used primarily within AWS EC2.

This defaults to the IPv4 endpoint: http://169.254.169.254. One can alternatively use the IPv6 endpoint http://fd00:ec2::254.

pub fn with_proxy_url(self, proxy_url: impl Into<String>) -> AmazonS3Builder

Set the proxy_url to be used by the underlying client

pub fn with_proxy_ca_certificate( self, proxy_ca_certificate: impl Into<String> ) -> AmazonS3Builder

Set a trusted proxy CA certificate

pub fn with_proxy_excludes( self, proxy_excludes: impl Into<String> ) -> AmazonS3Builder

Set a list of hosts to exclude from proxy connections

pub fn with_client_options(self, options: ClientOptions) -> AmazonS3Builder

Sets the client options, overriding any already set

pub fn with_copy_if_not_exists( self, config: S3CopyIfNotExists ) -> AmazonS3Builder

Configure how to provide copy_if_not_exists

pub fn with_conditional_put(self, config: S3ConditionalPut) -> AmazonS3Builder

Configure how to provide conditional put operations

pub fn with_disable_tagging(self, ignore: bool) -> AmazonS3Builder

If set to true will ignore any tags provided to put_opts

pub fn with_sse_kms_encryption( self, kms_key_id: impl Into<String> ) -> AmazonS3Builder

Use SSE-KMS for server side encryption.

pub fn with_dsse_kms_encryption( self, kms_key_id: impl Into<String> ) -> AmazonS3Builder

Use dual server side encryption for server side encryption.

pub fn with_bucket_key(self, enabled: bool) -> AmazonS3Builder

Set whether to enable bucket key for server side encryption. This overrides the bucket default setting for bucket keys.

When bucket keys are disabled, each object is encrypted with a unique data key. When bucket keys are enabled, a single data key is used for the entire bucket, reducing overhead of encryption.

pub fn build(self) -> Result<AmazonS3, Error>

Create a AmazonS3 instance from the provided values, consuming self.

Trait Implementations

impl Clone for AmazonS3Builder

fn clone(&self) -> AmazonS3Builder

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AmazonS3Builder

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter.

impl Default for AmazonS3Builder

fn default() -> AmazonS3Builder

Returns the “default value” for a type.