Struct Manager

pub struct Manager {
    pub region: String,
    pub cli: Client,
}

Implements AWS S3 manager.

Fields

region: String

cli: Client

Implementations

impl Manager

pub fn new(shared_config: &AwsSdkConfig) -> Self

pub async fn create_bucket(&self, s3_bucket: &str) -> Result<()>

Creates a S3 bucket.

pub async fn put_bucket_object_expire_configuration( &self, s3_bucket: &str, days_to_prefixes: HashMap<i32, Vec<String>>, ) -> Result<()>

Put object expire configuration on the bucket. ref. https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutBucketLifecycleConfiguration.html ref. https://docs.aws.amazon.com/AmazonS3/latest/API/API_LifecycleRule.html ref. https://docs.aws.amazon.com/AmazonS3/latest/API/API_LifecycleExpiration.html

pub async fn delete_bucket(&self, s3_bucket: &str) -> Result<()>

Deletes a S3 bucket.

pub async fn bucket_exists(&self, s3_bucket: &str) -> Result<bool>

Returns true if the bucket exists.

pub async fn delete_objects( &self, s3_bucket: &str, prefix: Option<&str>, ) -> Result<()>

Deletes objects by “prefix”. If “prefix” is “None”, empties a S3 bucket, deleting all files. ref. https://github.com/awslabs/aws-sdk-rust/blob/main/examples/s3/src/bin/delete-objects.rs

“If a single piece of data must be accessible from more than one task concurrently, then it must be shared using synchronization primitives such as Arc.” ref. https://tokio.rs/tokio/tutorial/spawning

pub async fn list_objects( &self, s3_bucket: &str, prefix: Option<&str>, ) -> Result<Vec<Object>>

List objects in the bucket with an optional prefix, in the descending order of “last_modified” timestamps. “bucket_name” implies the suffix “/”, so no need to prefix sub-directory with “/”. Passing “bucket_name” + “directory” is enough!

e.g. “foo-mydatabucket” for bucket_name “mydata/myprefix/” for prefix

pub async fn put_object( &self, file_path: &str, s3_bucket: &str, s3_key: &str, ) -> Result<()>

Writes an object to a S3 bucket using stream.

WARN: use stream! otherwise it can cause OOM – don’t do the following! “fs::read” reads all data onto memory “.body(ByteStream::from(contents))” passes the whole data to an API call

“If a single piece of data must be accessible from more than one task concurrently, then it must be shared using synchronization primitives such as Arc.” ref. https://tokio.rs/tokio/tutorial/spawning

ref. https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html

pub async fn put_object_with_retries( &self, file_path: &str, s3_bucket: &str, s3_key: &str, timeout: Duration, interval: Duration, ) -> Result<()>

pub async fn put_object_with_metadata( &self, file_path: &str, s3_bucket: &str, s3_key: &str, metadata: Option<HashMap<String, String>>, ) -> Result<()>

Writes an object with the metadata. ref. https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html ref. https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html

pub async fn put_object_with_metadata_with_retries( &self, file_path: &str, s3_bucket: &str, s3_key: &str, metadata: Option<HashMap<String, String>>, timeout: Duration, interval: Duration, ) -> Result<()>

pub async fn put_byte_stream_with_metadata( &self, byte_stream: ByteStream, s3_bucket: &str, s3_key: &str, metadata: Option<HashMap<String, String>>, ) -> Result<()>

Writes a byte stream with the metadata. ref. https://docs.aws.amazon.com/AmazonS3/latest/API/API_PutObject.html ref. https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingMetadata.html

pub async fn put_bytes_with_metadata_with_retries( &self, b: Vec<u8>, s3_bucket: &str, s3_key: &str, metadata: Option<HashMap<String, String>>, timeout: Duration, interval: Duration, ) -> Result<()>

pub async fn exists( &self, s3_bucket: &str, s3_key: &str, ) -> Result<Option<HeadObjectOutput>>

Returns “None” if the S3 file does not exist.

pub async fn exists_with_retries( &self, s3_bucket: &str, s3_key: &str, timeout: Duration, interval: Duration, ) -> Result<Option<HeadObjectOutput>>

pub async fn get_object( &self, s3_bucket: &str, s3_key: &str, file_path: &str, overwrite: bool, ) -> Result<bool>

Downloads an object from a S3 bucket using stream.

WARN: use stream! otherwise it can cause OOM – don’t do the following! “aws_smithy_http::byte_stream:ByteStream.collect” reads all the data into memory “File.write_all_buf(&mut bytes)” to write bytes

“If a single piece of data must be accessible from more than one task concurrently, then it must be shared using synchronization primitives such as Arc.” ref. https://tokio.rs/tokio/tutorial/spawning

Returns “true” if the file got downloaded successfully. Returns “false” if the S3 file does not exist.

pub async fn get_object_with_retries( &self, s3_bucket: &str, s3_key: &str, file_path: &str, overwrite: bool, timeout: Duration, interval: Duration, ) -> Result<bool>

Downloads an object from a S3 bucket using stream.

WARN: use stream! otherwise it can cause OOM – don’t do the following! “aws_smithy_http::byte_stream:ByteStream.collect” reads all the data into memory “File.write_all_buf(&mut bytes)” to write bytes

“If a single piece of data must be accessible from more than one task concurrently, then it must be shared using synchronization primitives such as Arc.” ref. https://tokio.rs/tokio/tutorial/spawning

Returns “true” if the file exists and got downloaded successfully.

pub async fn download_executable_with_retries( &self, s3_bucket: &str, source_s3_path: &str, target_file_path: &str, overwrite: bool, timeout: Duration, interval: Duration, ) -> Result<bool>

Returns “true” if successfully downloaded, or skipped to not overwrite. Returns “false” if not exists.

Trait Implementations

impl Clone for Manager

fn clone(&self) -> Manager

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Manager

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.