Struct S3ObjectBlockClient 

pub struct S3ObjectBlockClient { /* private fields */ }

S3-compatible object storage client for block operations.

This client implements ObjectBlockOps using the AWS S3 SDK. It supports parallel block operations and uses rayon for CPU-bound memory copies.

Key Formatting

Uses a KeyFormatter to convert SequenceHash to object keys. The formatter can embed rank, namespace, or other prefixes for key uniqueness across workers.

Implementations

impl S3ObjectBlockClient

pub async fn new(config: S3Config) -> Result<Self>

Create a new S3ObjectBlockClient with default key formatting.

Arguments

• config - S3 configuration

Errors

Returns an error if the S3 client cannot be initialized.

pub async fn with_key_formatter( config: S3Config, key_formatter: Arc<dyn KeyFormatter>, ) -> Result<Self>

Create a new S3ObjectBlockClient with a custom key formatter.

Arguments

• config - S3 configuration
• key_formatter - Custom key formatter for SequenceHash → String conversion

Errors

Returns an error if the S3 client cannot be initialized.

pub fn from_client(client: Client, config: S3Config) -> Self

Create from an existing AWS S3 client with default key formatting.

pub fn from_client_with_formatter( client: Client, config: S3Config, key_formatter: Arc<dyn KeyFormatter>, ) -> Self

Create from an existing AWS S3 client with a custom key formatter.

pub fn client(&self) -> &Client

Get a reference to the S3 client.

pub fn config(&self) -> &S3Config

Get a reference to the configuration.

pub fn key_formatter(&self) -> &Arc<dyn KeyFormatter>

Get a reference to the key formatter.

pub fn bucket(&self) -> &str

Get a reference to the bucket name.

pub async fn ensure_bucket_exists(&self) -> Result<()>

Ensure the bucket exists, creating it if necessary.

pub async fn put_if_not_exists(&self, key: &str, data: Bytes) -> Result<bool>

Put an object with a conditional check (If-None-Match: *).

This performs an atomic write that only succeeds if the object does not already exist. Returns:

• Ok(true) if the object was created successfully
• Ok(false) if the object already exists (PreconditionFailed)
• Err(...) for other errors

Arguments

• key - Object key
• data - Object data to write

pub async fn get_object(&self, key: &str) -> Result<Option<Bytes>>

Get an object’s raw bytes.

Arguments

• key - Object key

Returns

• Ok(Some(bytes)) if the object exists
• Ok(None) if the object does not exist
• Err(...) for other errors

pub async fn delete_object(&self, key: &str) -> Result<bool>

Delete an object.

Arguments

• key - Object key

Returns

• Ok(true) if the object was deleted
• Ok(false) if the object did not exist
• Err(...) for other errors

pub async fn has_object(&self, key: &str) -> Result<bool>

Check if an object exists (HEAD request).

Arguments

• key - Object key

Returns

• Ok(true) if the object exists
• Ok(false) if the object does not exist
• Err(...) for other errors

pub async fn put_object(&self, key: &str, data: Bytes) -> Result<()>

Put an object unconditionally (overwrite if exists).

Arguments

• key - Object key
• data - Object data to write

pub fn put_blocks_with_layout( &self, keys: Vec<SequenceHash>, layout: PhysicalLayout, block_ids: Vec<BlockId>, ) -> BoxFuture<'static, Vec<Result<SequenceHash, SequenceHash>>>

Put blocks to object storage using a physical layout.

This is the internal implementation that workers call after resolving the logical layout handle to a physical layout.

Arguments

• keys - Sequence hashes identifying each block
• layout - Physical layout containing the block data
• block_ids - Block IDs within the layout to upload

Returns a vector of results for each block:

• Ok(hash) indicates the block was successfully stored
• Err(hash) indicates the block failed to store

pub fn get_blocks_with_layout( &self, keys: Vec<SequenceHash>, layout: PhysicalLayout, block_ids: Vec<BlockId>, ) -> BoxFuture<'static, Vec<Result<SequenceHash, SequenceHash>>>

Get blocks from object storage into a physical layout.

This is the internal implementation that workers call after resolving the logical layout handle to a physical layout.

Arguments

• keys - Sequence hashes identifying each block
• layout - Physical layout to write the block data into
• block_ids - Block IDs within the layout to download into

Returns a vector of results for each block:

• Ok(hash) indicates the block was successfully retrieved
• Err(hash) indicates the block failed to retrieve

pub async fn get_object_with_etag( &self, key: &str, ) -> Result<Option<(Bytes, Option<String>)>>

Get an object’s raw bytes along with its ETag.

Used for conditional updates (CAS-style operations) where the caller needs the current ETag to perform a conditional PUT.

Returns

• Ok(Some((bytes, etag))) if the object exists
• Ok(None) if the object does not exist
• Err(...) for other errors

pub async fn put_object_if_match( &self, key: &str, data: Bytes, etag: &str, ) -> Result<bool>

Put an object with an ETag precondition (If-Match).

This performs a conditional write that only succeeds if the object’s current ETag matches the provided value. Used for CAS-style atomic updates.

Returns

• Ok(true) if the write succeeded (ETag matched)
• Ok(false) if the ETag did not match (412 PreconditionFailed — lost the race)
• Err(...) for other errors

Trait Implementations

impl ObjectBlockOps for S3ObjectBlockClient

fn has_blocks( &self, keys: Vec<SequenceHash>, ) -> BoxFuture<'static, Vec<(SequenceHash, Option<usize>)>>

Check if blocks exist in object storage.

fn put_blocks( &self, keys: Vec<SequenceHash>, _src_layout: LogicalLayoutHandle, _block_ids: Vec<BlockId>, ) -> BoxFuture<'static, Vec<Result<SequenceHash, SequenceHash>>>

Put blocks to object storage.

fn get_blocks( &self, keys: Vec<SequenceHash>, _dst_layout: LogicalLayoutHandle, _block_ids: Vec<BlockId>, ) -> BoxFuture<'static, Vec<Result<SequenceHash, SequenceHash>>>

Get blocks from object storage.

fn put_blocks_with_layout( &self, keys: Vec<SequenceHash>, layout: PhysicalLayout, block_ids: Vec<BlockId>, ) -> BoxFuture<'static, Vec<Result<SequenceHash, SequenceHash>>>

Put blocks to object storage using a resolved physical layout.

fn get_blocks_with_layout( &self, keys: Vec<SequenceHash>, layout: PhysicalLayout, block_ids: Vec<BlockId>, ) -> BoxFuture<'static, Vec<Result<SequenceHash, SequenceHash>>>

Get blocks from object storage into a resolved physical layout.