Trait CreateProgressCallback

pub trait CreateProgressCallback: Sync + Send {
    // Provided methods
    fn input_objects_download_starting(
        &self,
        total_objects: usize,
        total_bytes: u64,
    ) { ... }
    fn input_object_download_started(
        &self,
        bucket: &str,
        key: &str,
        version_id: Option<&str>,
        size: u64,
    ) { ... }
    fn input_part_unordered_downloaded(
        &self,
        bucket: &str,
        key: &str,
        version_id: Option<&str>,
        part_number: usize,
        part_size: usize,
    ) { ... }
    fn input_part_downloaded(
        &self,
        bucket: &str,
        key: &str,
        version_id: Option<&str>,
        part_number: usize,
        part_size: usize,
    ) { ... }
    fn input_object_download_completed(
        &self,
        bucket: &str,
        key: &str,
        version_id: Option<&str>,
        size: u64,
    ) { ... }
    fn input_objects_download_completed(
        &self,
        total_bytes: u64,
        duration: Duration,
    ) { ... }
    fn archive_initialized(
        &self,
        total_objects: usize,
        total_bytes: u64,
        estimated_archive_size: u64,
    ) { ... }
    fn archive_part_written(
        &self,
        bucket: &str,
        key: &str,
        version_id: Option<&str>,
        part_number: usize,
        part_size: usize,
    ) { ... }
    fn archive_object_written(
        &self,
        bucket: &str,
        key: &str,
        version_id: Option<&str>,
        timestamp: DateTime<Utc>,
        byte_offset: u64,
        size: u64,
    ) { ... }
    fn archive_bytes_written(&self, bytes_written: usize) { ... }
    fn archive_writes_completed(&self, total_bytes_written: u64) { ... }
    fn archive_bytes_uploaded(&self, bytes_uploaded: usize) { ... }
    fn archive_upload_completed(&self, size: u64, duration: Duration) { ... }
}

A trait which callers can implement to get detailed progress updates as archive creation is progressing.

Provided Methods

fn input_objects_download_starting( &self, total_objects: usize, total_bytes: u64, )

The process of downloading all of the input objects is about to start

fn input_object_download_started( &self, bucket: &str, key: &str, version_id: Option<&str>, size: u64, )

The download of a new input object is starting.

In truth downloads happen in parallel, but they are yielded in precisely the order they will appear in the tar archive, so for progress reporting purposes when the first part of an input object is available for writing to the tar archive, we say the download of that object has started.

fn input_part_unordered_downloaded( &self, bucket: &str, key: &str, version_id: Option<&str>, part_number: usize, part_size: usize, )

Part of the data of one of the input objects was downloaded from object storage.

Unlike the started and completed events, which are generated from an ordered sequence of input object part downloads, this event is reported as soon as the bytes come down the wire for a part. Because of how concurrent tasks are executed, it’s possible that part 10 of an object downloads first, before part 0. This event fires as soon as part 10 has finished downloading, but the started event won’t fire until part 0 finishes.

For an event that is guaranteed to fire after started and before completed, see input_part_downloaded

fn input_part_downloaded( &self, bucket: &str, key: &str, version_id: Option<&str>, part_number: usize, part_size: usize, )

Part of the data of one of the input objects was downloaded from object storage.

unlike input_part_unordered_downloaded, this event doesn’t fire the moment the part is downloaded over the wire, but only when that part becomes available on the stream of input object parts in order. For rendering a progress bar of the progress of downloading an object, this is what you want, but it might be the case that this part actually was downloaded into memory several seconds ago, or possibly longer for slow networks.

fn input_object_download_completed( &self, bucket: &str, key: &str, version_id: Option<&str>, size: u64, )

An entire input object has been downloaded successfully

fn input_objects_download_completed(&self, total_bytes: u64, duration: Duration)

All input objects have now been downloaded.

That doesn’t mean the work is done; there can still be ongoing tasks either writing some of that downloaded data to the tar builder, or uploading writes to the tar archive to object storage.

fn archive_initialized( &self, total_objects: usize, total_bytes: u64, estimated_archive_size: u64, )

The tar archive has been initialized but not yet written to.

The total_* args refer to the number and total size of all input objects which are to be written to this archive. It’s hard to predict the actual tar archive size without getting into the weeds of the tar archive format, but an estimated size is provided to help scale progress bars in the UI.

fn archive_part_written( &self, bucket: &str, key: &str, version_id: Option<&str>, part_number: usize, part_size: usize, )

Part of the data of one of the input objects has been written to the tar archive

That doesn’t mean the data written has been uploaded to remote object storage yet, it could still be buffered locally.

fn archive_object_written( &self, bucket: &str, key: &str, version_id: Option<&str>, timestamp: DateTime<Utc>, byte_offset: u64, size: u64, )

An entire input object was written successfully to a tar archive.

byte_offset is the byte offset in the archive stream where the data for this object starts. Data is guaranteed to be stored in one contiguous sequence, therefore it’s possible to find the data for this object from the byte_offset and size parameters.

That doesn’t mean the data written has been uploaded to remote object storage yet, it could still be buffered locally.

fn archive_bytes_written(&self, bytes_written: usize)

The tar builder has written some bytes to the std::io::Write instance that it’s layered on top of.

This event happens after a tar_archive_part_written event and sees the total number of bytes written, including any tar headers.

Notes

This event can be reported from a synchronous context, because it’s captured at the level of the std::io::Write implementation itself.

fn archive_writes_completed(&self, total_bytes_written: u64)

The tar archive has been completed, and will see no further writes.

There may still be upload activity in process, uploading previous tar writes to object storage from a local buffer.

fn archive_bytes_uploaded(&self, bytes_uploaded: usize)

Some bytes have been uploaded to the tar archive in object storage.

bytes_uploaded is not a total; it’s the amount of bytes uploaded just now by the caller. The receiver of this event will need to maintain a running total if one is desired.

If the tar archive is not being directed to object storage, then this event will never fire

fn archive_upload_completed(&self, size: u64, duration: Duration)

The tar archive’s previously completed writes have all been flushed from their buffers and uploaded to object storage (or a file or a stream dependng on where the tar archive is located).

This is the final event that can happen. Once this event fires, the job is done.

If the tar archive is not being directed to object storage, then this event will never fire

Implementors