Crate taskcluster_download

Support for downloading data from the object service.

This crate provides a set of functions to download data from the object service. These functions negotiate a download method with the object service, then perform the download, following all of the Taskcluster recommended practices.

Each function takes the necessary metadata for the download, a handle to the a destination for the data, and a taskcluster::Object client. The destination can take a variety of forms, as described below. The client must be configured with the necessary credentials to access the object service.

Convenience Functions

Most uses of this crate can utilize one of the following convenience functions:

• download_to_buf – download data to a fixed-size buffer;
• download_to_vec – download data to a dynamically allocated buffer; or
• download_to_file – download data to a tokio::fs::File.

Artifacts

This crate also supports downloads of artifacts, including all defined storage types. This combines a query to the queue service to request the artifact metadata with the appropriate process to download that data. In the case of error artifacts, a Result::Err is returned.

• download_artifact_to_buf – download artifact data to a fixed-size buffer;
• download_artifact_to_vec – download artifact data to a dynamically allocated buffer; or
• download_artifact_to_file – download artifact data to a tokio::fs::File.

Factories

A download may be retried, in which case the download function must have a means to truncate the data destination and begin writing from the beginning. This is accomplished with the AsyncWriterFactory trait, which defines a get_writer method to generate a fresh tokio::io::AsyncWrite for each attempt. Users for whom the supplied convenience functions are inadequate can add their own implementation of this trait.

Structs

CursorWriterFactory

A CusorWriterFactory creates AsyncWrite objects from a std::io::Cursor, allowing downloads to in-memory buffers. It is specialized for Vec (which grows indefinitely) and &mut [u8] (which has a fixed maximum size)

FileWriterFactory

A FileWriterFactory creates AsyncWrite objects by rewinding and cloning a tokio::fs::File. The file must be open in write mode and must be clone-able (that is, File::try_clone() must succeed) in order to support retried uploads.

Traits

AsyncWriterFactory

An AsyncWriterFactory can produce, on demand, an AsyncWrite object. In the event of a download failure, the restarted download will use a fresh writer to restart writing at the beginning.

Functions

download_artifact_to_buf

Download an artifact into the given buffer and return the slice of that buffer containing the artifact. If the artifact is larger than the buffer, then resulting error can be downcast to std::io::Error with kind WriteZero and the somewhat cryptic message “write zero byte into writer”. Returns (slice, content_type). If run_id is None then the latest run will be used.

download_artifact_to_file

Download an artifact into the given File. The file must be open in write mode and must be clone-able (that is, File::try_clone() must succeed) in order to support retried downloads. The File is returned with all write operations complete but with unspecified position. If run_id is None then the latest run will be used. Returns (file, content_type).

download_artifact_to_vec

Download an artifact to a Vec and return that. If the artifact is unexpectedly large, this may exhaust system memory and panic. If run_id is None then the latest run will be used. Returns (data, content_type).

download_artifact_with_factory

Download an artifact using an AsyncWriterFactory. This is useful for advanced cases where one of the convenience functions is not adequate. Returns the artifact’s content type. If run_id is None then the latest run will be used. Returns the content type.

download_to_buf

Download an object into the given buffer and return the slice of that buffer containing the object. If the object is larger than the buffer, then resulting error can be downcast to std::io::Error with kind WriteZero and the somewhat cryptic message “write zero byte into writer”. Returns (slice, content_type)

download_to_file

Download an object into the given File. The file must be open in write mode and must be clone-able (that is, File::try_clone() must succeed) in order to support retried downloads. The File is returned with all write operations complete but with unspecified position. Returns (file, content_type).

download_to_vec

Download an object to a Vec and return that. If the object is unexpectedly large, this may exhaust system memory and panic. Returns (data, content_type)

download_with_factory

Download an object using an AsyncWriterFactory. This is useful for advanced cases where one of the convenience functions is not adequate. Returns the object’s content type.