Struct Fetcher

pub struct Fetcher<Data> { /* private fields */ }

An asynchronous file fetcher for clients fetching files.

The futures generated by the fetcher are compatible with single and multi-threaded runtimes, allowing you to choose between the runtime that works best for your application. A single-threaded runtime is generally recommended for fetching files, as your network connection is unlikely to be faster than a single CPU core.

Implementations

impl<Data> Fetcher<Data>

pub fn new(client: Client) -> Self

Constructs a new Fetcher.

impl<Data> Fetcher<Data>

pub fn connections_per_file(self, value: u16) -> Self

The number of concurrent connections to sustain per file being fetched.

Note

Defaults to 1 connection

pub fn delay_between_requests(self, value: u64) -> Self

Configure the delay between file requests.

Note

Defaults to no delay

pub fn retries(self, value: u16) -> Self

The number of attempts to make when a request fails.

Note

Defaults to 3 retries.

pub fn max_part_size(self, value: u32) -> Self

The maximum size of a part file when downloading in parts.

Note

Defaults to 2 MiB.

pub fn progress_interval(self, value: u64) -> Self

Time in ms between progress messages

Note

Defaults to 500.

pub fn timeout(self, value: Duration) -> Self

The time to wait between chunks before giving up.

pub fn events(self, value: impl Into<Arc<EventSender<Arc<Data>>>>) -> Self

Holds a sender for submitting events to.

pub fn shutdown(self, value: ShutdownManager<()>) -> Self

Utilized to know when to shut down the fetching process.

impl<Data: Send + Sync + 'static> Fetcher<Data>

pub fn build(self) -> Arc<Self>

Finalizes the fetcher to prepare it for fetch tasks.

pub fn stream_from( self: Arc<Self>, inputs: impl Stream<Item = (Source, Arc<Data>)> + Send + 'static, concurrent: usize, ) -> Pin<Box<dyn Stream<Item = AsyncFetchOutput<Data>> + Send + 'static>>

Given an input stream of source fetches, returns an output stream of fetch results.

Spawns up to concurrent + 1 number of concurrent async tasks on the runtime. One task for managing the fetch tasks, and one task per fetch request.

pub async fn request( self: Arc<Self>, uris: Arc<[Box<str>]>, to: Arc<Path>, extra: Arc<Data>, ) -> Result<(), Error>

Request a file from one or more URIs.

At least one URI must be provided as a source for the file. Each additional URI serves as a mirror for failover and load-balancing purposes.

Trait Implementations

impl<Data> Default for Fetcher<Data>

fn default() -> Self

Returns the “default value” for a type.