Crate async_fetcher
source ·Expand description
Asynchronously fetch files from HTTP servers
- Concurrently fetch multiple files at the same time.
- Define alternative mirrors for the source of a file.
- Use multiple concurrent connections per file.
- Use mirrors for concurrent connections.
- Resume a download which has been interrupted.
- Progress events for fetches
let (events_tx, events_rx) = tokio::sync::mpsc::unbounded_channel();
let shutdown = async_shutdown::ShutdownManager::new();
let results_stream = Fetcher::default()
// Define a max number of ranged connections per file.
.connections_per_file(4)
// Max size of a connection's part, concatenated on completion.
.max_part_size(4 * 1024 * 1024)
// The channel for sending progress notifications.
.events(events_tx)
// Maximum number of retry attempts.
.retries(3)
// Cancels the fetching process when a shutdown is triggered.
.shutdown(shutdown)
// How long to wait before aborting a download that hasn't progressed.
.timeout(Duration::from_secs(15))
// Finalize the struct into an `Arc` for use with fetching.
.build()
// Take a stream of `Source` inputs and generate a stream of fetches.
// Spawns
.stream_from(input_stream, 4)
Modules§
Structs§
- An asynchronous file fetcher for clients fetching files.
- Information about a source being fetched.
- Constructs a
Source
.
Enums§
- A checksum of a
Source
as a fixed-sized byte array. - An error that can occur from a failed checksum validation.
- The underlying Client used for the Fetcher
- An error from the asynchronous file fetcher.
- Events which are submitted by the fetcher.
- The
&str
representation of aChecksum
. - The
String
representation of aChecksum
.
Functions§
- Generates a stream of futures that validate checksums.
- Accepts a stream of future file
parts
and concatenates them into thedest
file. - Validates the checksum of a single file
Type Aliases§
- The result of a fetched task from a stream of input sources.
- A channel for sending
FetchEvent
s to.