Crate async_fetcher

Asynchronously fetch files from HTTP servers

• Concurrently fetch multiple files at the same time.
• Define alternative mirrors for the source of a file.
• Use multiple concurrent connections per file.
• Use mirrors for concurrent connections.
• Resume a download which has been interrupted.
• Progress events for fetches

let (events_tx, events_rx) = tokio::sync::mpsc::unbounded_channel();

let shutdown = async_shutdown::ShutdownManager::new();

let results_stream = Fetcher::default()
    // Define a max number of ranged connections per file.
    .connections_per_file(4)
    // Max size of a connection's part, concatenated on completion.
    .max_part_size(4 * 1024 * 1024)
    // The channel for sending progress notifications.
    .events(events_tx)
    // Maximum number of retry attempts.
    .retries(3)
    // Cancels the fetching process when a shutdown is triggered.
    .shutdown(shutdown)
    // How long to wait before aborting a download that hasn't progressed.
    .timeout(Duration::from_secs(15))
    // Finalize the struct into an `Arc` for use with fetching.
    .build()
    // Take a stream of `Source` inputs and generate a stream of fetches.
    // Spawns
    .stream_from(input_stream, 4)

Modules

• iface

Structs

• Fetcher
  An asynchronous file fetcher for clients fetching files.
• Source
  Information about a source being fetched.
• SourceBuilder
  Constructs a Source.

Enums

• Checksum
  A checksum of a Source as a fixed-sized byte array.
• ChecksumError
  An error that can occur from a failed checksum validation.
• Client
  The underlying Client used for the Fetcher
• Error
  An error from the asynchronous file fetcher.
• FetchEvent
  Events which are submitted by the fetcher.
• SumStr
  The &str representation of a Checksum.
• SumStrBuf
  The String representation of a Checksum.

Functions

• checksum_stream
  Generates a stream of futures that validate checksums.
• concatenator
  Accepts a stream of future file parts and concatenates them into the dest file.
• validate_checksum
  Validates the checksum of a single file

Type Aliases

• AsyncFetchOutput
  The result of a fetched task from a stream of input sources.
• EventSender
  A channel for sending FetchEvents to.